CMD: A Cross Mechanism Domain Adaptation Dataset for 3D Object Detection
Authors: 
Jinhao Deng, Wei Ye, Hai Wu, Xun Huang, Qiming Xia, Xin Li, Jin Fang, Wei Li, Chenglu Wen, Cheng WangAuthors Info & Claims
Pages 219 - 236
Abstract
Point cloud data, representing the precise 3D layout of the scene, quickly drives the research of 3D object detection. However, the challenge arises due to the rapid iteration of 3D sensors, which leads to significantly different distributions in point clouds. This, in turn, results in subpar performance of 3D cross-sensor object detection. This paper introduces a Cross Mechanism Dataset, named CMD, to support research tackling this challenge. CMD is the first domain adaptation dataset, comprehensively encompassing diverse mechanical sensors and various scenes for 3D object detection. In terms of sensors, CMD includes 32-beam LiDAR, 128-beam LiDAR, solid-state LiDAR, 4D millimeter-wave radar, and cameras, all of which are well-synchronized and calibrated. Regarding the scenes, CMD consists of 50 sequences collocated from different scenarios, ranging from campuses to highways. Furthermore, we validated the effectiveness of various domain adaptation methods in mitigating sensor-based domain differences. We also proposed a DIG method to reduce domain disparities from the perspectives of Density, Intensity, and Geometry, which effectively bridges the domain gap between different sensors. The experimental results on the CMD dataset show that our proposed DIG method outperforms the state-of-the-art techniques, demonstrating the effectiveness of our baseline method. The dataset and the corresponding code are available at https://github.com/im-djh/CMD.
References
[1]
IEEE standard for a precision clock synchronization protocol for networked measurement and control systems. IEEE Std 1588-2008 (Revision of IEEE Std 1588-2002), pp. 1–269 (2008).
[2]
Atkinson, A.C., Riani, M., Corbellini, A.: The box–cox transformation: review and extensions (2021)
[3]
Bai, X., et al.: TransFusion: robust LiDAR-camera fusion for 3D object detection with transformers. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
[4]
Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools Prof. Program. 25(11), 120–123 (2000)
[5]
Caesar, H., et al.: nuScenes: a multimodal dataset for autonomous driving. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020).
[6]
Chang, M.F., et al.: Argoverse: 3D tracking and forecasting with rich maps. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8748–8757 (2019)
[7]
Chen, Y., et al.: FocalFormer3D: focusing on hard instance for 3D object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8394–8405 (2023)
[8]
Chen, Y., Liu, J., Zhang, X., Qi, X., Jia, J.: VoxelNeXt: fully sparse VoxelNet for 3D object detection and tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21674–21683 (2023)
[9]
Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., Li, H.: Voxel R-CNN: towards high performance voxel-based 3D object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence (2021)
[10]
Ding, G., Zhang, M., Li, E., Hao, Q.: JST: joint self-training for unsupervised domain adaptation on 2D &3D object detection. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 477–483. IEEE (2022)
[11]
Fang, J., Zhou, D., Zhao, J., Tang, C., Xu, C.Z., Zhang, L.: LiDAR-CS dataset: LiDAR point cloud dataset with cross-sensors for 3D object detection (2023)
[12]
Fetić, A., Jurić, D., Osmanković, D.: The procedure of a camera calibration using camera calibration toolbox for MATLAB. In: 2012 Proceedings of the 35th International Convention MIPRO, pp. 1752–1757. IEEE (2012)
[13]
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (2012).
[14]
Hegde, D., Sindagi, V., Kilic, V., Cooper, A.B., Foster, M., Patel, V.: Uncertainty-aware mean teacher for source-free unsupervised domain adaptive 3D object detection. arXiv preprint arXiv:2109.14651 (2021)
[15]
Houston, J., et al.: One thousand and one hours: self-driving motion prediction dataset. In: Conference on Robot Learning, pp. 409–418. PMLR (2021)
[16]
Hu, Q., Liu, D., Hu, W.: Density-insensitive unsupervised domain adaption on 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17556–17566 (2023)
[17]
Huang, X., Wu, H., Li, X., Fan, X., Wen, C., Wang, C.: Sunshine to rainstorm: cross-weather knowledge distillation for robust 3D object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 2409–2416 (2024)
[18]
Liu, Z., et al.: BEVFusion: multi-task multi-sensor fusion with unified bird’s-eye view representation. ArXiv (2022)
[19]
Luo, Z., et al.: Unsupervised domain adaptive 3D detection with multi-level consistency. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8866–8875 (2021)
[20]
MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, CA, USA, vol. 1, pp. 281–297 (1967)
[21]
Mao, J., et al.: One million scenes for autonomous driving: once dataset. Cornell University - arXiv (2021)
[22]
Paek, D.H., Kong, S.H., Wijaya, K.: K-Radar: 4D radar object detection dataset and benchmark for autonomous driving in various weather conditions (2022)
[23]
Palffy A, Pool E, Baratam S, Kooij J, and Gavrila D Multi-class road user detection with 3+1D radar in the view-of-delft dataset IEEE Robot. Autom. Lett. 2022 7 2 4961-4968
[24]
Peng, X., Zhu, X., Ma, Y.: CL3D: unsupervised domain adaptation for cross-LiDAR 3D detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 2047–2055 (2023)
[25]
Raj T, Hanim Hashim F, Baseri Huddin A, Ibrahim MF, and Hussain A A survey on LiDAR scanning mechanisms Electronics 2020 9 5 741
[26]
Rochan, M., Chen, X., Grandhi, A., Corral-Soto, E.R., Liu, B.: Domain adaptation in 3D object detection with gradual batch alternation training. arXiv preprint arXiv:2210.10180 (2022)
[27]
Roriz R, Cabral J, and Gomes T Automotive LiDAR technology: a survey IEEE Trans. Intell. Transp. Syst. 2021 23 7 6282-6297
[28]
Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: Robotics: Science and Systems, Seattle, WA, vol. 2, p. 435 (2009)
[29]
Shi, S., et al.: PV-RCNN: point-voxel feature set abstraction for 3D object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10526–10535 (2020)
[30]
Sun, P., et al.: Scalability in perception for autonomous driving: waymo open dataset. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020).
[31]
OpenPCDet Development Team: OpenPCDet: an open-source toolbox for 3D object detection from point clouds (2020). https://github.com/open-mmlab/OpenPCDet
[32]
Tsai, D., Berrio, J.S., Shan, M., Nebot, E., Worrall, S.: MS3D: leveraging multiple detectors for unsupervised domain adaptation in 3D object detection. arXiv preprint arXiv:2304.02431 (2023)
[33]
Tsai, D., Berrio, J.S., Shan, M., Nebot, E., Worrall, S.: Viewer-centred surface completion for unsupervised domain adaptation in 3D object detection. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 9346–9353. IEEE (2023)
[34]
Tsai D, Berrio JS, Shan M, Worrall S, and Nebot E See eye to eye: a lidar-agnostic 3D detection framework for unsupervised multi-target domain adaptation IEEE Robot. Autom. Lett. 2022 7 3 7904-7911
[35]
Wang, Y., et al.: Train in Germany, test in the USA: making 3D object detectors generalize. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11713–11723 (2020)
[36]
Wang, Y., Yin, J., Li, W., Frossard, P., Yang, R., Shen, J.: SSDA3D: semi-supervised domain adaptation for 3D object detection from point cloud. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 2707–2715 (2023)
[37]
Wang, Z., et al.: Cirrus: a long-range bi-pattern LiDAR dataset. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 5744–5750. IEEE (2021)
[38]
Wei Y, Wei Z, Rao Y, Li J, Zhou J, and Lu J Avidan S, Brostow G, Cissé M, Farinella GM, and Hassner T LiDAR distillation: bridging the beam-induced domain gap for 3D object detection European Conference on Computer Vision 2022 2022 Cham Springer 179-195
[39]
Wu H, Deng J, Wen C, Li X, and Wang C CasA: a cascade attention network for 3D object detection from LiDAR point clouds IEEE Trans. Geosci. Remote Sens. 2022 60 1-11
[40]
Wu, H., Wen, C., Li, W., Yang, R., Wang, C.: Learning transformation-equivariant features for 3D object detection (2022)
[41]
Wu, H., Wen, C., Shi, S., Li, X., Wang, C.: Virtual sparse convolution for multimodal 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21653–21662 (2023)
[42]
Xia Q et al. 3-D HANet: a flexible 3-D heatmap auxiliary network for object detection IEEE Trans. Geosci. Remote Sens. 2023 61 1-13
[43]
Xia, Q., et al.: CoIn: contrastive instance feature mining for outdoor 3D object detection with very limited annotations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6254–6263 (2023)
[44]
Xiao, P., et al.: PandaSet: advanced sensor suite dataset for autonomous driving. In: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) (2021).
[45]
Yang, J., Shi, S., Wang, Z., Li, H., Qi, X.: ST3D: self-training for unsupervised domain adaptation on 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10368–10378 (2021)
[46]
Yang J, Shi S, Wang Z, Li H, and Qi X ST3D++: denoised self-training for unsupervised domain adaptation on 3D object detection IEEE Trans. Pattern Anal. Mach. Intell. 2022 45 5 6354-6371
[47]
Yang, Z., Sun, Y., Liu, S., Jia, J.: 3DSSD: point-based 3D single stage object detector. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11040–11048 (2020)
[48]
Yihan, Z., et al.: Learning transferable features for point cloud detection via 3D contrastive co-training. In: Advances in Neural Information Processing Systems, vol. 34, pp. 21493–21504 (2021)
[49]
Yin, T., Zhou, X., Krähenbühl, P.: Center-based 3D object detection and tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
[50]
You, Y., et al.: Exploiting playbacks in unsupervised domain adaptation for 3D object detection in self-driving cars. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 5070–5077. IEEE (2022)
[51]
You, Y., et al.: Unsupervised adaptation from repeated traversals for autonomous driving. In: Advances in Neural Information Processing Systems, vol. 35, pp. 27716–27729 (2022)
[52]
Zheng, L., et al.: TJ4DRadSet: a 4D radar dataset for autonomous driving (2022)
Index Terms
- CMD: A Cross Mechanism Domain Adaptation Dataset for 3D Object Detection
Index terms have been assigned to the content through auto-classification.
Recommendations
Monocular 3D Object Detection via Feature Domain Adaptation
Computer Vision – ECCV 2020AbstractMonocular 3D object detection is a challenging task due to unreliable depth, resulting in a distinct performance gap between monocular and LiDAR-based approaches. In this paper, we propose a novel domain adaptation based monocular 3D object ...
DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes
AbstractCross-view multi-object tracking aims to link objects between frames and camera views with substantial overlaps. Although cross-view multi-object tracking has received increased attention in recent years, existing datasets still have several ...
Effective 3D object detection based on detector and tracker
In recent years, the understanding the semantics of 3D scenes has been a wide interesting researching point in many application. However, 3D scenes detection remains many problems, due to the difficulty in acquiring sufficient 3D model towards training ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sep 2024
582 pages
ISBN:978-3-031-72997-3
DOI:10.1007/978-3-031-72998-0
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 30 September 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 04 Jan 2025