More Web Proxy on the site http://driver.im/

research-article

3D object detection based on the fusion of projected point cloud and image features

Authors:

Ruijuan WangAuthors Info & Claims

EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering

Pages 1473 - 1478

https://doi.org/10.1145/3573428.3573687

Published: 15 March 2023 Publication History

Abstract

The complementary advantages of point cloud and image can provide more accurate 3D and semantic information to the model. Aiming at the problems that most existing methods adopt a single fusion strategy and thus fail to achieve deep fusion of image and point cloud features, this paper studies and analyzes the existing fusion strategy of image and point cloud data, and proposes a model based on the fusion of projected point cloud and image features. The model utilizes a projection fusion and feature fusion strategy, introduces a wide threshold processing in the projection module, meanwhile applies the fusion of point clouds and image features after projection cropping, finally integrates both features in depth by adding a weight fusion layer in the feature fusion stage. Extensive experiments on the public KITTI dataset demonstrate that mAP of the proposed method is improved by 3.34% in the average values of easy difficulty compared with similar models, indicating that the algorithm is more effective in 3D object detection with point cloud and image fusion.

References

[1]

Dai A, Chang A X, Savva M, Scannet: Richly-annotated 3d reconstructions of indoor scenes [C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 5828-5839.

[2]

Chen X, Ma H, Wan J, Multi-view 3d object detection network for autonomous driving [C]// Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017: 1907-1915.

[3]

Xu D, Anguelov D, Jain A. Pointfusion: Deep sensor fusion for 3d bounding box estimation [C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2018: 244-253.

[4]

Qi C R, Su H, Mo K, Pointnet: Deep learning on point sets for 3d classification and segmentation [C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 652-660.

[5]

J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. L. Waslander, "Joint 3D Proposal Generation and Object Detection from View Aggregation," 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 1-8.

Digital Library

[6]

Wang X, Xu L, Sun H, On-road vehicle detection and tracking using MMW radar and monovision fusion[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(7): 2075-2084.

[7]

Qi C R, Liu W, Wu C, Frustum pointnets for 3d object detection from rgb-d data [C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2018: 918-927.

[8]

Qi C R, Yi L, Su H, Pointnet++: Deep hierarchical feature learning on point sets in a metric space [C]// Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, 2017: 04-09.

[9]

Yang Z, Sun Y, Liu S, 3DSSD: Point-Based 3D Single Stage Object Detector [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 0. 2020.

[10]

Shi S, Wang X, Li H. PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019.

[11]

Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? the kitti vision benchmark suite [C]// 2012 IEEE conference on computer vision and pattern recognition, 2012: 3354-3361.

[12]

Huang T, Liu Z, Chen X, Epnet: Enhancing point features with image semantics for 3d object detection [C]// European Conference on Computer Vision, 2020: 35-52.

[13]

Wang D Z, Posner I. Voting for Voting in Online Point Cloud Object Detection [C]// Robotics: Science and Systems 2015. 2015.

[14]

Xiaozhi, Chen, Kaustav. 3D Object Proposals using Stereo Imagery for Accurate Object Class Detection. [J]. IEEE transactions on pattern analysis and machine intelligence, 2017, PP (99):1-1.

[15]

Pang S, Morris D, Radha H. CLOCs: Camera-LiDAR object candidates fusion for 3D object detection [C]// 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020: 10386-10393.

Cited By

Wang YAbd Rahman ANor Rashid FRazali M(2024)Tackling Heterogeneous Light Detection and Ranging-Camera Alignment Challenges in Dynamic Environments: A Review for Object DetectionSensors10.3390/s2423785524:23(7855)Online publication date: 9-Dec-2024
https://doi.org/10.3390/s24237855

Recommendations

Point cloud 3D object detection method based on density information-local feature fusion
Abstract
Nowadays, three-dimensional (3D) point cloud is widely used in unmanned driving, high-precision mapping, robot grasping, mapping and virtual reality (VR) / augmented reality (AR), etc. Especially, many studies have focused on object detection ...
PCDR-DFF: multi-modal 3D object detection based on point cloud diversity representation and dual feature fusion
Abstract
Recently, multi-modal 3D object detection techniques based on point clouds and images have received increasing attention. However, existing methods for multi-modal feature fusion are often relatively singular, and single point cloud representation ...
Two-Stage Feature Attention Fusion for Radar-Camera 3D Object Detection
ADMIT '23: Proceedings of the 2023 2nd International Conference on Algorithms, Data Mining, and Information Technology

Multi-sensor fusion is essential for 3D object detection in intelligent transportation due to it makes best use of cross-modality information, in which feature-level fusion of millimeter-wave radar and camera has been a hot topic. Existing research ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering

October 2022

1999 pages

ISBN:9781450397148

DOI:10.1145/3573428

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

EITCE 2022

EITCE 2022: 2022 6th International Conference on Electronic Information Technology and Computer Engineering

October 21 - 23, 2022

Xiamen, China

Acceptance Rates

Overall Acceptance Rate 508 of 972 submissions, 52%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
53
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang YAbd Rahman ANor Rashid FRazali M(2024)Tackling Heterogeneous Light Detection and Ranging-Camera Alignment Challenges in Dynamic Environments: A Review for Object DetectionSensors10.3390/s2423785524:23(7855)Online publication date: 9-Dec-2024
https://doi.org/10.3390/s24237855

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents