Ghost-Probe: NLOS Pedestrian Rushing Detection with Monocular Camera for Automated Driving
Authors: 
Hao Yan, Feng Lin, Jin Li, Meng Zhang, Zhisheng Yan, Jian Xiao, Kui RenAuthors Info & Claims
Pages 390 - 402
Abstract
One of the most serious factors compromising driving safety is when people in drivers' non-line-of-sight areas rush out suddenly. Existing studies on non-line-of-sight imaging rely on expensive equipment or are limited to severe laboratory conditions (e.g., massive planar reflectors and controlled illumination), rendering these technologies inapplicable in complex driving scenarios. In this paper, we propose a non-line-of-sight moving obstacle detection system Ghost-Probe, which can provide an advanced driver assistance system (ADAS) with sufficient time to respond and stop safely. We design a shadow signal discriminator to assess the weak shadows created by a moving obstacle, such as pedestrians in the blind area, while simultaneously filtering out the impacts of other complicated illumination. Note that we merely use commercial monocular cameras and our system is robust to a wide range of lighting scenarios and planar reflectors. We evaluate the generalizability of our approach using the datasets collected in real-world driving scenarios with a variety of road surface and lighting circumstances. The results indicate that our system can detect the moving pedestrian in the non-line-of-sight area at a distance of 20 meters and offer the ADAS system advance warning to keep a safe distance.
References
[1]
National Highway Traffic Safety Administration(NHTSA). 2021. NHTSA promotes safe behaviors on our nation's roads. (2021). https://www.nhtsa.gov/road-safety
[2]
automotive World. 2021. V2X is close. Here's what still needs to happen. (2021). https://www.automotiveworld.com/articles/v2x-is-close-heres-what-still-needs-to-happen/
[3]
Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020).
[4]
Gary Bradski and Adrian Kaehler. 2008. Learning OpenCV: Computer vision with the OpenCV library. " O'Reilly Media, Inc.".
[5]
Yanpeng Cao, Rui Liang, Jiangxin Yang, Yanlong Cao, Zewei He, Jian Chen, and Xin Li. 2022. Computational framework for steady-state NLOS localization under changing ambient illumination conditions. Optics Express 30, 2 (2022), 2438--2452.
[6]
Tim Charlet. 2015. Headlight Use Laws for All 50 States. (2015). https://www.yourmechanic.com/article/headlight-use-laws-for-all-50-states
[7]
Wenzheng Chen, Fangyin Wei, Kiriakos N Kutulakos, Szymon Rusinkiewicz, and Felix Heide. 2020. Learned feature embeddings for non-line-of-sight imaging and recognition. ACM Transactions on Graphics (ToG) 39, 6 (2020), 1--18.
[8]
Zhihao Chen, Liang Wan, Lei Zhu, Jia Shen, Huazhu Fu, Wennan Liu, and Jing Qin. 2021. Triple-cooperative video shadow detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2715--2724.
[9]
CSGNetwork. 2023. Brake Distance. (2023). http://www.csgnetwork.com/stopdistcalc.html
[10]
Graham D Finlayson, Mark S Drew, and Cheng Lu. 2009. Entropy minimization for shadow removal. International Journal of Computer Vision 85, 1 (2009), 35--57.
[11]
Graham D Finlayson, Steven D Hordley, Cheng Lu, and Mark S Drew. 2005. On the removal of shadows from images. IEEE transactions on pattern analysis and machine intelligence 28, 1 (2005), 59--68.
[12]
Martin A. Fischler and Robert C. Bolles. 1981. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun. ACM 24, 6 (jun 1981), 381--395.
[13]
Maciej Gryka, Michael Terry, and Gabriel J Brostow. 2015. Learning to remove soft shadows. ACM Transactions on Graphics (TOG) 34, 5 (2015), 1--15.
[14]
Xiaowei Hu, Lei Zhu, Chi-Wing Fu, Jing Qin, and Pheng-Ann Heng. 2018. Direction-aware spatial context features for shadow detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7454--7462.
[15]
Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. 2017. Flownet 2.0: Evolution of optical flow estimation with deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2462--2470.
[16]
Julian Iseringhausen and Matthias B Hullin. 2020. Non-line-of-sight reconstruction using efficient transient rendering. ACM Transactions on Graphics (ToG) 39, 1 (2020), 1--14.
[17]
Achuta Kadambi, Hang Zhao, Boxin Shi, and Ramesh Raskar. 2016. Occluded imaging with time-of-flight sensors. ACM Transactions on Graphics (ToG) 35, 2 (2016), 1--12.
[18]
Salman H Khan, Mohammed Bennamoun, Ferdous Sohel, and Roberto Togneri. 2015. Automatic shadow detection and removal from a single image. IEEE transactions on pattern analysis and machine intelligence 38, 3 (2015), 431--446.
[19]
Martin Laurenzis, Andreas Velten, and Jonathan Klein. 2016. Dual-mode optical sensing: three-dimensional imaging and seeing around a corner. Optical Engineering 56, 3 (2016), 031202.
[20]
Xin Lei, Liangyu He, Yixuan Tan, Ken Xingze Wang, Xinggang Wang, Yihan Du, Shanhui Fan, and Zongfu Yu. 2019. Direct object recognition without line-of-sight using optical coherence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11737--11746.
[21]
Guan-Ting Lin, Vinay Malligere Shivanna, and Jiun-In Guo. 2020. A Deep-learning model with task-specific bounding box regressors and conditional back-propagation for moving object detection in ADAS applications. Sensors 20, 18 (2020), 5269.
[22]
Zhenguang Liu, Shuang Wu, Shuyuan Jin, Qi Liu, Shijian Lu, Roger Zimmermann, and Li Cheng. 2019. Towards Natural and Accurate Future Motion Prediction of Humans and Animals. In CVPR. 10004--10012.
[23]
Taichi Nakashima and Yoshito Yabuta. 2018. Object Detection by using Interframe Difference Algorithm. In 2018 12th France-Japan and 10th Europe-Asia Congress on Mechatronics. 98--102.
[24]
Felix Naser, Igor Gilitschenski, Guy Rosman, Alexander Amini, Fredo Durand, Antonio Torralba, Gregory W Wornell, William T Freeman, Sertac Karaman, and Daniela Rus. 2018. Shadowcam: Real-time detection of moving obstacles behind a corner for autonomous vehicles. In 2018 21st International Conference on Intelligent Transportation Systems (ITSC). IEEE, 560--567.
[25]
The National Safety Council of America. 2018. The Most Dangerous Time to Drive. (2018). https://www.nsc.org/road/safety-topics/driving-at-night?
[26]
Rohit Pandharkar, Andreas Velten, Andrew Bardagjy, Everett Lawson, Moungi Bawendi, and Ramesh Raskar. 2011. Estimating motion and size of moving non-line-of-sight objects in cluttered environments. In CVPR 2011. IEEE, 265--272.
[27]
Prashant W Patil and Subrahmanyam Murala. 2018. MSFgNet: A novel compact end-to-end deep network for moving object detection. IEEE Transactions on Intelligent Transportation Systems 20, 11 (2018), 4066--4077.
[28]
Han Qiu, Yuchen Ma, Zeming Li, Songtao Liu, and Jian Sun. 2020. Borderdet: Border feature for dense object detection. In European Conference on Computer Vision. Springer, 549--564.
[29]
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015).
[30]
Chang-Gyun Roh, Jisoo Kim, and I-Jeong Im. 2020. Analysis of impact of rain conditions on ADAS. Sensors 20, 23 (2020), 6720.
[31]
Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: An efficient alternative to SIFT or SURF. In 2011 International Conference on Computer Vision. 2564--2571.
[32]
Sheila W Seidel, John Murray-Bruce, Yanting Ma, Christopher Yu, William T Freeman, and Vivek K Goyal. 2020. Two-dimensional non-line-of-sight scene estimation from a single edge occluder. IEEE Transactions on Computational Imaging 7 (2020), 58--72.
[33]
Dongeek Shin, Ahmed Kirmani, Vivek K Goyal, and Jeffrey H Shapiro. 2015. Photon-efficient computational 3-D and reflectivity imaging with single-photon detectors. IEEE Transactions on Computational Imaging 1, 2 (2015), 112--125.
[34]
Le-Anh Tran, Truong-Dong Do, Dong-Chul Park, and My-Ha Le. 2021. Enhancement of Robustness in Object Detection Module for Advanced Driver Assistance Systems. In 2021 International Conference on System Science and Engineering (ICSSE). IEEE, 158--163.
[35]
Arizona State University. 2023. Pedestrian Injuries and Fatalities. (2023). https://popcenter.asu.edu/content/pedestrian-injuries-fatalities-0
[36]
Cadillac vehicles. 2016. super-cruise: the first hands free driver-assistance technology for compatible roads. (2016). https://www.gmc.com/connectivity-technology/super-cruise
[37]
Tianyu Wang, Xiaowei Hu, Chi-Wing Fu, and Pheng-Ann Heng. 2021. Singlestage instance shadow detection with bidirectional relation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1--11.
[38]
Tianyu Wang, Xiaowei Hu, Qiong Wang, Pheng-Ann Heng, and Chi-Wing Fu. 2020. Instance shadow detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1880--1889.
[39]
Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. 2021. FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops. 913--922.
[40]
World Health Organization (WHO). 2016. Road safety. (2016). https://www.who.int/data/gho/data/themes/road-safety
[41]
A. Woo, P. Poulin, and A. Fournier. 1990. A survey of shadow algorithms. IEEE Computer Graphics and Applications 10, 6 (1990), 13--32.
[42]
Timothy Woodford, Xinyu Zhang, Eugene Chai, and Karthikeyan Sundaresan. 2022. Mosaic: leveraging diverse reflector geometries for omnidirectional aroundcorner automotive radar. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services. 155--167.
[43]
Jing Wu, Xin Du, Yun-fang Zhu, and Wei-kang Gu. 2008. Adaptive fuzzy filter algorithm for real-time video denoising. In 2008 9th International Conference on Signal Processing. 1287--1291.
[44]
Xian-Tao Wu, Wen Wu, Lin-Lin Zhang, and Yi Wan. 2022. Don't worry about noisy labels in soft shadow detection. The Visual Computer (2022), 1--12.
[45]
Marie Yahiaoui, Hazem Rashed, Letizia Mariotti, Ganesh Sistu, Ian Clancy, Lucie Yahiaoui, Varun Ravi Kumar, and Senthil Yogamani. 2019. Fisheyemodnet: Moving object detection on surround-view cameras for autonomous driving. arXiv preprint arXiv:1908.11789 (2019).
[46]
Michael Ying Yang, Wentong Liao, Xinbo Li, and Bodo Rosenhahn. 2018. Deep learning for vehicle detection in aerial images. In 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 3079--3083.
[47]
Rongjie Yu, Yin Zheng, and Xiaobo Qu. 2021. Dynamic driving environment complexity quantification method and its verification. Transportation Research Part C: Emerging Technologies 127 (2021), 103051.
[48]
Quanlong Zheng, Xiaotian Qiao, Ying Cao, and Rynson WH Lau. 2019. Distraction-aware shadow detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5167--5176.
[49]
Haibo Zhou, Wenchao Xu, Jiacheng Chen, and Wei Wang. 2020. Evolutionary V2X Technologies Toward the Internet of Vehicles: Challenges and Opportunities. Proc. IEEE 108, 2 (2020), 308--323.
[50]
Barbara Zitova and Jan Flusser. 2003. Image registration methods: a survey. Image and vision computing 21, 11 (2003), 977--1000.
Index Terms
- Ghost-Probe: NLOS Pedestrian Rushing Detection with Monocular Camera for Automated Driving
Recommendations
A Monitoring System to Understand Postal Motorcyclist's Driving Behavior
ITSC '15: Proceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation SystemsIn Korea, motorcycle accidents constitute about 5% of total traffic accidents, however, the fatality rate approximates 12%, which is disproportionately higher per mile driven compared to other vehicular fatalities. Most of these fatalities are related ...
Transition to Automated: The Interaction of Activating the In-vehicle Automated Driving System
Design, User Experience, and Usability. Application DomainsAbstractAutomated driving system (ADS) becomes more popularized in recent years, and the level 2 and level 3 ADS have already been equipped on several production car models in the market. The activating interaction of the ADS is very important since it is ...
Gaze guidance reduces the number of vehicle-pedestrian collisions in a driving simulator
APGV '11: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and VisualizationDriving and visual perception are tightly linked. Every moment, a multitude of visual stimuli compete for the driver's limited attentional resources. Despite modern safety measures, traffic accidents still remain a major source of fatalities. A large ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2023
574 pages
ISBN:9798400704147
DOI:10.1145/3625687
- General Chair:
- Rasit Eskicioglu,
- Program Chair:
- Polly Huang,
- Program Co-chair:
- Neal Patwari
Copyright © 2023 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- SIGARCH: ACM Special Interest Group on Computer Architecture
- SIGBED: ACM Special Interest Group on Embedded Systems
- SIGMETRICS: ACM Special Interest Group on Measurement and Evaluation
- SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
- SIGOPS: ACM Special Interest Group on Operating Systems
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 April 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
SenSys '23
SenSys '23: 21st ACM Conference on Embedded Networked Sensor Systems
November 12 - 17, 2023
Istanbul, Turkiye
Acceptance Rates
Overall Acceptance Rate 174 of 867 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 127Total Downloads
- Downloads (Last 12 months)127
- Downloads (Last 6 weeks)13
Reflects downloads up to 14 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in