Abstract
Semi-autonomous vehicles require monitoring the driver to check if he is supervising the system and/or ready to take over. Most cars rely on steering-wheel sensors to detect hands and do not monitor the non-driving related task the driver might be performing. We present a camera-based system with a multi-branch architecture, which provides the number of hands on the steering wheel, on a tablet representing a secondary task and the tablet position. It also tackles a common issue with other camera-based systems: a free hand in front of the steering wheel can be classified as grasping it. Moreover, our system deals with cases when the driver might use a tablet on the steering wheel, as he is allowed to do in autonomous mode. These two points are critical to assess the time the driver will need to take over. Finally, combining both steering wheel and camera systems would also make vehicles harder to trick and therefore safer.
Video available at: https://www.youtube.com/watch?v=qfYOM4sdWr4
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Schmitz, J.-C., Tilgner, S., Kalischewski, K., Wagner, D., Kummert, A.: Hands on wheel classification based on depth images and neural networks. MATEC Web Conf. 308, 06003 (2020). https://doi.org/10.1051/matecconf/202030806003
Le, T.H.N., Zheng, Y., Zhu, C., Luu, K., Savvides, M.: Multiple scale faster-RCNN approach to driver’s cell-phone usage and hands on steering wheel detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Las Vegas, NV, USA, pp. 46–53, June 2016. https://doi.org/10.1109/CVPRW.2016.13
Zhang, L., Yu, G., Zhou, B., Wang, Z., Xu, G.: Detection algorithm of takeover behavior of automatic vehicles’ drivers based on deep learning. In: 2019 4th International Conference on Intelligent Transportation Engineering (ICITE), pp. 126–130, September 2019. https://doi.org/10.1109/ICITE.2019.8880230
Siddharth, Rangesh, A., Ohn-Bar, E., Trivedi, M.M.: Driver hand localization and grasp analysis: a vision-based real-time approach. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), pp. 2545–2550, November 2016. https://doi.org/10.1109/ITSC.2016.7795965
Car and Driver: It’s Not Just Tesla: All Other Driver-Assist Systems Work without Drivers, Too, 11 August 2021. https://www.youtube.com/watch?v=trrmgpzPVQg. Accessed 20 Apr 2022
Acknowledgements
Results obtained in close collaboration with Nervtech (Slovenia) and the University of Granada (Spain). This research was conducted within the project HADRIAN (Holistic Approach for Driver Role Integration and Automation - Allocation for European Mobility Needs), which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 875597, https://hadrianproject.eu/. The European Climate, Infrastructure and Environment Executive Agency (CINEA) is not responsible for any use that may be made of the information it contains.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Morvillier, R., Prat, C., Aloui, S. (2023). A Camera-Based System to Detect Driver Hands on the Steering Wheel in Semi-autonomous Vehicles. In: Amini, MR., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2022. Lecture Notes in Computer Science(), vol 13718. Springer, Cham. https://doi.org/10.1007/978-3-031-26422-1_42
Download citation
DOI: https://doi.org/10.1007/978-3-031-26422-1_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26421-4
Online ISBN: 978-3-031-26422-1
eBook Packages: Computer ScienceComputer Science (R0)