The Impact of Auditory Warning Types and Emergency Obstacle Avoidance Takeover Scenarios on Takeover Behavior
Authors: 
Xuenan Li, Zhaoyang XuAuthors Info & Claims
Pages 134 - 143
Abstract
Auditory signals are crucial in Level 3 autonomous vehicle takeovers, complementing visual information and reducing the demand on visual attention. This study investigated the effects of different auditory warning types (auditory icon, earcon, speech, spearcon) and emergency obstacle avoidance takeover scenarios (caused by road conditions, vehicle movement, vulnerable road users) and their interactions on driver takeover behavior. Generalized Linear Mixed Models (GLMMs) were used to analyze the data of simulated driving experiments from 40 participants. The results indicate that, for emergency obstacle avoidance takeover scenarios caused by road conditions, speech warnings exhibit high perceived acceptability but low perceived urgency; whereas in situations caused by vehicle motion or vulnerable road users, the result is the opposite. Speech and earcon warnings significantly outperform auditory icon and spearcon warnings in terms of perceived satisfaction. By integrating scale and simulated driving data, we discuss the roles of different auditory warning types in various takeover scenarios.
Supplemental Material
MP4 File
Presentation video - short version
- Download
- 37.42 MB
References
[1]
Richard W. Backs, John K. Lenneman, Jacob M. Wetzel, and Paul Green. 2003. Cardiac Measures of Driver Workload during Simulated Driving with and without Visual Occlusion. Hum Factors 45, 4 (December 2003), 525–538. https://doi.org/10.1518/hfes.45.4.525.27089
[2]
Carryl L. Baldwin, Jesse L. Eisert, Andre Garcia, Bridget Lewis, Stephanie M. Pratt, and Christian Gonzalez. 2012. Multimodal urgency coding: auditory, visual, and tactile parameters and their impact on perceived urgency. Work 41, (2012), 3586–3591. https://doi.org/10.3233/WOR-2012-0669-3586
[3]
Carryl L. Baldwin and Bridget A. Lewis. 2014. Perceived urgency mapping across modalities within a driving context. Applied Ergonomics 45, 5 (September 2014), 1270–1277. https://doi.org/10.1016/j.apergo.2013.05.002
[4]
Mattias Brannstrom, Erik Coelingh, and Jonas Sjoberg. 2010. Model-Based Threat Assessment for Avoiding Arbitrary Vehicle Collisions. IEEE Trans. Intell. Transp. Syst. 11, 3 (September 2010), 658–669. https://doi.org/10.1109/TITS.2010.2048314
[5]
Albert S. Bregman. 1997. Auditory Scene Analysis: The Perceptual Organization of Sound. Leonardo Music Journal 2, 2 (1997), 185–185. https://doi.org/10.1016/S0166-2236(96)01022-3
[6]
Stephen A. Brewster, Peter C. Wright, and Alistair D. N. Edwards. 1993. An evaluation of earcons for use in auditory human-computer interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’93, 1993. ACM Press, Amsterdam, The Netherlands, 222–227. https://doi.org/10.1145/169059.169179
[7]
Chunlei Chai, Yu Lei, Haoran Wei, Changxu Wu, Wei Zhang, Preben Hansen, Hao Fan, and Jinlei Shi. 2024. The effects of various auditory takeover requests: A simulated driving study considering the modality of non-driving-related tasks. Appl. Ergon. 118, (July 2024), 104252. https://doi.org/10.1016/j.apergo.2024.104252
[8]
Tilman Dingler, Jeffrey Lindsay, and Bruce N. Walker. 2008. Learnabiltiy of Sound Cues for Environmental Features: Auditory Icons, Earcons, Spearcons, and Speech. (June 2008). Retrieved April 29, 2024 from http://hdl.handle.net/1853/49940
[9]
Aboubakr El Jouhri, Ashraf El Sharkawy, Hakan Paksoy, Omar Youssif, Xiaolin He, Soyeon Kim, and Riender Happee. 2023. The influence of a color themed HMI on trust and take-over performance in automated vehicles. Front. Psychol. 14, (July 2023), 1128285. https://doi.org/10.3389/fpsyg.2023.1128285
[10]
William Gaver. 1989. The SonicFinder: An Interface That Uses Auditory Icons. Human-Comp. Interaction 4, 1 (March 1989), 67–94. https://doi.org/10.1207/s15327051hci0401_3
[11]
Rob Gray. 2011. Looming Auditory Collision Warnings for Driving. Hum Factors 53, 1 (February 2011), 63–74. https://doi.org/10.1177/0018720810397833
[12]
Cristy Ho. 2004. Using spatial warning signals to capture a driver's visual attention. In Proceedings of the 6th international conference on Multimodal interfaces (ICMI ’04), October 13, 2004. Association for Computing Machinery, New York, NY, USA, 350. https://doi.org/10.1145/1027933.1028008
[13]
HUANG Yu-xiang, LI Qi-wei, and YANG Zhen. 2023. Impact of Auditory Types and Error Types on Driver Takeover Performance. Packaging Engineering 44, 20 (2023), 34–41. https://doi.org/10.19554/j.cnki.1001-3563.2023.20.005
[14]
Sae International. 2014. Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems. (2014). Retrieved April 27, 2024 from http://www.mendeley.com/catalog/taxonomy-definitions-terms-related-onroad-motor-vehicle-automated-driving-systems/
[15]
Myounghoon Jeon. 2019. Multimodal Displays for Take-over in Level 3 Automated Vehicles while Playing a Game. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA ’19), May 02, 2019. Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3313056
[16]
Jinke D. Van Der Laan and Adriaan Heino and Dick De Waard. 1997. A simple procedure for the assessment of acceptance of advanced transport telematics. Transportation Research Part C: Emerging Technologies (1997). https://doi.org/10.1016/S0968-090X(96)00025-3
[17]
Sangjin Ko, Harsh Sanghavi, Yiqi Zhang, and Myounghoon Jeon. 2022. Modeling the effects of perceived intuitiveness and urgency of various auditory warnings on driver takeover performance in automated vehicles. Transportation Research Part F: Traffic Psychology and Behaviour 90, (October 2022), 70–83. https://doi.org/10.1016/j.trf.2022.08.008
[18]
Takanori Komatsu, Kazuki Kobayashi, Seiji Yamada, Kotaro Funakoshi, and Mikio Nakano. 2012. Can users live with overconfident or unconfident systems? a comparison of artificial subtle expressions with human-like expression. In CHI ’12 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’12), May 05, 2012. Association for Computing Machinery, New York, NY, USA, 1595–1600. https://doi.org/10.1145/2212776.2223678
[19]
Hannu Kukka, Jorge Goncalves, Kai Wang, Tommi Puolamaa, Julien Louis, Mounib Mazouzi, and Leire Roa Barco. 2016. Utilizing Audio Cues to Raise Awareness and Entice Interaction on Public Displays. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS ’16), June 04, 2016. Association for Computing Machinery, New York, NY, USA, 807–811. https://doi.org/10.1145/2901790.2901856
[20]
Pontus Larsson and Mathias Niemand. 2015. Using Sound to Reduce Visual Distraction from In-vehicle Human-Machine Interfaces. Traffic Inj. Prev. 16, (June 2015), S25–S30. https://doi.org/10.1080/15389588.2015.1020111
[21]
Lei Zhaoli, Ma Shu, Li Hongting, and Yang Zhen. 2022. The Impact of Different Types of Auditory Warnings on Working Memory. Front. Psychol. 13, (February 2022), 780657. https://doi.org/10.3389/fpsyg.2022.780657
[22]
Shengqin Li and Qingqing Zhao. 2023. Research on the Emergency Obstacle Avoidance Strategy of Intelligent Vehicles Based on a Safety Distance Model. IEEE Access 11, (2023), 7124–7134. https://doi.org/10.1109/ACCESS.2023.3238056
[23]
Shuling Li, Tingru Zhang, Wei Zhang, Na Liu, and Gaoyan Lyu. 2020. Effects of speech-based intervention with positive comments on reduction of driver's anger state and perceived workload, and improvement of driving performance. Applied Ergonomics 86, (July 2020), 103098. https://doi.org/10.1016/j.apergo.2020.103098
[24]
Yi Li, Zhaoze Xuan, and Xianyu Li. 2023. A Study on the Entire Take-Over Process-Based Emergency Obstacle Avoidance Behavior. IJERPH 20, 4 (February 2023), 3069. https://doi.org/10.3390/ijerph20043069
[25]
LI Zhenlong, PAN Mengniu, QU Yansong, ZHAO Xiaohua, GONG Jianguo, and WANG Qiuhong. 2023. A Method for Evaluating the Safety over the Takeover Process of the Level 3 Automated Vehicles Based on IAHP-EWM-LDM. Journal of Transport Information and Safety 41, 4 (2023), 14-23+100.
[26]
Denis McKeown and Sarah Isherwood. 2007. Mapping Candidate Within-Vehicle Auditory Displays to Their Referents. Hum Factors 49, 3 (June 2007), 417–428. https://doi.org/10.1518/001872007X200067
[27]
Fanxing Meng and Charles Spence. 2015. Tactile warning signals for in-vehicle systems. Accident Analysis & Prevention 75, (February 2015), 333–346. https://doi.org/10.1016/j.aap.2014.12.013
[28]
Alexandra Neary, Simon Y. W. Li, Isaac Salisbury, Robert G. Loeb, and Penelope M. Sanderson. 2023. Effects of multitasking on interpreting a spearcon sequence display for monitoring multiple patients. Applied Ergonomics 112, (October 2023), 104072. https://doi.org/10.1016/j.apergo.2023.104072
[29]
Neli Ovcharova, Michael Fausten, and Frank Gauterin. 2012. Effectiveness of forward collision warnings for different driver attention states. In 2012 IEEE Intelligent Vehicles Symposium, June 2012. 944–949. https://doi.org/10.1109/IVS.2012.6232232
[30]
Janghee Park, Dongchan Kim, and Kunsoo Huh. 2021. Emergency Collision Avoidance by Steering in Critical Situations. Int.J Automot. Technol. 22, 1 (February 2021), 173–184. https://doi.org/10.1007/s12239-021-0018-2
[31]
Agnes Petocz, Peter E. Keller, and Catherine J. Stevens. 2008. Auditory warnings, signal-referent relations, and natural indicators: Re-thinking theory and application. Journal of Experimental Psychology: Applied 14, 2 (June 2008), 165–178. https://doi.org/10.1037/1076-898X.14.2.165
[32]
Erin Richie, Thomas Offer-Westort, Raghavendran Shankar, and Myounghoon Jeon. 2018. Auditory Displays for Take-Over in Semi-automated Vehicles. In Digital Human Modeling. Applications in Health, Safety, Ergonomics, and Risk Management, 2018. Springer International Publishing, Cham, 623–634. https://doi.org/10.1007/978-3-319-91397-1_51
[33]
Luka Rukonic, Marie-Anne Mwange, and Suzanne Kieffer. 2021. UX Design and Evaluation of Warning Alerts for Semi-autonomous Cars with Elderly Drivers: In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2021. SCITEPRESS - Science and Technology Publications, Online Streaming, — Select a Country —, 25–36. https://doi.org/10.5220/0010237000250036
[34]
Edin Sabic and Jing Chen. 2016. Threshold of Spearcon Recognition for Auditory Menus. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 60, 1 (September 2016), 1539–1543. https://doi.org/10.1177/1541931213601353
[35]
Edin Šabić, Jing Chen, and Justin A. MacDonald. 2021. Toward a Better Understanding of In-Vehicle Auditory Warnings and Background Noise. Hum Factors 63, 2 (March 2021), 312–335. https://doi.org/10.1177/0018720819879311
[36]
Frank Schumann, Michael B. Steinborn, Jens Kürten, Liyu Cao, Barbara Friederike Händel, and Lynn Huestegge. 2022. Restoration of Attention by Rest in a Multitasking World: Theory, Methodology, and Empirical Evidence. Front. Psychol. 13, (April 2022). https://doi.org/10.3389/fpsyg.2022.867978
[37]
Jiaqing Song, Yuwei Wang, Xiaojiang An, Shu Ma, Duming Wang, Tian Gan, Hongqi Shi, Zhen Yang, and Hongyan Liu. 2022. Novel sonification designs: Compressed, iconic, and pitch-dynamic auditory icons boost driving behavior. Appl. Ergon. 103, (September 2022), 103797. https://doi.org/10.1016/j.apergo.2022.103797
[38]
Jiaqing Song, Bohan Wu, Shu Ma, Duming Wang, Tian Gan, Zhen Yang, and Hongyan Liu. 2022. Danger or avoidance indication: Dynamics interact with semantics in auditory icons to avoid collisions. Int. J. Ind. Ergon. 92, (November 2022), 103353. https://doi.org/10.1016/j.ergon.2022.103353
[39]
Monika Srbinovska, Isaac S. Salisbury, Robert G. Loeb, and Penelope M. Sanderson. 2021. Spearcon Compression Levels Influence the Gap in Comprehension Between Untrained and Trained Listeners. J. Exp. Psychol.-Appl. 27, 1 (March 2021), 69–83. https://doi.org/10.1037/xap0000330
[40]
Klara Steinhauser, Felix Leist, Kathrin Maier, Vera Michel, Nikolai Pärsch, Philip Rigley, Franz Wurm, and Marco Steinhauser. 2018. Effects of emotions on driving behavior. Transportation Research Part F: Traffic Psychology and Behaviour 59, (November 2018), 150–163. https://doi.org/10.1016/j.trf.2018.08.012
[41]
Tao ZHONG, Hai FANG, Ja-yue GUO, and Wei WANG. 2021. The Warning Effectiveness of Notification Sound on Driving Response. Packaging Engineering 42, 18 (2021), 144–150. https://doi.org/10.19554/j.cnki.1001-3563.2021.18.015
[42]
James Richard Unverricht. 2023. Attention and Task Engagement During Automated Driving. Ph.D. Old Dominion University, United States – Virginia. Retrieved April 19, 2024 from https://www.proquest.com/docview/2869176794/abstract/B969AEFDAF854210PQ/1?sourcetype=Dissertations%20&%20Theses
[43]
Akos Vetek and Saija Lemmelä. 2011. Could a dialog save your life? analyzing the effects of speech interaction strategies while driving. In Proceedings of the 13th international conference on multimodal interfaces (ICMI ’11), November 14, 2011. Association for Computing Machinery, New York, NY, USA, 145–152. https://doi.org/10.1145/2070481.2070506
[44]
WANG Bo-sen, DlNG Bing-yan, and QIU Shi-jie. 2017. Auditory Display Design Based on Information Sonification. Packaging Engineering 38, 20 (2017), 184–187. https://doi.org/10.19554/j.cnki.1001-3563.2017.20.037
[45]
Wang Linyan, Zhang Huijun, and Hu Hongyu. 2022. Effect of Driving Experience on Take-Over Performance of L3 Automatic Driving. Automotive Engineering 44, 10 (2022), 1521–1526. https://doi.org/10.19562/j.chinasae.qcgc.2022.10.006
[46]
WANG Weiyu, MA Hui, and WANG Chao. 2023. The effect of characteristics of voice and sound field on the elderly's speech intelligibility and subjective evaluation of voice alarms. Journal of Applied Acoustics 42, 4 (2023), 844–852. https://doi.org/10.11684/j.issn.1000-310X.2023.04.020
[47]
James Whitmire, Justin F. Morgan, Tal Oron-Gilad, and P.A. Hancock. 2011. The effect of in-vehicle warning systems on speed compliance in work zones. Transportation Research Part F: Traffic Psychology and Behaviour 14, 5 (September 2011), 331–340. https://doi.org/10.1016/j.trf.2011.04.001
[48]
Frederik Wiehr, Anke Hirsch, Lukas Schmitz, Nina Knieriemen, Antonio Krüger, Alisa Kovtunova, Stefan Borgwardt, Ernie Chang, Vera Demberg, Marcel Steinmetz, and Jörg Hoffmann. 2021. Why Do I Have to Take Over Control? Evaluating Safe Handovers with Advance Notice and Explanations in HAD. In Proceedings of the 2021 International Conference on Multimodal Interaction (ICMI ’21), October 18, 2021. Association for Computing Machinery, New York, NY, USA, 308–317. https://doi.org/10.1145/3462244.3479884
[49]
Michael S Wogalter, Michael J Kalsher, and Raheel Rashid. 1999. Effect of signal word and source attribution on judgments of warning credibility and compliance likelihood. International Journal of Industrial Ergonomics 24, 2 (May 1999), 185–192. https://doi.org/10.1016/S0169-8141(98)00025-0
[50]
Xingwei Wu and Linda Ng Boyle. 2021. Auditory Messages for Intersection Movement Assist (IMA) Systems: Effects of Speech- and Nonspeech-Based Cues. Hum Factors 63, 2 (March 2021), 336–347. https://doi.org/10.1177/0018720819891977
[51]
Xiong hui. 2023. On-board lmage based Multi-ObjectTracking and Motion Prediction forVuinerable Road Users. doctor. Tsinghua University. https://doi.org/10.27266/d.cnki.gqhau.2021.000059
[52]
H.C. Yadav, Rajeev Jain, Sandarbh Shukla, Shwetank Avikal, and P.K. Mishra. 2013. Prioritization of aesthetic attributes of car profile. International Journal of Industrial Ergonomics 43, 4 (July 2013), 296–303. https://doi.org/10.1016/j.ergon.2013.04.008
[53]
YAN Li-xin, FENG Jin-pei, GUo Jun-hua, and GONG Yi-ke. 2024. Analysis of characteristics of the takeover behavior of co-driving intelligent vehicles under different dangerous scenarios. Journal of Jilin University(Engineering and Technology Edition) 54, 3 (2024), 683–691. https://doi.org/10.13229/j.cnki.jdxbgxb.20220580
[54]
Nam Yoonjae. 2013. An Empirical Analysis of Auditory Interfaces in Human-computer Interaction. International Journal of Contents 9, 3 (2013), 29–34. https://doi.org/10.5392/IJoC.2013.9.2.029
[55]
Zhao Xiaohua, Chen Haolin, LI Zhenlong, Li Haijian, Gong Jianguo, and Fu Qiang. 2022. Influence Characteristics of Automated Driving TakeoverBehavior in Different Scenarios. China Journal of Highway and Transport 35, 9 (2022), 195–214. https://doi.org/10.19721/j.cnki.1001-7372.2022.09.015
[56]
Zhong Tao. 2021. Research on interactive design of automobile prompt sound. doctor. Guangdong University of Technology. https://doi.org/10.27029/d.cnki.ggdgu.2021.000049
Index Terms
- The Impact of Auditory Warning Types and Emergency Obstacle Avoidance Takeover Scenarios on Takeover Behavior
Index terms have been assigned to the content through auto-classification.
Recommendations
DeepTake: Prediction of Driver Takeover Behavior using Multimodal Data
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsAutomated vehicles promise a future where drivers can engage in non-driving tasks without hands on the steering wheels for a prolonged period. Nevertheless, automated vehicles may still need to occasionally hand the control back to drivers due to ...
Manual takeover after highly automated driving: Influence of budget time and Lane Change Assist on takeover performance
ECCE '21: Proceedings of the 32nd European Conference on Cognitive ErgonomicsThe current study aimed at investigating the effectiveness of additional assistance system to support driver performance during a takeover request in a conditionally automated vehicle, that is, Level 3 of SAE taxonomy. Seventy drivers participated in ...
Modeling the effects of auditory display takeover requests on drivers' behavior in autonomous vehicles
AutomotiveUI '19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct ProceedingsIn semi-autonomous vehicles (SAE level 3) that require driver's engagement in critical situations, it is important to secure reliable control transitions. There have been many studies on investigating appropriate auditory displays for takeover request (...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2024
725 pages
ISBN:9798400704628
DOI:10.1145/3678957
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 November 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Ministry of Education industry-university cooperative education project
- Xie Youbai Design Science Research Fund
- The Fundamental Research Funds for the Central Universities
Conference
ICMI '24
ICMI '24: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
November 4 - 8, 2024
San Jose, Costa Rica
Acceptance Rates
Overall Acceptance Rate 453 of 1,080 submissions, 42%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 44Total Downloads
- Downloads (Last 12 months)44
- Downloads (Last 6 weeks)44
Reflects downloads up to 12 Dec 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in