Abstract
Remote operation of highly automated vehicles (HAVs) may include occasional assistance from a human remote operator that is located outside the HAVs. Remote assistance typically delegates only high-level guidance tasks to the remote operators such as authorizing a driving maneuver or specifying a new driving path. As remote assistance is fairly unexplored, there are still several research challenges. These challenges were discussed by experts from academia and industry in a multidisciplinary workshop at the 2023 IEEE Intelligent Vehicles Symposium. As a result of the workshop, this paper presents a list of most pressing research questions in the following areas: human-machine interaction and human factors, design of the remote station, design of the HAVs. It also outlines a roadmap for future research on remote assistance of HAV, thereby informing interdisciplinary studies and facilitating the benefits of HAVs before full autonomy can be reached.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
References
Amador, O., Aramrattana, M., Vinel, A.: A survey on remote operation of road vehicles. IEEE Access 10, 130135–130154 (2022). https://doi.org/10.1109/ACCESS.2022.3229168
Andersson, J., Söderman, M.: Navigating the challenges of remote operations of automated road vehicles: a socio-technical perspective. In: Ahram, T., Karwowski, W., Russo, D., Di Bucchianico, G.(eds.) Intelligent Human Systems Integration (IHSI 2024), vol. 119 (2024). https://doi.org/10.54941/ahfe1004466
Bogdoll, D., Orf, S., Töttel, L., Zöllner, J.M.: Taxonomy and survey on remote human input systems for driving automation systems. In: Arai, K. (ed.) Advances in Information and Communication, pp. 94–108. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-030-98015-3_6
Bout, M., Brenden, A.P., Klingegård, M., Habibovic, A., Böckle, M.P.: A head-mounted display to support teleoperations of shared automated vehicles. In: AutomotiveUI 2017 pp. 62-66. Association for Computing Machinery, New York (2017). https://doi.org/10.1145/3131726.3131758
BSI: BSI Flex 1890 Connected and automated mobility - Vocabulary. BSI Flex 1890 v5.0:2023-04, The British Standards Institution (Apr 2023)
Consortia SIT: AVSC Best Practice for ADS Remote Assistance Use Case. AVSC-I-04-2023, SAE Industry Technologies Consortia (Nov 2023)
Fairfield, N., Seth Herbach, J., Furman, V.: Remote Assistance for Autonomous Vehicles in Predetermined Situations, U.S. Patent US-20150248131-A1, Sept. 3 (2015)
Kettwich, C., Schrank, A., Avsar, H., Oehl, M.: A helping human hand: relevant scenarios for the remote operation of highly automated vehicles in public transport. Appl. Sci. 12(9), 4350 (2022). https://doi.org/10.3390/app12094350
Kettwich, C., Schrank, A., Oehl, M.: Teleoperation of highly automated vehicles in public transport: user-centered design of a human-machine interface for remote-operation and its expert usability evaluation. Multimodal Technol. Interact. 5(5), 26 (2021). https://doi.org/10.3390/mti5050026
Koskinen, H., Schrank, A., Lehtonen, E., Oehl, M.: Analyzing the remote operation task to support highly automated vehicles – suggesting the core task analysis to ensure the human-centered design of the remote operation station. In: Proceedings of the 26th International Conference on Human-Computer Interaction (2024)
Larsson, P., de Souza, J.B.R., Begnert, J.: An auditory display for remote road vehicle operation that increases awareness and presence. In: The 28th International Conference on Auditory Display, pp. 113–120. Georgia Institute of Technology, Norrköping, Sweden (Jun 2023)
Li, S., Zhang, Y., Blythe, P., Edwards, S., Ji, Y.: Remote driving as the failsafe: Qualitative investigation of users’ perceptions and requirements towards the 5g-enabled level 4 automated vehicles. Transport. Res. F: Traffic Psychol. Behav. 100, 211–230 (2024)
Majstorović, D., Diermeyer, F.: Dynamic collaborative path planning for remote assistance of highly-automated vehicles. In: 2023 IEEE International Automated Vehicle Validation Conference (IAVVC), pp. 1–6 (2023). https://doi.org/10.1109/IAVVC57316.2023.10328097
Mharolkar, S., Kircali, D., Yang, H., Yang, X., Wang, D.: High-fidelity teleoperation for heavy-duty vehicles. In: 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), pp. 6171–6174 (2023). https://doi.org/10.1109/ITSC57777.2023.10422036
Neumeier, S., Wintersberger, P., Frison, A.K., Becher, A., Facchi, C., Riener, A.: Teleoperation: the holy grail to solve problems of automated driving? sure, but latency matters. In: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2019, pp. 186-197. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3342197.3344534, https://doi.org/10.1145/3342197.3344534
SAE: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. SAE J3016_202104, SAE International (Apr 2021)
Schrank, A., Kettwich, C.: Roles in the teleoperation of highly automates vehicles in public transport. In: Humanist VCE Proceedings of the 7th Humanist Conference (2021). https://www.humanist-vce.eu/_files/archives/ace9ed_64d1f0cf8a164dd185a50e52238fa1a4.zip?dn=Rhodes-2021.zip
Schrank, A., Walocha, F., Brandenburg, S., Oehl, M.: Human-centered design and evaluation of a workplace for the remote assistance of highly automated vehicles. Cognition Technol. Work (2024). https://doi.org/10.1007/s10111-024-00753-x
Schrank, A., Wendorff, N., Oehl, M.: Assisting the remote assistant: Augmenting degraded video streams with additional sensor data to improve situation awareness in complex urban traffic. In: Proceedings of the 26th International Conference on Human-Computer Interaction (2024)
Skogsmo, I., Andersson, J., Jernberg, C., Aramrattana, M.: One2many : remote operation of multiple vehicles. Tech. Rep. 1164A, Swedish National Road and Transport Research Institute, Traffic and road users (2023)
UNECE: Informal document No.7 - Remote activities related to driving - (Finland, Germany and the United Kingdom) (September 2023), https://unece.org/sites/default/files/2023-09/ECE-TRANS-WP1-2023-Informal-%20document%20No7e.pdf
UNECE: ECE/TRANS/WP.1/2024/3 - Remote activities related to driving - (Finland, Germany and the United Kingdom) (January 2024), https://unece.org/sites/default/files/2024-02/ECE-TRANS-WP1-2024-3e.pdf
Veitch, E., Andreas Alsos, O.: A systematic review of human-ai interaction in autonomous ship systems. Saf. Sci. 152, 105778 (2022)
Zhang, T.: Toward automated vehicle teleoperation: vision, opportunities, and challenges. IEEE Internet Things J. 7(12), 11347–11354 (2020). https://doi.org/10.1109/JIOT.2020.3028766
Zhao, L., Nybacka, M., Drugge, L., Rothhämel, M., Habibovic, A., Hvitfeldt, H.: The influence of motion-cueing, sound and vibration feedback on driving behavior and experience - a virtual teleoperation experiment. IEEE Trans. Intell. Trans. Syst., 1–13 (2024). https://doi.org/10.1109/TITS.2024.3353465, https://ieeexplore.ieee.org/document/10414381
Zhao, L., Nybacka, M., Rothhämel, M., Habibovic, A., Papaioannou, G., Drugge, L.: Driving experience and behavior change in remote driving - an explorative experimental study. IEEE Trans. Intell. Veh. 1–15 (2023). https://doi.org/10.1109/TIV.2023.3344890
Acknowledgments
This study was funded by Sweden’s innovation agency (VINNOVA) (grant number 2022-01647).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
Azra Habibovic was employed by Scania CV AB. The rest of authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Aramrattana, M. et al. (2024). A Roadmap Towards Remote Assistance: Outcomes from Multidisciplinary Workshop at the 2023 Intelligent Vehicles Symposium. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2024 Posters. HCII 2024. Communications in Computer and Information Science, vol 2118. Springer, Cham. https://doi.org/10.1007/978-3-031-61963-2_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-61963-2_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-61962-5
Online ISBN: 978-3-031-61963-2
eBook Packages: Computer ScienceComputer Science (R0)