[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1109/IV51971.2022.9827394guideproceedingsArticle/Chapter ViewAbstractPublication PagesConference Proceedingsacm-pubtype
research-article

How Simulation based Test Methods will substitute the Proving Ground Testing?

Published: 04 June 2022 Publication History

Abstract

Advanced Driver Assistance Systems have been integrated in vehicles with the aim of reducing traffic accidents caused by human error. Due to the safety-critical application of these systems, a reliable implementation requires an extensive testing process of hardware and software units as well as the entire integrated system. The increasing complexity of these systems, which incrementally have more control over the vehicle, makes purely real-world safeguarding impracticable. As a solution, complementary virtual tests are used to support the validation process in a safe and accelerated way. However, a method for distributing test cases across real and virtual domains according to the relevance of the test method has yet to be defined. In this paper, we investigate different levels of reality in testing ADAS/automated driving functions and derive advantages, disadvantages, and system limitations for each. An emergency braking system serves as a use case, which is evaluated using Hardware-in-the-Loop, Sensor-in-the-Loop, Vehicle-in-the-Loop test methods, as well as vehicle tests on a proving ground. The obtained results show that simulation-based testing is a useful complement to real-world testing. However, various phenomena such as actuator delays or camera lenses have a large impact on the obtained results and need to be taken into account (in models) to ensure the realism of the simulation-based approaches. In addition, the inclusion of real components increases the deviations between test repetitions, being necessary to conduct real tests to evaluate the final performance of the integrated system.

References

[1]
European Commission. Directorate General for Mobility and Transport, Next steps towards ‘Vision Zero’:EU road safety policy framework 2021 2030. Publications Office, 2020. [Online]. Available: https://data.europa.eu/doi/10.2832/261629
[2]
H. Tan, F. Zhao, H. Hao, and Z. Liu, “Estimate of safety impact of lane keeping assistant system on fatalities and injuries reduction for china: Scenarios through 2030,” Traffic Injury Prevention, vol. 21, no. 2, pp. 156–162, 2020. 32023126.
[3]
J. Kovaceva, A. Bálint, R. Schindler, and A. Schneider, “Safety benefit assessment of autonomous emergency braking and steering systems for the protection of cyclists and pedestrians based on a combination of computer simulation and real-world test results,” Accident Analysis & Prevention, vol. 136, p. 105352, 2020.
[4]
European New Car Assessment Programme NCAP, “Assessment protocol – safety assist safe driving,” 2021, last accessed 04 January 2022. [Online]. Available: https://cdn.euroncap.com/media/67268/euro-ncap-assessment-protocol-sa-safe-driving-v100.pdf
[5]
European New Car Assessment Programme NCAP, “Assessment protocol – vulnerable road user protection,” 2021, last accessed 04 January 2022. [Online]. Available: https://cdn.euroncap.com/media/67328/euro-ncap-assessment-protocol-vru-v110.pdf
[6]
European New Car Assessment Programme NCAP, “Test protocoll – aeb/lss vru systems,” 2021, last accessed 04 January 2022. [Online]. Available: https://cdn.euroncap.com/media/64154/euro-ncap-aeb-lss-vru-test-protocol-v400.pdf
[7]
Economic Commission for Europe, “New Assessment/Test Method for Automated Driving,” UNECE, Geneva, CH, White Paper, 2021.
[8]
Daimler, “Erste international gültige systemgenehmigung für hochau-tomatisiertes fahren,” Daimler, 12 2021. [Online]. Available: https://www.daimler.com/innovation/produktinnovation/autonomes-fahren/systemgenehmigung-fuer-hochautomatisiertes-fahren.html
[9]
Society of Automotive Engineers, “Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles,” SAE, Warrendale, Pennsylvania, US, Standard, May 2021.
[10]
P. Koopman and M. Wagner, “Challenges in autonomous vehicle testing and validation,” SAE International Journal of Transportation Safety, vol. 4, pp. 15–24, 04 2016.
[11]
P. Koopman and F. Fratrik, “How many operational design domains, objects, and events?” in Safe AI 2019: AAAI Workshop on Artificial Intelligence Safety, 01 2018.
[12]
N. Kalra and S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” Transportation Research Part A: Policy and Practice, vol. 94, pp. 182–193, Dec. 2016.
[13]
International Organization for Standardization, “Road vehicles – Functional safety,” ISO, Geneva, CH, Standard, 2018.
[14]
Aptiv, Audi, Apollo, BMW, Continental, Daimler, FCA, here, Infenieon, Intel and Volkswagen, “Safety first for automated driving,” 2019. [Online]. Available: https://www.daimler.com/documents/innovation/other/safety-first-for-automated-driving.pdf
[15]
H. Winner, S. Hakuli, F. Lotz, and C. Singer, Handbuch Fahreras-sistenzsysteme Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort. Springer Vieweg, 2015.
[16]
“Regulation no 140 of the economic commission for europe of the united nations (un/ece) — uniform provisions concerning the approval of passenger cars with regard to electronic stability control (esc) systems [2018/1592],” Geneva, CH, pp. 17–35, Oct 2018.
[17]
C. Rösener, J. Sauerbier, A. Zlocki, F. Fahrenkrog, L. Wang, A. Varhe-lyi, E. Gelder, S. Breunig, F. Tango, and J. Lanati, “A comprehensive evaluation approach for highly automated driving,” in 25th Enhanced Safety of Vehicles, 06 2017, pp. 1–13.
[18]
C.-H. Yu, Y.-Z. Chen, and I.-C. Kuo, “The benefit of simulation test application on the development of autonomous driving system,” in 2020 International Automatic Control Conference (CACS), 2020, pp. 1–5.
[19]
S. Riedmaier, J. Nesensohn, C. Gutenkunst, T. Düser, B. Schick, and H. Abdellatif, “Validation of x-in-the-loop approaches for virtual homologation of automated driving functions,” in 11th Graz Symposium Virtual Vehicle, 05 2018.
[20]
G. Sievers, C. Seiger, M. Peperhowe, H. Krumm, S. Graf, and H. Hanselmann, “Driving simulation technologies for sensor simulation in sil and hil environments,” in 17th Driving Simulation & Virtual Reality Conference & Exhibition, 09 2018.
[21]
F. Reway, A. Hoffmann, D. Wachtel, W. Huber, A. Knoll, and E. Ribeiro, “Test method for measuring the simulation-to-reality gap of camera-based object detection algorithms for autonomous driving,” in 2020 IEEE Intelligent Vehicles Symposium (IV), 2020, pp. 1249–1256.
[22]
D. Wachtel, S. Schröder, F. Reway, W. Huber, and M. Vossiek, “Validation of a radar sensor model under non-ideal conditions for testing automated driving systems,” in 2021 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops), 2021, pp. 83–89.
[23]
M. Funk Drechsler, J. B. Peintner, G. Seifert, W. Huber, and A. Riener, “Mixed reality environment for testing automated vehicle and pedestrian interaction,” in 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, ser. AutomotiveUI ’21 Adjunct. New York, NY, USA: Association for Computing Machinery, 2021, p. 229–232.
[24]
J. Peintner, M. Funk Drechsler, F. Reway, G. Seifert, W. Huber, and A. Riener, “Mixed reality environment for complex scenario testing,” in Mensch Und Computer 2021, ser. MuC ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 605–608.
[25]
M. F. Drechsler, J. Peintner, F. Reway, G. Seifert, A. Riener, and W. Huber, “Mire, a mixed reality environment for testing of automated driving functions,” IEEE Transactions on Vehicular Technology, vol. 71, no. 4, pp. 3443–3456, 2022.
[26]
K. Varda, “Protocol buffers: Google’s data inter-change format,” Google, Tech. Rep., 6 2008. [Online]. Available: http://google-opensource.blogspot.com/2008/07/protocol-buffers-googles-data.html
[27]
M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Ng, “Ros: an open-source robot operating system,” in ICRA Workshop on Open Source Software, vol. 3, 01 2009.
[28]
European Automobile Manufacturers’ Association, “Articulated pedestrian target acea specifications,” 2016.
[29]
J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” CoRR, vol. abs/1804.02767, 2018. [Online]. Available: http://arxiv.org/abs/1804.02767
[30]
F. Reway, W. Huber, and E. P. Ribeiro, “Test methodology for vision-based adas algorithms with an automotive camera-in-the-loop,” in 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), 2018, pp. 1–7.
[31]
M. Botsch and W. Utschick, Fahrzeugsicherheit und automatisiertes Fahren: Methoden der Signalverarbeitung und des maschinellen Lernens. Hanser, 062020, pp. 164–180.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings
2022 IEEE Intelligent Vehicles Symposium (IV)
Jun 2022
1858 pages

Publisher

IEEE Press

Publication History

Published: 04 June 2022

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 03 Jan 2025

Other Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media