More Web Proxy on the site http://driver.im/

research-article

EMIDAS: explainable social interaction-based pedestrian intention detection across street

Authors:

Matthias Klusch,

Patrick Gebhard,

Tanja SchneebergerAuthors Info & Claims

SAC '21: Proceedings of the 36th Annual ACM Symposium on Applied Computing

Pages 107 - 115

https://doi.org/10.1145/3412841.3441891

Published: 22 April 2021 Publication History

Abstract

An explainable, accurate, and fast prediction of pedestrian movements in streets is an essential requirement for self-driving cars and remains a daunting challenge. Current algorithmic approaches rely solely on visual information. The information about social interaction between pedestrians across the street is not considered yet. The intention to cross the street can be influenced by social interaction with another pedestrian across the street, which comes with observable social signals such as hand waving. This paper presents EMIDAS, a dynamic Bayesian network model that uses various social signals to predict the intention to meet another pedestrian across the street. For training and evaluating this model, we adopted typical procedures from the area of social signal analysis, which consists of collecting real prototypical scenarios, annotating them concerning the pedestrians' intention to cross the street, and creating scenes from the car's field of view to test the model. This approach's benefit is that it can be employed to explain the reasoning and its underlying knowledge base. Both aspects are essential for future self-driving cars, especially when considering that such future cars have to maintain a level of trust towards the car's passengers.

References

[1]

Wagner, J., André, E. (2018): Real-time sensing of affect and social signals in a multimodal framework: a practical approach. In The Handbook of Multimodal-Multisensor Interfaces: Signal Processing, Architectures, and Detection of Emotion and Cognition-Volume 2 (pp. 227--261).

Digital Library

[2]

Abduallah, M.; Kun, Q.; Mohamed, E.; Claudel, C. (2020): Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).

[3]

Alahi, A.; Goel, K.; Ramanathan, V; Robicquet, A.; Fei-Fei, L.; Savarese, S. (2016): Social LSTM: Human trajectory prediction in crowded spaces. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).

[4]

Baur, T. (2018): Cooperative and transparent machine learning for the context-sensitive analysis of social interactions. Dissertation, CSD, University of Augsburg, Germany.

[5]

BayesFusion, LLC (2020): GeNIe Modeler User Manuel (Version 2.5.R4). Retrieved from https://support.bayesfusion.com/docs/GeNIe.pdf. Accessed: June 29, 2020.

[6]

Birdwhistell, R. L. (2011): Kinesics and context: Essays on body motion communication. University of Pennsylvania press.

[7]

Blaiotta C. (2019): Learning generative socially aware models of pedestrian motion. IEEE Robotics and Automation Letters, 4(4):3433--3440.

[8]

Clark, H. H. and Krych, M. A. (2004) Speaking while monitoring addressees for understanding. Journal of memory and language, 50(1):62--81.

[9]

Ferrer G.; Sanfeliu A (2014): Behavior estimation for a complete framework for human motion prediction in crowded environments. In Proc. of IEEE Int. Conf. on Robotics and Automation (ICRA).

[10]

Good, I.J. (1985): Weight of Evidence: A Brief Survey. Bayesian Statistics, (2), 249--270.

[11]

Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., Alahi, A. (2018): Social GAN: Socially Acceptable Trajectories with Generative Adversarial Nnetworks. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (pp. 2255--2264).

[12]

Lee, N., Choi, W., Vernaza, P., Choy, C. B., Torr, P. H., Chandraker, M. (2017): Desire: Distant Future Prediction in Dynamic Scenes with Interacting Agents. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (pp. 336--345).

[13]

Liang, J., Jiang, L., Niebles, J. C., Hauptmann, A. G., Fei-Fei, L. (2019): Peeking into the future: Predicting Future Person Activities and Locations in Videos. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (pp. 5725--5734).

[14]

Madigan, D., Mosurski, K., Almond, R. G. (1997): Graphical explanation in belief networks. Journal of Computational and Graphical Statistics, 6(2), 160--181.

[15]

Mehrabian, A. (1980): Public Places and Private Spaces: The Psychology of Work, Play, and Living Environments. Basic Books.

[16]

Muscholl, N., Poibrenski, A., Klusch, M. and Gebhard, P. (2020) SIMP3: Social Interaction-Based Multi-Pedestrian Path Prediction By Self-Driving Cars In: Proc. of IEEE International Symp. on Computational Intelligence in Vehicles and Transportation Systems (CIVTS) (pp. t.b.a.).

[17]

Pearson, E. S. (1931): The Test of Significance for the Correlation Coefficient. Journal of the American Statistical Association, 26(174), 128--134.

[18]

Picard, R. W. (1997): Affective Computing. MIT Press, Cambridge, MA.

[19]

Rasouli, A., Kotseruba, I., Kunic, T., Tsotsos, J. K. (2019): PIE: A large-scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction. In Proc. of the IEEE International Conf. on Computer Vision (pp. 6262--6271).

[20]

Vinciarelli, A., Salamin, H., Pantic, M. (2009): Social Signal Pprocessing: Understanding Social Interactions Through Nonverbal Behavior Analysis. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition Workshops (pp. 42--49).

[21]

Vinciarelli, A., Esposito, A. (2018): Multimodal Analysis of Social Signals. In The Handbook of Multimodal-Multisensor Interfaces: Signal Processing, Architectures, and Detection of Emotion and Cognition-Volume 2 (pp. 203--226).

Digital Library

[22]

Xue, H., Huynh, D. Q., Reynolds, M. (2018): SS-LSTM: A hierarchical LSTM model for Pedestrian Trajectory Prediction. In Proc. of IEEE Conf. on Applications of Computer Vision (WACV) (pp. 1186--1194). IEEE.

[23]

Yuan, C., Lim, H., Lu, T. C. (2011): Most relevant explanation in Bayesian networks. Journal of Artificial Intelligence Research, 42, 309--352.

Digital Library

Cited By

Hussien MMelo ABallardini AMaldonado CIzquierdo RSotelo M(2025)RAG-based explainable prediction of road users behaviors for automated driving using knowledge graphs and large language modelsExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125914265:COnline publication date: 15-Mar-2025
https://dl.acm.org/doi/10.1016/j.eswa.2024.125914
Kuznietsov AGyevnar BWang CPeters SAlbrecht S(2024)Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic ReviewIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.347446925:12(19342-19364)Online publication date: Dec-2024
https://doi.org/10.1109/TITS.2024.3474469
Melo AHerrera-Quintero LSalinas CSotelo M(2024)Knowledge-based explainable pedestrian behavior predictor2024 IEEE Intelligent Vehicles Symposium (IV)10.1109/IV55156.2024.10588605(3348-3355)Online publication date: 2-Jun-2024
https://doi.org/10.1109/IV55156.2024.10588605
Show More Cited By

Recommendations

PIEPredict++: An Improved Pedestrian Intention Estimation Model Incorporating Comprehensive Environment Information
Advances in Visual Computing
Abstract
Predicting pedestrians’ highly variable crossing intentions has attracted considerable attention in autonomous driving research. This paper introduces PIEPredict++, a new model for predicting pedestrian intentions. It uses dynamic and static ...
Numerical Analysis of Tractor Accidents using Driving Simulator for Autonomous Driving Tractor
ICMRE'19: Proceedings of the 5th International Conference on Mechatronics and Robotics Engineering

Autonomous driving of automobiles is a hot research topic in recent years. The autonomous driving tractor also has been studied in the agricultural field as well as an autonomous driving automobile. On the other hand, tractor accidents frequently occur ...
Switching Back to Manual Driving: How Does it Compare to Simply Driving Away After Parking?
Automotive'UI 16: Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Is there a difference in behavior when drivers start driving after parking compared to taking over from an autonomous driving car? In the former, the driving context switch (from static to driving) might be bigger than the latter, where drivers are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '21: Proceedings of the 36th Annual ACM Symposium on Applied Computing

March 2021

2075 pages

ISBN:9781450381048

DOI:10.1145/3412841

Conference Chairs:
Chih-Cheng Hung
Kennesaw State University
,
Jiman Hong
Soongsil University, South Korea
,
Program Chairs:
Alessio Bechini
University of Pisa, Italy
,
Eunjee Song
Baylor University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

German Research Foundation (DFG)
German Ministry for Research

Conference

SAC '21

Sponsor:

SIGAPP

SAC '21: The 36th ACM/SIGAPP Symposium on Applied Computing

March 22 - 26, 2021

Virtual Event, Republic of Korea

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25

Sponsor:
sigapp

The 40th ACM/SIGAPP Symposium on Applied Computing

March 31 - April 4, 2025

Catania , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
106
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hussien MMelo ABallardini AMaldonado CIzquierdo RSotelo M(2025)RAG-based explainable prediction of road users behaviors for automated driving using knowledge graphs and large language modelsExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125914265:COnline publication date: 15-Mar-2025
https://dl.acm.org/doi/10.1016/j.eswa.2024.125914
Kuznietsov AGyevnar BWang CPeters SAlbrecht S(2024)Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic ReviewIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.347446925:12(19342-19364)Online publication date: Dec-2024
https://doi.org/10.1109/TITS.2024.3474469
Melo AHerrera-Quintero LSalinas CSotelo M(2024)Knowledge-based explainable pedestrian behavior predictor2024 IEEE Intelligent Vehicles Symposium (IV)10.1109/IV55156.2024.10588605(3348-3355)Online publication date: 2-Jun-2024
https://doi.org/10.1109/IV55156.2024.10588605
Pang HHuang C(2024)Pedestrian Detection: An Explainable Approach2024 International Conference on Machine Learning and Applications (ICMLA)10.1109/ICMLA61862.2024.00146(985-990)Online publication date: 18-Dec-2024
https://doi.org/10.1109/ICMLA61862.2024.00146
Capy SVenture GRaksincharoensak P(2023)Pedestrians and Cyclists’ Intention Estimation for the Purpose of Autonomous DrivingInternational Journal of Automotive Engineering10.20485/jsaeijae.14.1_1014:1(10-19)Online publication date: 2023
https://doi.org/10.20485/jsaeijae.14.1_10

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten