EMIDAS: explainable social interaction-based pedestrian intention detection across street

An explainable, accurate, and fast prediction of pedestrian movements in streets is an essential requirement for self-driving cars and remains a daunting challenge. Current algorithmic approaches rely solely on visual information. The information about social interaction between pedestrians across the street is not considered yet. The intention to cross the street can be influenced by social interaction with another pedestrian across the street, which comes with observable social signals such as hand waving. This paper presents EMIDAS, a dynamic Bayesian network model that uses various social signals to predict the intention to meet another pedestrian across the street. For training and evaluating this model, we adopted typical procedures from the area of social signal analysis, which consists of collecting real prototypical scenarios, annotating them concerning the pedestrians' intention to cross the street, and creating scenes from the car's field of view to test the model. This approach's benefit is that it can be employed to explain the reasoning and its underlying knowledge base. Both aspects are essential for future self-driving cars, especially when considering that such future cars have to maintain a level of trust towards the car's passengers.