Model-based exploration of the frontier of behaviours for deep learning system testing

This is a presentation video of my talk at FSE 2020 on our paper accepted in the research track. In this paper, we introduce the notion of frontier of behaviours, i.e., the inputs at which the deep learningsystem starts to misbehave. If the frontier of misbehaviours is outside the validity domain of the system, the quality check is passed. Otherwise, the inputs at the intersection represent quality deficiencies of the system. We developed DeepJanus, a search-based tool that generates frontier inputs for deep learningsystems. The experimental results obtained for the lane keeping component of a self-driving car show that the frontier of a well trained system contains almost exclusively unrealistic roads that violate the best practices of civil engineering, while the frontier of a poorly trained one includes many valid inputs that point to serious deficiencies of the system.

This is a presentation video of my talk at FSE 2020 on our paper accepted in the research track. In this paper, we introduce the notion of frontier of behaviours, i.e., the inputs at which the deep learningsystem starts to misbehave. If the frontier of misbehaviours is outside the validity domain of the system, the quality check is passed. Otherwise, the inputs at the intersection represent quality deficiencies of the system. We developed DeepJanus, a search-based tool that generates frontier inputs for deep learningsystems. The experimental results obtained for the lane keeping component of a self-driving car show that the frontier of a well trained system contains almost exclusively unrealistic roads that violate the best practices of civil engineering, while the frontier of a poorly trained one includes many valid inputs that point to serious deficiencies of the system.