Towards Learning Analytics in Cybersecurity Capture the Flag Games

Capture the Flag games are software applications designed to exercise cybersecurity concepts, practice using security tools, and understand cyber attacks and defense. We develop and employ these games at our university for training purposes, unlike in the traditional competitive setting. During the gameplay, it is possible to collect data about players' in-game actions, such as typed commands or solution attempts, including the timing of these actions. Although such data was previously employed in computer security research, to the best of our knowledge, there were few attempts to use this data primarily to improve education. In particular, we see an open and challenging research problem in creating an artificial intelligence assistant that would facilitate the learning of each player. Our goal is to propose, apply, and experimentally evaluate data analysis and machine learning techniques to derive information about the players' interactions from the in-game data. We want to use this information to automatically provide each player with a personalized formative assessment. Such assessment will help the players identify their mastered concepts and areas for improvement, along with suggestions and actionable steps to take. Furthermore, we want to identify high- or low-performing players during the game, and subsequently, offer them game tasks more suitable to their skill level. These interventions would supplement or even replace feedback from instructors, which would significantly increase the learning impact of the games, enable more students to learn cybersecurity skills at an individual pace, and lower the costs.