Keywords

1 Introduction

The use of video game technologies has drastically increased over the last two decades, they are now ubiquitous and part of the mainstream media culture [22]. In the United States, 91% of children and teenagers between the ages of 2 and 17 play video-games [7]. The emerging of electronic sports, also known as e-sports led the gaming industry to grow considerably, attracting large investments as well as new research opportunities. Traditionally, hand-held controllers have been used to control video games, however, more recent technological advancements have allowed researchers to explore new interaction and control modalities for video games such as body gestures (e.g. Microsoft Kinect and Playstation Move). However, both hand-held and gesture controllers present limitations for users with physical disabilities (e.g. upper limb differences or those in a wheelchair). Brain-Computer Interfaces (BCI) provides a direct path of communication between the human brain and electronic systems, which can be explored to create brain-controlled games. BCIs are capable of decoding users’ brain activity and converting it directly into game commands, without the use of muscle movements. Therefore, BCIs allow users with physical disabilities to play video-games they might not have been able to otherwise. Additionally, BCIs also interest able-bodied gamers as a new control modality capable of creating different gaming experiences.

Video-games can be categorized as a form of art [15], therefore brain-controlled games fall under the category of Artistic BCI. In artistic BCI applications, the user manipulates a physical or audiovisual digital environment with their brain-waves to build a creative work [21]. Additionally, these applications offer a new modality for humans to express themselves creatively [21], and gamers are known to use creativity during the game-play to compete against others, solve puzzles, and pass obstacles [2]. The artistic aspect of brain-controlled games allows players not only to express themselves by actively controlling an avatar but also from an affective BCI perspective. For example, the game might change its audiovisual features (e.g. sky color) depending on the player’s affective state measured with a BCI. As BCIs become more robust, affordable, and make their way from research laboratories into users’ homes, the video game community is expected to be among the first to embrace this technology for entertainment purposes as gamers are known to be early-adopters of new technologies [13].

A well-known game genre is racing games, such as car racing. Recently, a new form of racing has emerged in the form of drone racing [3]. Additionally, integrating racing drones and BCIs has been a focus for researchers, with brain-drone race competitions being held at various universities [18]. As brain-drone racing grows as a sport, it is expected that brain-drone racing video-games and simulations will also grow in popularity. Analogous to popular car racing games, a simulation can be used to overcome potential hassles of racing physical drones. For example, players are able to practice at a lower cost, without the risk of crashes, and does not require special authorizations for flying unlike physical drones (e.g. FAA regulations in the USA). This paper presents a brain-controlled drone racing game built with the Unity engine. The game allows users to control specific movements of a racing drone using an electroencephalography (EEG) headset while performing motor imagery. Figure 1 shows a player wearing the EEG headset while playing the game. Motor imagery is the imagination of muscle movement, resulting in signals from the motor cortex area of the brain allowing the use of BCI devices to infer the user’s intent [11]. The game allows the selection of different drones to be used as the racing avatar, types of races (e.g. lap-based vs. drag), and customization of distractions (e.g. sound and visual). As the game requires the user to focus to perform a motor imagery task, customized distractions can be used to increase the difficulty and improve players’ ability to focus. The current version of the racing simulation is integrated with the Emotiv Insight BCI, a non-invasive 5-channel EEG headset that reads electrical signals from the scalp using semi-dry electrodes.

Fig. 1.
figure 1

Player controlling brain-controlled drone racing game.

Full size image

The main contribution of this paper is a qualitative analysis of a user study conducted with 54 participants. It provides a discussion of users’ perceptions towards brain-controlled games, an analysis of how much users would pay for brain-controlled games, and insights in the future of the technology.

2 Related Work

Previous research found that non-invasive EEG headsets are the most suitable BCI technology for gaming due to high temporal resolution, low cost (compared to other BCI’s), safety, and portability [11]. Active control using BCIs has been used for a variety of game genres: action, simulation, puzzle, strategy, and role-playing games [11]. Both traditional games, such as Pacman and Pong [9], as well as modern role-playing games (e.g. World of Warcraft) [10] have been integrated with brain-control. Additionally, the passive use of BCI to detect and evaluate the players’ cognitive activity during game-play has also been explored [12]. Passive control can be used to create an additional information channel between the user and the game, allowing the software to adapt the experience according to the user’s current brain activity. This is beneficial, as adapting the game to the user’s affective state allows for adjustments to the flow of information, providing a more effective and pleasant experience for the player [6]. Other researchers have made use of motor imagery-based BCIs to create video games such as Pinball [16] and a two-player football game [5].

Brain-controlled drones is a concept that has existed for almost a decade with studies dating back to 2010 [1]. More recently, the sport of drone racing has increased its popularity as it has been on mainstream television (ESPN) since 2016. Integrating drone racing with BCI technology allows for the creation of a universal sport, in which all participants compete fairly, independent of body type, gender, or disabilities; such brain-drone racing competitions have been implemented at the University of Florida [17], University of South Florida [20], and University of Alabama [14]. Analogous to racing games that simulate real car racing, it is natural that as brain-drone racing increases in popularity, the trend will follow with simulations. The first brain-drone racing game is described in [19], where the authors describe the game and how the game-play increased the players’ positive affective states. This study differs from the above, as it collects and analyzes qualitative data in regard to users’ perceptions towards BCI, to gain insight into the future of brain-controlled games.

3 Methodology

3.1 Study Design

This study presents a qualitative analysis of an exploratory user study. During the experiment, participants controlled a virtual racing drone in a video-game using a non-invasive 5-channel EEG headset. Each participant attended one session, which lasted approximately 30 min. First, the participant completed a pre-survey containing questions related to demographics, handedness, gaming background, coffee and energy drink consumption, and how many hours participants slept the night prior. With these questions researchers are able to explore possible correlations between these factors and how they participant interacted with the game (e.g. time required to complete the lap). Following, participants completed a training lap around the race track to get familiar with the BCI and the game. Each participant than completed three additional laps, allowing them to experience the game for a longer period. Each lap time was recorded for post-analysis. Additionally, to understand users’ perception towards brain-controlled drone racing games and user experience while playing it, participants were asked to answer the following questions in a post-survey:

  1. 1.

    Do you envision any specific use for BCI devices for gaming?

  2. 2.

    Did you feel uncomfortable during the experiment? Please specify.

  3. 3.

    How much would you pay for a Brain Controlled Game (device and software bundle)?

  4. 4.

    Do you have any additional comments about the experiment or the idea of using BCI devices for controlling video games?

3.2 Procedures

At the beginning of each session, a member of the research team explained the experiment and acquired informed consent from the participant to take part in the study. Following, participants were provided a laptop where they filled-out a pre-questionnaire using Qualtrics Survey software. Following, participants were instructed on how to perform motor imagery and were assisted with wearing the BCI headset; a member of the research team ensured good electrode-skin contact and signal quality before proceeding. Each participant went through a training phase, where electrical activity was acquired during a relaxation (baseline) state and during a motor imagery task. This training phase is necessary by the Emotiv software to later detect and decode player’s brain activity into commands in real-time. At this stage, each participant had a trained profile with two states that were translated into two game commands: accelerate and decelerate the drone. Following this, the participant controlled the drone to perform four laps around the race track. At the end of each lap, the time required to complete that lap was recorded in an electronic table. Lastly, each participant was asked to complete a post-experiment questionnaire and provide feedback through the Qualtrics Survey.

3.3 Participants

A total of 54 participants (33 males and 21 females) were recruited from the University of South Florida to participate in the experiment. From these, 45 participants were between 18 and 24 years old, 8 were between 25 and 34, and 1 participant was between 35 and 44. Additionally, 33 participants stated that they play video-games on a regular basis, while 21 participants do not.

Fig. 2.
figure 2

Emotiv Insight EEG headset used to control the game.

Full size image

3.4 Equipment

The Emotiv Insight brain-computer interface showed in Fig. 2 was used in the experiment. This device is a non-invasive 5-channel EEG headset that reads the electrical activity from the user’s scalp through the use of semi-dry electrodes. The channels are located at the AF3, AF4, T7, T8, and Pz locations according to the 10–20 International System [8]. Additionally, the Emotiv official software suite was used to interface the headset and the game. First, the Emotiv ControlPanel was used to read and decode the brain activity while the participant performed motor imagery into either a neutral(drone hovering) and a command (drone flying forward) state. Following, the Emotiv Emokey software interprets the ControlPanel output (neutral vs command) and emulates keyboard strokes to control the game accordingly to the current participant’s brain activity.

3.5 User Interaction

The brain-controlled drone racing game, displayed in Fig. 3, was developed using the Unity engine and allows players to control the drone avatar using their brain-activity. To interact with the game, players perform a motor imagery task. In other words, they must imagine a muscle movement without physically performing it. Although the current game version offers different tracks, the one used in this experiment allows the participant to control the speed of the drone only, and not the direction in which the drone flies. This approach was used to facilitate the game control, as users are not familiar with controlling a video-game with their brain-waves. Instead, the drone follows a pre-programmed path, while the player can accelerate the drone by performing motor imagery or decelerate by not imagining muscle movements. In this game, the player’s objective is to increase their focus on the motor imagery task, this allows the drone to fly faster and complete laps in the shortest time possible.

4 Results and Discussion

4.1 User Perception Towards BCI in Gaming

Overall, participants provided positive feedback regarding the concept of brain-controlled video games. Many expressed highly enthusiastic responses after the experiment. Additionally, many participants also stated that they believe BCIs will become popular in the gaming industry. Following are examples of the feedback (quotes) provided by players:

  • Very interesting and the game was fun!

  • It was an enjoyable experience, I can see this taking off in a decade or so from now similar to how VR games came into existence.

  • It is very interesting, I think it is the future of gaming.

  • I believe it is incredible and very accurate. This kind of technology could open the doors for VR or AR-Brain Controlled interactions and many different experiences, not just in gaming.

  • It was a lot of fun!

  • It was a good game, and using BCI for video games can become very popular.

  • The game was impressive.

  • I’m not much of a video game player, but it seems to be as attractive as VR.

  • I thought it was fun and interesting to use your brain to control devices.

Fig. 3.
figure 3

Brain-controlled drone racing game.

Full size image

The excitement demonstrated in these responses indicates that brain-computer interfaces have the potential to become the next generation game controller. Moreover, only 4 out of 54 participants said they would not buy a brain-controlled game, and some added comments such as “I would definitely buy one if the technology got to a point where it was very accurate and reasonably priced”. As shown in Fig. 4 and Fig. 5, most participants stated that they would pay between $200 and $500 US dollars for a gaming bundle (device + software). One possible shortcoming for the technology is that brain-controlled games require the use of a BCI headset, which can possibly cause discomfort to players. However, in this experiment only 1 out of 54 participants expressed physical discomfort while wearing the BCI headset, suggesting that discomfort will not be a problem for most players. Combined, the above results show that BCIs can be used for gaming purposes and that users have a positive perception towards this concept. BCIs will lead to a new game modality, providing a different user experience in which they are not only entertained, but can also exercise their brain.

4.2 Future of Gaming BCI

Gaming Genres. Participants were asked if they envisioned any specific use of BCIs in the gaming industry. Sixteen participants stated that they would like to see BCIs integrated with specific game genres. The most commonly mentioned genre was first person shooting games mentioned by 6 players, followed by racing (n = 4), Massive Multiplayer Online Role-Playing Games (MMORPG) (n = 3), and puzzle games (n = 2). Other game genres were mentioned by a single participant each: fighting and combat, aircraft simulators, strategy, and adventure games. The variety of responses demonstrate that users expectations towards brain-controlled games goes beyond drone racing. Additionally, it also serves as a guide for future researchers, specifying which game genres players are most interested in. Future studies can further explore some of the more popular genres mentioned. This would include focus group studies to discuss with users in more detail what they look for in their preferred games.

Fig. 4.
figure 4

Monetary value all participants would pay for a brain-controlled video game.

Full size image
Fig. 5.
figure 5

Monetary value participants would pay for a brain-controlled video game by gender.

Full size image

Virtual Reality. Participants expressed their desire to see brain-computer interfaces integrated with virtual reality (VR) technologies. Various of them stated that BCIs integrated with the immersion provided by VR would create enjoyable user experiences. Their feedback can be summarized by the statement of a participant: “It would be really cool to be able to use brain computer interface devices along with virtual reality to create a more immersive gaming experience.” The proliferation of virtual reality technologies in the last decade has increased the development of VR games. Consumer devices such as the Oculus’ Rift and GO, as well as the HTC Vive, allow users to immerse themselves into virtual environments while remaining at home. Currently, BCI-VR integration is possible by combining these VR headsets with BCIs. Additionally, there are few dedicated headsets providing both BCI and VR capabilities (e.g. Neurable DK1), however, the options are limited. In the near future, it is expected to see a variety of new BCI devices incorporated into VR headsets thus giving rise to a new gaming experience. Researchers can explore the immersive capabilities of VR, coupled with the novelty and challenging aspects of controlling games with brain waves to create new user experiences, which is an appealing idea to the gaming community. Furthermore, due to the current popularity of VR games, researchers can also use its integration with BCIs to increase the popularity of brain-controlled games.

Increased Accessibility. Participants also provided their remarks on how this technology can be used to include people with severe physical disabilities in the gaming hobby. They remarked that BCIs would allow players with paralysis and other types of disabilities to play games, which would not be viable using standard game controllers. Therefore, inclusivity is a main benefit of brain-controlled games as players can control games using their brain-waves without muscle movements, even users with severe physical disabilities can experience games that they may not otherwise be able to. Furthermore, this type of game allows every player to compete at an equal level, regardless of whether or not they have physical disabilities, another advantage not provided by other types of control modalities. In addition to these ethical considerations, according to [4] there are also financial and legal reasons why accessible games are beneficial. Future studies conducted with brain-drone racing games and users with physical disabilities such as upper limb differences could give further insight into the requirements necessary for BCI games to be successful in regard to accessibility.

Improving Attention. Participants reported an increased level of focus while playing the game with comments such as “It actually allowed me to focus which made me feel good”. They also commented that they would like to see BCI games specifically designed to help improve attention in the future. These responses support the claim that brain-controlled games can be useful as a form of brain exercise, where users can play the games while exercising their abilities to focus on motor imagery tasks. Furthermore, such comments are aligned with the finding from [19], where the authors found that brain-controlled games increase attention levels of participants. This characteristic of brain-controlled games can be further explored to create brain exercises. Future research to evaluate brain-controlled game capability as a fun form of therapy for ADHD users is suggested, such as long-term studies measuring the effects of BCI games on user concentration.

5 Conclusion

This paper presents a brain-controlled drone racing game and the results of a qualitative study performed with 54 participants. The qualitative analysis suggests that users feel excited about BCI technology incorporation with video games, and that there is a positive future for their commercialization and use. The majority (50 out of 54) of participants stated that they would purchase a brain-controlled game, and most of them would pay between $200 and $500 for it. The findings also demonstrate that participants would like to see the integration of brain-controlled games with virtual reality. Additionally, the most desired brain-controlled game genres are first-person view shooting, followed by racing and MMORPG. Users also expressed their belief that BCIs have the potential to increase entertainment in gaming as well as further physical disability inclusion initiatives in the gaming industry, as they allow users to experience games with only the use of their brain waves.