The Impact of Gamification Factor in the Acceptance of Cybersecurity Awareness Augmented Reality Game (CybAR)

While digital technology has enabled innovation, economic growth and productivity, it has also led to a dramatic increase in the number of cyber-attacks. Several recent incidents of cyber-attack resulted in substantial financial losses to a number of organizations [32]. Security professionals and researchers are therefore expending considerable effort to identify effective methods of increasing cybersecurity awareness [14]. In this context, gamification is seen as a promising technology that can be integrated with cybersecurity awareness training to tackle cybersecurity threats. Gamification can be defined as the application of game design principles in non-gaming contexts [35]. The current project was motivated by the lack of research into the use of gamification through mobile augmented reality (AR) techniques as an educational tool about cybersecurity threats, with the aim of raising overall cybersecurity awareness. For this purpose, an AR based game, CybAR, was developed for the Android platform to build cybersecurity awareness. The main features of our AR game are its interactivity and its graphic presentation of the negative consequences of careless cybersecurity behavior. Previous research on AR acceptance factors is scarce, leaving an almost unexplored area of research. The UTAUT2 model was selected for use in the present study since it incorporates the gamification feature construct, thus providing opportunity for new insights into factors affecting the acceptance of new technology. UTAUT2 is an extension of UTAUT, which had been identified as the most comprehensive and predictive technology acceptance model [41, 42]. Therefore, UTAUT2, with the addition of gamification factors, provided the theoretical framework for this investigation of AR application acceptance. Our study follows the authors’ recommendation in [6] that the UTAUT2 model with gamification impact factor needs to be explored. Thus, in this study, gamification was introduced as an independent variable in the conceptual framework (see Fig. 1) to assess its relationship with the intention and use behaviour of the AR app. The data were collected from 95 Macquarie University students in Australia. Using PLS 3.0-SEM, the results showed that the gamification factor had a positive association with behavioral intentions and use behaviour of the CybAR app. Gamification can help to make cybersecurity awareness training more enjoyable and increase users’ perceptions of the acceptance of a technology [42]. To the best of our knowledge, the present study is the first to combine UTAUT2 factors and gamification feature constructs in work on acceptance of an AR app, using data from Australia. Hence it enriches the existing literature on AR app acceptance and provides new insights into how game techniques influence individual behaviour toward risky cybersecurity practices.

2.1 Cybersecurity and Augmented Reality

The United States Army defines cybersecurity as a combination of the underlying hardware, communication nodes, and a social layer of human and cognitive elements [17]. Dhamija et al. [18] reported that hackers could effectively target users due to the public’s general lack of awareness about cybersecurity. Therefore, education is vital to protect users against cyber-attacks. Many organizations provide materials for preventing cyber-attacks, such as email bulletins or educational websites. However, Kumaraguru et al. [25] found that these materials only work if people continue to pay attention to them. Recently, gamification has emerged as a promising technique to improve cybersecurity awareness and increase the effectiveness of cybersecurity measures [22]. Several cybersecurity games have been developed for this purpose to better engage users and change user behavior to avoid cyber-attacks. The majority of these games are browser-based apps, but it may be more effective to employ mobile phone game apps [43]. AR is a recently developed technology that enhances users’ experience by overlaying computational information on to their reality [1, 3]. In spite of the popularity of widespread AR applications, there exists no AR based application that can be used to educate users about cybersecurity attacks and help to raise their cybersecurity awareness. Azuma [5] defined AR as a type of interactive, reality-based display environment that takes the capabilities of computer-generated display, sound, text and effects to enhance the user’s real-world experience. AR combines real and computer-based scenes and images to deliver a unified but enhanced view of the world. AR systems have been used in different fields, such as education, marketing, medicine, and tourism [4, 26]. However, their application in the context of education about a safe cybersecurity behavior has been less explored. Accordingly, we developed an AR game, CybAR, whose purpose is to demonstrate the negative consequences of risky cybersecurity behaviour and the value of safe cybersecurity behavior. Our aim in this study is to identify the role of gamification factor on affecting users’ acceptance of CybAR.

2.2 Unified Theory of Acceptance and Use of Technology (UTAUT)

Previous research in psychology and sociology has generated various theoretical frameworks to explain the relationship between technology acceptance and usage. One of these is the unified theory of acceptance and use of technology (UTAUT). UTAUT [41] was developed through a review and consolidation of the constructs of eight well-known theories that previous researchers had employed to investigate information systems’ usage behavior: the theory of reasoned action (TRA), the technology acceptance model (TAM), the motivational model (MM) [16], the theory of planned behavior (TPB), the PC utilization model (MPCU) [40], innovation diffusion theory (IDT), social cognitive theory (SCT) [29], and an integrated model of technology acceptance and planned behaviour (TAM-TPB) [38]. The model identifies four factors [42]: performance expectancy, effort expectancy, social influence, and facilitating conditions. Since its appearance, the UTAUT model has been widely used to explore user acceptance of mobile technologies [4], and has been tested and applied to several technologies [2, 13]. Even though UTAUT provides a very detailed model, it has some limitations [33]. Therefore, the authors in [42] developed UTAUT2 in 2012, extending and adapting the theory to the consumer context. UTAUT2 now has seven factors: performance expectancy, effort expectancy, social influence, facilitating conditions, hedonic motivation, price value, and habit. Of the three new factors, hedonic motivation was added due to its importance as a key predictor in much earlier research [41], price because in a consumer context users must be able to afford the costs associated with the service use, and habit, because previous studies reported it to be a critical factor in the context of technology use [28]. In this model, the moderating variables are age, gender, and experience; voluntariness from the previous UTAUT is excluded. In our study, we dropped the moderating variables (age, gender, and experience) and the price construct because they were not relevant in our study, and substituted them with gamification impact as a construct variable.

2.3 Gamification

Mobile games are played by millions of adolescents and adults around the world [9]. They are highly diverse in relation to their purposes, interactivity and technology. Quality mobile games have been shown to improve concentration, enhance retention of information, and bring about behavioural change [34]. Over recent decades, mobile games - both serious games and gamification - have been designed for serious purposes: to educate, motivate, and persuade users in health, educational and other settings [11]. Serious games use gaming as the primary medium, whereas gamification involves the addition of game elements to non-game contexts as an innovative technique of influencing the motivation or engagement of people to solve real problems [15], and driving behaviours and producing the desired effect and results [11]. The potential for employing serious games and gamification to educate users about cybersecurity has been under-researched.

A small number of games to promote cyberattack awareness have been developed to better engage learners and change user behavior [22]. It has been claimed that well-designed games for user education can effectively mitigate cyber threats [43]. Examples of such games include CyberCIEGE [39] and anti-phishing gaming tools [31, 37]. Most of these games, however, are technically oriented and, more importantly, are limited to web-based games. Although evaluations of these games have demonstrated an improvement in players’ ability to identify phishing websites, their designs are not particularly effective at teaching players how to detect other forms of cybersecurity threat such as identity theft, ransomware and phishing through social media networks. Accordingly, we incorporated in our game all potential cyberattack techniques and which provides a more engaging experience for the acquisition of practical and conceptual knowledge. In this study, we show the potential effect of employing game mechanics and game design techniques to influence users’ acceptance of an AR app using the UTAUT2 model.

The research model proposed in this study is shown in Fig. 1. The theoretical framework is based on a combination of factors derived from the unified theory of acceptance and use of technology (UTAUT). The UTAUT factors are performance expectancy, effort expectancy, social influence, hedonic motivation, facilitating conditions and habit as predictors of behavioral intention. Behavioral intention is also a predictor of use behaviour. The model also identifies gamification factor as independent variable affecting behavioral intention and use behavior.

Research model.

Full size image

3.1 UTAUT

UTAUT was considered to be the most complete model for predicting information technology acceptance [30] until the development of UTAUT2 (see Table 1). Compared to its predecessor, UTAUT2 shows significant improvement in explained variance in studies of behavioral intention and technology use [42]. It is therefore employed in the present study, which show the relationship between independent variables (performance expectancy, effort expectancy, social influence, hedonic motivation, facilitating conditions and habit) and dependent variables, namely, behavior intention and initial use (adoption behavior), in relation to an AR app.

Based on the results of previous studies, performance expectancy, effort expectancy, social influence, hedonic motivation, facilitating conditions and habit are expected to be the most important factors that directly influence the intention of using the app [42]. Also, previous technology adoption studies mainly focused on behavioral intent without actually assessing initial use. However, recent findings have questioned the strength of the relationship between behavioral intent and use behavior in various contexts [24]. In relation to the gamification factor, technology acceptance has attracted considerable attention in different areas but few studies have applied UTAUT2 with gamification factor included in the model in the context of Augmented Reality and cybersecurity awareness. Thus, the goal of this study was to determine the impact of gamification as a predictor of behaviour intention and use behaviour of the CybAR app. It is our belief that the use of game techniques in a non-game context such as cybersecurity awareness will have a strong impact, increasing the rate of users’ acceptance [6]. Therefore, the more entertainment value the AR app can provide, the greater will be the acceptance intention of users. Therefore, we hypothesised the following:

• Gamification impact will positively affect behavioural intention and use behaviour of the CybAR app.

3.2 Cybersecurity Awareness Game Using Augmented Reality (CybAR)

There are some shortcomings in existing gamified approaches to education about cybersecurity challenges. Thus, our goal was to replace training programs that typically focus on reading about cybersecurity with a serious Augmented Reality game that mimics the actual forms of cybersecurity attacks. One of the main aims of CybAR was to provide more comprehensive education about cybersecurity attacks and to do so in a way that closely matches how they occur in the real world. Another goal of CybAR was to educate a less technically sophisticated audience to increase their awareness of the potential for cybersecurity attacks in their day-to-day online behaviour, rather than to teach specific technical or management skills. Cybersecurity Awareness using Augmented Reality (CybAR) is an AR mobile application game that not only teaches cybersecurity concepts but also demonstrates the consequences of actual cybersecurity attacks through immediate feedback. The CybAR application was developed using a Unity and Vuforia platform (see Fig. 2) and incorporates a commercial AR technique developed for the Android platform. The main characteristics of the AR game are interactivity and the display of the shocking consequences of careless cybersecurity habits. Before players begin a CybAR game, they should download and access an electronic booklet that contains all 20 tasks.

(a) CybAR app. (b) CybAR main page.

Full size image

They can see the AR content for each task by moving the camera on the SCAN ME QR code next to each task to view the answers (see Fig. 3).

(a) Example of CybAR Task 1. (b) Example of CybAR Task 2.

Full size image

Each task requires players to choose the right option for the given scenario. This helps users to increase their cybersecurity awareness. CybAR includes important game elements such as scores, progress, levels and a leaderboard. Our goal was to replace traditional training programs that typically emphasise reading about cybersecurity with an AR game that simulates the actual methods of cybersecurity attacks. For each completed task, we developed responses inspired by protection motivation theory (PMT) [36] to trigger more secure behaviour among users.

• If the user has implemented the task correctly, a coping message appears, stating that it is easy to minimise the risk of cyberattack by making the right decision fir the task (see Fig. 4(a)).

• If the user has implemented the task incorrectly, a fear appeal message warns that their behaviour could leave them vulnerable to cyberattacks (see Fig. 4(b)).

(a) Correct answer feedback. (b) Wrong answer feedback.

Full size image

A total of 95 students played the CybAR game and then completed an online questionnaire hosted on the Qualtrics platform. The study criteria required all participants to be 18 years of age or older with experience using IT assets in a professional capacity. Also, participants need an Android tablet or Android phone. Invitations to participate were distributed to students at Macquarie University via email, social media (Facebook and Twitter) and flyers posted in different locations on campus. All participants received information about the purpose of the game and the nature of the study and gave informed consent.

4.1 Measurements

The questionnaire items have been derived from different literature. The questionnaire contains three parts: UTAUT2 data constructs, gamification questions and general information and demographic characteristics. The items and scales for the UTAUT2 constructs were adapted from [41, 42], the use behaviour from [16] and gamification impact from [6]. In relation to the UTAUT model, the items for each of the constructs (performance expectancy, effort expectancy, facilitating conditions, hedonic motivation, social influence, habit, behavioral intentions and use behavior constructs) were 5, 6, 4, 5, 3, 4, 3, and 2 items for each construct, respectively. There are 3 adapted items for gamification factor. Thus a total of 35 items are used to measure the 9 constructs of our proposed theoretical model (See Table 2). In both UTAUT components and gamification motivation factor, a five-point Likert scale ranging from “extremely disagree” = 1 to “extremely agree” = 5, is used to measure all the items.

Two software tools were employed in data analysis. First, the survey data were recorded by Qualtrics and imported to SPSS. SPSS software is readily available and can be used to generate descriptive statistics and support the process of data analysis. Various analyses were performed using SPSS. Descriptive statistics were used to analyse each variable separately and to summarise the demographic characteristics of participants. Second, SmartPLS Version 3.0 was used for analytics. The Partial Least Squares (PLS) was used in this study as it is a rigorous technique for structural equation modeling (SEM). Before any analyses were conducted, data normality for each measured item was tested for skewness. The skewness values for the constructs were between −3 and +3. This indicated that the items were almost normally distributed, so further calculations were performed, as elaborated below.

5.1 Characteristics of Participants

We received far fewer responses than we had expected. Although the questionnaire link and invitation letter were sent to 300 respondents, and they were asked to pass it on to their friends, only 124 questionnaires were received. After filtering, 29 of these were found to be incomplete. There was a fairly equal distribution of males (56%) and females (44%). Regarding the age groups, the largest group of respondents (39%) was aged 25–34, followed by those aged 35–44 (22%), 18–24 (28%) and 45–54 (8%). Only 3% of participants belonged to the 55+ category. Most respondents were highly educated; 62% were undergraduate university students; 17% were postgraduate students; 16% were enrolled in a 2-year college degree; and 5% were high school students.

5.2 Model Validation

This section describes the assessment and testing of the proposed model using SEM. Because PLS does not provide goodness-of-fit criteria, the procedure for testing PLS was performed in two stages: assessing the reliability and validity of the measurement model; and testing the hypotheses in the structural model.

Measurement Model: The measurement model is evaluated by estimating the internal consistency reliability. The internal consistency reliability is assessed using the values for Cronbach’s alpha, composite reliability and average variance extracted (AVE) [10]. Cronbach’s alpha is a measure of internal consistency that measures the correlation between items in a scale. The Cronbach’s alpha for each construct had to be greater than 0.7 [27]. Composite reliability is similar to Cronbach’s alpha. It measures the actual factor loadings rather than assuming that each item is equally weighted. The standardised path loading of each item should be statistically significant. In addition, the loadings should, ideally, be at least greater than 0.7. AVE indicates the amount of variance in a measure that is due to the hypothesised underlying latent variable. The average variance extracted (AVE) for each construct has to exceed 0.5. Values greater than 0.50 are considered satisfactory. They indicate that at least 50% of the variance in the answers to the items is due to the hypothesised underlying latent variable.

All scales reached a composite reliability value of at least 0.78 (ranging from 0.783 to 0.917). Thus, they exceeded the 0.70 threshold for composite reliability. In addition, the scales exhibited high internal consistency; the lowest Cronbach’s alpha was 0.76, which is well above the 0.70 threshold for confirmatory research. The AVE for each construct was greater than 0.5 (ranging from 0.693 to 0.861). Therefore, the internal consistency reliability for the constructs was confirmed.

Construct validity consists of convergent validity and discriminate validity. Convergent validity is achieved when each measurement item correlates strongly with its proposed theoretical construct. It is checked by testing the factor loadings of the outer model. The outer model loadings for all items are all above 0.50. Therefore, convergent validity was established [19]. Discriminant validity is achieved when each measurement item correlates weakly with all other proposed constructs than the one to which it is theoretically associated. The discriminant validity of the measurement model is tested using two criteria suggested by Gefen and Straub [20]: (1) item loading to construct correlations is larger than its loading on any other constructs; and (2) the square root of the AVE for each latent construct should be greater than the correlations between that construct and other constructs in the model. The lowest acceptable value is 0.50. All items showed substantially higher loading than other factors except 2 items failed the test and were excluded, and the square root of the AVE for each construct exceeded the correlations between that construct and the other constructs. Therefore, discriminant validity was established. Table 3 shows the Discriminant Validity that all the scales used in the survey satisfy the requirements. The square roots of the AVE-s are shown in bold. Off diagonal elements are correlation between constructs. The AVE form the constructs should be greater than the AVE shared between the item and other items in the proposed model.

Structural Model: The structural model was also analyzed using SmartPLS version 3.0 as mentioned above. This proposed model presents 9 hypotheses that were used to examine the relationships between the latent variables. The structural model was assessed by evaluating the path coefficients and coefficient of determination. Path coefficients are explained with the t-statistics computed using bootstrapping 500 samples. The tests point to positive or negative relationships between exogenous constructs and endogenous variables and the strength of these relationships. Path coefficients should be directionally consistent with the hypothesis. Coefficient of determination as R2 values. R2 provides the amount of variance of dependent variables explained by the independent variables. In our analysis, the R2 coefficient of determination indicates the predictive power of the model for each dependent construct. According to [12], an R2 value of 0.67 in the PLS path model is considered substantial. Therefore, our model has the ability to explain the endogenous constructs. The UTAUT model explains 73.6% of variation in behavioural intention and 59.8% in use behaviour. According to the path coefficients and t-test values, we found adequate evidence for each hypothesis. The SEM results revealed that most of the proposed external variables have significant effect on avoidance motivation. Avoidance behaviour has significant influence on avoidance behaviour.

Due to space limitations, this paper only presents the preliminary results of the study, highlighting the gamification factor’s impact on users’ acceptance of the CybAR app. Comprehensive results, including all constructs and their correlations with use behaviour factors, and a summary of the hypothesis testing results will be presented in future work. The gamification was were found to be statistically significant in affecting behavioural intention to use the CybAR app and use behaviour of the CybAR app.

• Consistent with the hypothesis, the gamification factor had a positive influence on students’ behavioural intention to use AR systems, with path coefficient = 0.5 and t = 3.7 (p < 0.01, 1-tail).

• Similarly, use behaviour of the CybAR app was positively influenced by the gamification factor, with path coefficient = 0.41 and t = 2 (p < 0.05, 1-tail), thus supporting the hypothesis.

To the best of our knowledge, this is the first time that UTAUT2 and a gamification construct have been combined in a study of cybersecurity awareness application acceptance. The results indicated that gamification techniques such as points, progress, feedback and leaderboard positively affect the acceptance of the CybAR app. In other words, the results confirm a strong statistical relationship between gamification and behavioural intention as well as between gamification and use behaviour of the CybAR application. These findings are consistent with some previous research [6], but are not supported by others [8].

This study and its results have theoretical and practical implications. Theoretically, this research adds significantly to the existing literature on UTAUT2 and gamification. This study empirical validated and tested the UTAUT2 model and highlighted the importance of gamification constructs to understand behavioural intentions to use the CybAR app and actual use of the CybAR application in a cybersecurity awareness context. In doing so, we have presented theoretical evidence that the gamification factor is a significant predictor of technology acceptance in the cybersecurity awareness context, and this enhances the explanatory power of UTAUT2 For researchers, this study provides a basis for further work on acceptance and gamification in augmented reality applications. Practically, understanding the significant constructs in the design and implementation of cybersecurity awareness applications helps practitioners to achieve high user acceptance. The results of our study clearly indicate that cybersecurity awareness campaigns should incorporate gamification techniques in their design and implementation.

The careful application of gamification in cybersecurity awareness training can help increase individuals’ cybersecurity knowledge about cyber-attacks in an interesting and enjoyable way, in turn increasing user acceptance, engagement and satisfaction. Using the CybAR application, the research extends the unified theory of acceptance and use of technology (UTAUT2) and prior research to include the gamification factor. The theoretical model was tested in a quantitative study using structural equation modelling, conducted in Australia, with 95 Macquarie University students. The findings indicate that there is a significant relationship between the gamification factor and behavioural intention to use the CybAR app and actual use of the CybAR app. The gamification factor positively influenced behavioural intention and use behaviour of the CybAR app, confirming the potential of games techniques to raise cybersecurity awareness. The research model in our study explained 73.6% of variation in behavioural intention and 59.8% in use behaviour.

Several limitations of this research should be noted. First, the study employed a cross-sectional research design. Longitudinal data will enhance our understanding of what constructs affect individuals’ acceptance of using the CybAR app and enable to have more accurate findings from a specific group. Second, only quantitative data were collected in our study. Qualitative data generated from interviews or focus groups could yield insight into other factors that affect individuals’ behavioural intention and use behaviour of CybAR app. Third, interpretation of the results was limited by the small sample size (95). A larger sample would have improved the ability to generalise the findings to a wider population. It should be noted, however, that the use of SmartPLS as a data analysis tool overcomes this limitation since it can generalise results with a very small sample size. Forth, the study was conducted in one university Macquarie University so the results may not be applicable to all Australian universities, even if the education system and culture are the same. Fifth, this study examined by using a marker-based AR application that applicable only for Android devices. Similar application should be developed for iOS users. Finally, not all factors related to the higher education institution were taken into consideration. AR usage in such institutions will be better understood if other factors, such as cultural dimensions and personality traits, are taken into account.

Several scholars have emphasized the importance of integrating cultural dimensions into technology adoption models [7]. Therefore, our future work should apply UTAUT2 with different cultural backgrounds [21]. Also, in information systems research, personality factors have been used in various disciplines [23]. Hence the incorporation of personality traits into models such as UTAUT can reveal how personality influences individuals’ technology acceptance. Thus, in our future work to identify the personality traits that affect users’ acceptance of CybAR and increase their cybersecurity awareness.