Open AccessArticle

Prospective ICT Teachers’ Perceptions on the Didactic Utility and Player Experience of a Serious Game for Safe Internet Use and Digital Intelligence Competencies

Aikaterini Georgiadou

and

Stelios Xinogalos

Department of Applied Informatics, University of Macedonia, GR-54636 Thessaloniki, Greece

Author to whom correspondence should be addressed.

Computers 2023, 12(10), 193; https://doi.org/10.3390/computers12100193

Submission received: 31 July 2023 / Revised: 8 September 2023 / Accepted: 21 September 2023 / Published: 26 September 2023

(This article belongs to the Special Issue Game-Based Learning, Gamification in Education and Serious Games 2023)

Download

Browse Figures

Figure 1
24 DQ Competencies as defined in [<a href="#B2-computers-12-00193" class="html-bibr">2</a>]. "> Figure 2
The environment of the game: (A) Achieved game stage name: Information Collection, (B) Message Box: the text in Greek refers to the information the player receives, (C) Informative messages, (D) Help area: Hints, (E) Information collected. "> Figure 3
The use of the search engine and the resulting websites: (A) Search keywords options from top to bottom: Dog search, Lost dog, Found pets, (B) Resulted websites after search: the text in Greek represents sites names. (C) Help area: Hints. "> Figure 4
The malicious pop-up window: (A) Pop-up window with terms to be accepted or not: the text in Greek refers to the message body of the terms the player is asked to accept or not, (B) Help area: Hints. "> Figure 5
The post options: (A) Post option for the player to choose from: the text in Greek refers to the context of each ad, (B) Help area: Hints. "> Figure 6
The communication with other internet users: (A) Message box for internet user Max22: the text in Greek refers to the communication being held between the player and the internet user, (B) Message box for internet user Michelle: the text in Greek refers to the communication being held between the player and the internet user, (C) Help area: Hints, (D) Social media web page: the text in Greek refers to a translation of the web page in Greek. "> Figure 7
Returning the dog to its owner: (A) Help area: Hints. "> Figure 8
The deletion of the internet post: (A) Post deletion button: Delete Post, (B) Help area: Hints, (C) Social media web page: the text in Greek refers to a translation of the web page in Greek. "> Figure 9
Serious Game Design Assessment Framework [<a href="#B6-computers-12-00193" class="html-bibr">6</a>]. "> Figure 10
Coherence & Cohesion between SGDA elements for “Follow the Paws”. "> Figure 11
Results Map. ">

Versions Notes

Abstract

Nowadays, young students spend a lot of time playing video games and browsing on the Internet. Using the Internet has become even more widespread for young students due to the COVID-19 pandemic lockdown, which resulted in transferring several educational activities online. The Internet and generally the digital world that we live in offers many possibilities in our everyday lives, but it also entails dangers such as cyber threats and unethical use of personal data. It is widely accepted that everyone, especially young students, should be educated on safe Internet use and should be supported on acquiring other Digital Intelligence (DI) competencies as well. Towards this goal, we present the design and evaluation of the game “Follow the Paws” that aims to educate primary school students on safe Internet use and support them in acquiring relevant DI competencies. The game was designed taking into account relevant literature and was evaluated by 213 prospective Information and Communication Technology (ICT) teachers. The participants playtested the game and evaluated it through an online questionnaire that was based on validated instruments proposed in the literature. The participants evaluated positively to the didactic utility of the game and the anticipated player experience, while they highlighted several improvements to be taken into consideration in a future revision of the game. Based on the results, proposals for further research are presented, including DI competencies detection through the game and evaluating its actual effectiveness in the classroom.

Keywords:

serious games; digital intelligence; DQ Institute; safe internet use; SGDA framework; evaluation

1. Introduction

The current period of time is distinguished by rapid technological evolution, unequal access to modern Information and Communication Technologies (ICT) around the world and a constantly declining time for educating people to use technology properly. Consequently, a gap is being created, between the digital innovations and the knowledge that is required to handle them. This gap, combined with the need for quick transition of primary and secondary school students to the digital world due to the COVID-19 pandemic, resulted in increased cyber risks. The COVID-19 pandemic has caused rushed digitalization of primary and secondary student education, bringing even more in the forefront the younger ages and cyber-risks—such as bullying, technology addiction, and misinformation—that must be addressed [1].

It is important for children to cultivate digital skills, or else to develop their Digital Intelligence (DI), in order to adapt to the new requirements of the digital world and protect themselves from underlying threats. DI, as defined by the DQ (Digital Intelligence Quotient) Institute [2], stands for a comprehensive set of technical, cognitive, meta-cognitive, and socio-emotional competencies that are grounded in universal moral values, which enable individuals to face the challenges and harness the opportunities of digital life. Thus, individuals equipped with DQ become wise, competent, and future-ready digital citizens who successfully use, control, and create technology to enhance humanity.

On the other hand, serious games (SGs) are referred to as entertaining tools with a purpose of education, where players cultivate their knowledge and practice their skills through overcoming numerous hindrances during gaming [3]. Therefore, SGs provide the chance to create a simulated environment for players to experience situations that are impossible to achieve in the real world for reasons such as safety and cost [4].

SGs have proved to be useful in encouraging attitude change, in supporting the development of critical thinking, and in problem solving and developing decision-making skills [5]. Given the above, a SG within the prism of the digital world will not only educate and empower children’s digital skills, but will also entertain and give them the ability to recognize and curb real life risks.

In this paper, we present the design and evaluation of the game “Follow the Paws”, which aims to educate primary school students on safe Internet use and support them in acquiring relevant DI competencies. The game was designed by taking into account the Serious Game Design Assessment Framework proposed by Mitgutsch and Alvarado [6] and was evaluated by 213 undergraduate Informatics students attending a “Didactics of Informatics” course for prospective ICT teachers. The participants playtested the game at their homes and evaluated it through an online questionnaire based on the relevant literature [7,8,9]. The aim of this pilot evaluation was to investigate the potential of this game prior to using it in the classroom. The following research questions were investigated:

RQ1: How do prospective ICT teachers evaluate the didactic utility of “Follow the Paws” as a tool for educating primary school students on safe internet use and relevant digital intelligence competencies?

RQ2: What are prospective ICT teachers’ expectations of primary school students’ experience when playing “Follow the Paws”?

The rest of the paper is structured as follows. In Section 2 a brief literature review is presented. In Section 3 the design of “Follow the Paws” is presented, while in Section 4 materials and methods are briefly described. This is followed by the results of the study and a discussion in Section 5 and Section 6, respectively. The limitations of the study and plans for future work are presented in Section 7.

2. Literature Review

2.1. Digital Intelligence

DI, as defined by the DQ Institute, refers to a person’s ability to adapt to the modern digital world and to act consciously and responsibly within it, using digital media as tools to achieve their goals and improve their life quality. Digitally intelligent people can be defined as individuals able to effectively and flexibly manage advanced technology, as well as successfully collaborate with others in the digital world. Social, emotional and cognitive adaptation to digital reality are the main axes of DI.

To structure DI and embed digital competencies in a universal standard that can be used across organizations, incorporating the digital literacy, digital skills, readiness and flexibility, a global framework was required. The DQ Institute met this challenge by integrating over 25 leading digital proficiency frameworks worldwide, creating the DQ Framework.

In Figure 1, the 24 digital competencies that are held within the framework are presented in tabular form, where in the vertical axis, the three levels of DQ understanding and mastery are defined, and on the horizontal, the eight DQ areas.

In a closer view, this framework is structured around two categories: the eight areas of digital intelligence it examines and the three levels that govern them. These eight areas and three levels form the matrix of 24 digital intelligence competencies (Figure 1).

These eight domains and three levels form the matrix of 24 digital intelligence competencies (Figure 1).

2.2. Advantages of Serious Games

Although the range of applications and usability of SGs may suggest a range of benefits, in reality there is limited evidence to support the benefits of using SGs. Despite this, one of the advantages of SGs, according to Corti [4], is their ability to simulate environments, systems, situations, etc., allowing players to experience situations that might have been impossible to experience in real-world conditions. In this way, risks involved in various activities and behaviors can be avoided, while at the same time allowing players to make their own decisions, having as much time as they need, without the pressure of real life. Also, as pointed out, simulation games have been found to include use of metacognition and mental models, improved strategic thinking and insight, better psychomotor skills, and development of analytical and spatial skills, iconic skills, visual selective attention, computer skills, etc. [5]. Moreover, the work of Blumberg and Ismailer [10] attests, through an extensive body of research, the efficacy of digital games for promoting players’ learning. Finally, through the use of SGs, the participation of even the most timid students is being encouraged and, as empirically supported by Robertson and Miller [11], the less able children tended to be even more availed.

2.3. Relevant Educational Games

SGs development is attracting increased attention, leading to the creation of many educational games in the field of internet safety. Such a game is Google’s Interland (Be Internet Awesome), a 3D game that aims to help students to be safe and successful citizens in our networked world. The concepts taught are anti-bullying, strong passwords, being careful what you post online, and phishing detection [12]. Another SG around cyber safety and digital citizenship is “Pledge Planets”, a game within Messenger Kids launched by Meta (formerly Facebook) that helps kids learn and practice how to make healthy decisions online, stay safe and build resilience [13], aiming to develop their digital citizenship identity. In addition, Minecraft’s “CyberSafe: Home Sweet Hmm”, introduces fundamental cyber safety principles and demonstrates ways to stay safer online [14]. In DI, however, there is limited research in means of measure and detection, although the DQ Institute created a questionnaire that evaluates the digital citizenship level of individuals, which is called the Digital Citizenship Test (DCT). The DCT is an online/mobile test introduced to establish new ethical standards for individuals, particularly children and teens, by modeling for them the characteristics of a responsible digital citizen, in order to better protect them from threats posed by the Internet [15].

The aforementioned SGs and questionnaire aim to pursue internet safety and digital citizenship (digital intelligence life skills) separately and each addressing specific purposes. The SG that is presented in this paper is an attempt to combine, through a storyline, both technology usage and cyber-risk exposure within a realistic environment, proposing a means of addressing unitedly aspects of digital intelligent living.

3. Analysis and Design of Follow the Paws

3.1. Analysis of Follow the Paws

“Follow the Paws” is an SG or, more specifically, an educational game, aiming to consider whether the player can transfer the knowledge from the real world to the digital one and handle cyber risks, along with analyzing the player’s DI skills. The game starts with the main character of the game finding a little dog in his/her neighborhood park, while he/she is out for a walk. The player has to successfully find and return the dog back to its owner. Throughout the game, the main character opposes cyber risks such as phishing, exposure of private details and communication with face profiles on-line, meeting a secondary goal of the game, collecting information about fundamental principles of safe internet browsing. Meanwhile, another contribution of the game is to educate the user by providing informative messages during playing based on his/her actions, concerning safe browsing on the internet and how it can be improved. The main purpose is to either reward and justify correct actions of the user, or offer guidance and hints regarding incorrect practices of online behavior and use. Regarding the social aspect of the game, the core message through the plot is raising awareness on stray animals, while safe browsing and the protection of players from malicious users are at the epicenter of the concept.

“Follow the Paws” is targeted to primary school students as an educational tool on safe internet use, a simulator of real life cyber risks to better prepare students’ response on these types of matters, while, at the same time, can indicate some signs of DI.

The game can be installed on a standalone PC and was created using low-poly 3D animations and Third-to-First Person Cameras, in order to change the perspective along play and place even greater emphasis on the correlation of behaviors and actions of the real world with the digital one. The play time depends on the player and its perception of the game’s purpose. During the game, if needed, the player has at his/her disposal aids giving hints to continue to the next steps and complete the game. The only limitation in the provision of aids is the time required between the use of one aid to the next one, in order to give the player time to proceed with different solutions. An estimation of the time needed to complete the game is approximately 30 min.

In Figure 2, the main areas of the game board are presented: (A) the game stage progress; and (B) the message box where the dialog that takes place with every interaction is displayed, where the player can interact with other characters or objects he/she encounters by guiding the character near the interactive object and pressing the space button; (C) informative messages upon user’s actions; (D) the help area that displays the remaining time until the next available hint; and (E) the information collected.

The game, and the way that DI is applied in it, is presented in sections based on each one of the game’s stages. The game is divided into six stages, and each stage consists of different tasks that the player must complete. These stages also correspond to some digital areas, goals and skills, through the achievement of which, information can be collected for the detection of the player’s DI. At the same time, the player is asked to apply basic principles of safe internet use in order to protect himself/herself from cyber threats. In this way, it is also possible to collect certain information that can reflect the player’s knowledge and readiness in the area of safe internet use.

3.1.1. Information Collection

In the first stage, the player must try to collect information about the dog from his/her surroundings. This should be performed by checking if the dog is wearing a collar with contact details of the dog’s owner, and by trying to find someone around the park to ask if they have seen anyone looking for a dog.

At this stage of the game there is a combination of digital capabilities being explored, those of Digital Co-Creator Identity and Online Communication and Collaboration. The competencies considered derive from the actions of the player, by trying to simulate a logical act of the real world to the digital one. Specifically, the player alone has to think about approaching the dog and searching its collar as a result of investigating and identifying real problems in the digital world, whereas with the action of asking passers-by if they have seen this dog before or if anyone is looking for it, the ability to interact and cooperate in the digital world is examined.

3.1.2. Internet Search

As a continuation of the first stage, after the collection of information is completed the gameplay is transferred to the house so that the search process continues on the internet, on websites for lost dogs (Figure 3). During the player’s use of the search engine there will be suggestions regarding keywords that the player should select in order to perform the search. The player is asked to choose the most suitable ones, and then select a navigation site among the resulting links. The purpose is to select the most suitable sites based on the titles displayed.

After navigating to several sites, one of them displays a pop-up window in which the player is asked to accept certain terms in order to continue browsing the site (Figure 4). However, the conditions he/she is asked to accept violate player’s privacy, basically it is a form of attack. In order not to fall into the “trap” the player must not accept the terms, or notice at the edge of the pop-up window the exit button to close it.

The digital areas explored at this stage are Digital Literacy and specifically Media and Information Literacy. It examines the player’s ability to search and evaluate the information found based on the chosen keywords, the links to navigate, and use critical thinking. The field of Digital Security is also being investigated, with the aim of managing personal protection in cyberspace (Personal Cyber Security Management), as it is pointed out whether the player can perceive and avoid a potential threat in order to protect his/her personal data.

3.1.3. Ad Posting

In the 3rd stage, since nobody has appeared looking for the dog on the websites visited, the next step is for the main character to create and post an ad about the dog. At this point, the player is asked to create an ad on his/her own containing information and a photo of the dog, as well as some contact details. The whole process is completed as the player chooses from the given options, the most suitable profile for the ad to be posted (Figure 5).

Digital Literacy and Digital Rights are, at this stage, the areas of DI tested. Specifically, Content Creation and Computational Literacy are examined through the player’s ability to combine and create material and information in an effective and creative way online. Privacy Management refers to the user’s ability to discreetly manage personal information online.

3.1.4. Owner Authentication

The 4th stage focuses on how the player will manage the responses received after posting the ad (Figure 6). The player is asked to find out which of the people who answered to the ad is the real owner of the dog and which malicious users that are trying to mislead and take advantage of him/her. The game at the end of this stage leads the player to the real owner, with the use of the informative messages, even if the player follows incorrect actions. The aim of the stage is to put the player through the process of authenticating the owner, so even if he/she does not manage to avoid the cyber risks, the information of the real owner is being given to him/her to be able to continue to the next stage.

Through the process of verifying the real owner, with the help of the information collected in the first stage, the selection of a meeting point and the arrangement of how to recognize the owner there, the domain of Digital Rights is approached. The focus is on Privacy Management, which refers to the player’s ability to discreetly manage the personal information of others online. Also, it is linked to the goal of Digital Safety, with Behavioral Cyber-Risk Management being analyzed through the player’s ability to perceive and manage cybersecurity risks related to interpersonal online behaviors. The third and final area examined at this stage of the game is Digital Communication, based on Online Communications and Collaboration, which derives data through the player’s ability to communicate and collaborate efficiently with others using the internet.

3.1.5. Returning the Dog to Its Owner

In this stage, which follows the authentication of the owner, the selection of the meeting point for delivering the dog takes place (Figure 7). The player must ensure that getting to the meeting point does not require using any means of transport in which the dog cannot be carried. Upon the arrival of the player and the dog to the meeting point, an animation takes place displaying the dog running to its owner, indicating to the player the dog’s real owner.

The digital skill that we examine with this action concerns the Systems analysis and evaluation, in the composition of which the digital skill of Digital Co-Creator Identity is included in a secondary stage. The goal is for the player to connect real-world information, such as not being able to use certain public transportation with the dog, which will help him/her reach areas far from the neighborhood. For this reason, a suitable meeting point must be chosen based on the available means of transportation.

3.1.6. Deletion of the Post & Completion of the Game

Finally, to complete the game, after returning the dog to its owner, the player must consider deleting the post that was uploaded on the previous step (Figure 8).

Digital Communication, in terms of Digital Footprint Management, is the digital area under consideration. In detail, it is being examined whether the player considers deleting the post from the internet, realizing the parameters of keeping a no longer valid post on the internet, as well as some personal details.

3.1.7. Summary of DQ Competencies Covered in the Game

In Table 1, a summary of the DQ competencies covered in each one of the game’s stages and player’s actions is presented, along with the corresponding competency number and the digital maturity level.

3.2. Design of Follow the Paws

Mitgutsch et al. [6] developed a holistic assessment framework, Serious Game Design Assessment Framework (SGDA), that focuses on the cohesiveness among the essential design elements and the coherence in relation to the games’ purpose. Games that are designed with a specific purpose and with the intention to impact the players (purpose-based games) need to conceptualize this purpose in their design process. Following a two-step approach, firstly, the elements that the SGDA consists of, presented in Figure 9, are defined. Secondly, the coherence and cohesiveness is considered based on the overall consistency of the aforementioned design elements with the purpose of the game and the general formation of a confluent whole.

On this basis and in addition to the main aim of “Follow the Paws”, to provide an environment similar to the one of the real world, SGDA was selected. In the following paragraphs, we present how the game applied the framework’s axes.

The Purpose of this game is to offer a life-like environment that represents realistic cyber threats and contains game missions corresponding to specific DQ competencies. As a secondary impact “Follow the Paws” sensitizes the player to the matter of stray and lost animals.
The Content & Information of the game, refers to the data visible to the player, provided by the game or perceived by him/her during gameplay. Such data include supplementary information about the characters of the game, the back story, the time left to use the aid button, the educational messages that the player receives upon his actions during the game, the dialogues between characters, the messages at the completion of each game stage, the data that the player gathers during the gameplay in order to complete each step, etc.
The Game’s Framing, given the young ages of the game’s audience, is highlighted to consist of a basic play literacy when no specific knowledge is required from the player to complete the game. It is of high importance to have in mind that we are looking for the spontaneous and unforced behavior (own will) of the player, during the simulation of a physical condition taken from the real world.
The Mechanics, which govern the game, involve all the choices the player makes in order to achieve the main goal of the game, which is to return the lost dog to its owner. It includes all the actions that the player applies within the operations scope of the game, as well as the time it takes to move from one step to another. What is more is that all these actions contribute to the formation of a profile on the DQ axis, while also weaknesses in player’s behavior can be identified, in terms of safe internet use.
Regarding Fiction & Narrative, the storyline that uses a child and a dog as protagonists has been chosen in order to appeal to children of primary school ages regardless of their gender. Representing the main character in an age close to the age of the game’s target group makes the game even more relatable to the player, bringing the realism in the forefront. In parallel, the ending goal of the game familiarizes the player with the process required to return a lost pet to its owner, in an environment as close to reality as possible.
The Aesthetics & Graphics of “Follow the Paws” consist of animated cartoon figures, graphics with bright colors and music, corresponding to the age group in which the game is addressed and in favor of constructing an image and content interesting for a child.

Analyzing the coherence and cohesiveness between the SGDA elements (Figure 10), the purpose of offering an entertaining and stress free environment, simulating realistic cyber risks and underscoring DI skills is coherent with the core elements of the game. The infrastructure of the game guides the player, without delivering simplistic answers for the player’s next move, reflecting on the ability given to apply his/her own understanding. Moreover, the educational content that is given during the gameplay, based on the player’s activity on the internet, allows him/her to first perform even incorrect actions that, in reality, might place him/her at risk, emphasizing on achieving a realistic but yet educational environment.

On a closer look, a strong connection is formed between the game Mechanics, the Content & Information and the Purpose of the game (I in Figure 10), as the game mechanics encourage the player to act freely, while the content provides guidance and feedback. As a result, the correspondence with the DQ competencies remains unbiased and the education on safe internet use takes place. In addition, another triangle of coherence is held between the Fiction & Narrative, the Framing and the Aesthetics & Graphics (II in Figure 10). As a SG designed for primary school ages, it was important to be applicable to all children without being predetermined by the sex or the former knowledge of the player. This framing is represented through the selection of the Aesthetics & Graphics and the Fiction & Narrative, using bright colors, cartoon-like characters, background and character roles familiar to a child and a linear story that is applicable to real life.

Less distinct connections that are observed among the elements of the game are between the Framing and the Purpose (III in Figure 10) and between the Fiction & Narrative and Purpose (IV in Figure 10). Separately, both cases embrace the secondary purpose of the game by providing a storyline that engages the player in taking action to a situation of a lost dog, and make it possible even for a child to contribute to a real life event like this.

4. Materials and Methods

4.1. Context of the Study

The study took place as an activity in the context of an elective undergraduate course on the “Didactics of Informatics” at the Department of Applied Informatics, University of Macedonia, Greece. This course is part of a “Pedagogical and Teaching Proficiency Program” offered by the Department with the aim of preparing students as prospective Information and Communications Technologies (ICT) teachers in public and private schools. One of the modules in this course refers to game-based learning and serious/educational games. Specifically, the benefits of educational games as tools for supporting the teaching and learning of various fields of Informatics, ways of incorporating educational games in the learning process, educational game design and evaluation frameworks are analyzed and critically discussed with students. Among other issues, the MEEGA+ model for evaluating the quality of educational games [7] and the key criteria for game design proposed by Sanchez [9] were presented to students.

During the spring semester of the academic year 2022–2023, the students attending the course were asked to play-test and evaluate the game “Follow the Paws” after the module on educational games and game-based learning had been completed. An announcement was made and delivered to the institutional email addresses of students through the Learning Management System (LMS) used by the Institution and more specifically the course. Students were provided with a link for downloading the game and a link for the online questionnaire (Google form) and had a whole month for play-testing and evaluating the game.

4.2. Research Questions

The study aimed to investigate prospective ICT teachers’ acceptance of “Follow the Paws” as a tool for educating primary school students on safe internet use and relevant DI competencies, as well as the anticipated player experience of the game’s target group. Consequently, the study aimed at investigating the following research questions:

RQ2: What are prospective ICT teachers’ expectations of primary school students’ experience when playing “Follow the Paws”?

4.3. Participants

“Follow the Paws” was play-tested and evaluated by 213 undergraduate students. The majority of the students were in the 4th year of studies (N = 178, 83.5%), which is actually the year of studies that the course is offered (the program of studies lasts for 4 years). The age of the students was in the range of 19 to 46 years old, while the mean age was 22.2 (sd = 2.34). Finally, 70% of the students were male and 30% female.

4.4. Data Collection and Analysis

Data were collected through an online questionnaire prepared as a google form. The questionnaire was based on the MEEGA+ model for evaluating the quality of educational games [7,8] and the key criteria for game design proposed by Sanchez [9], since both instruments have been validated and heavily used for evaluating educational games. The MEEGA+ model and the accompanying questionnaire investigate two quality factors, namely player experience and perceived short term-learning. In our study, we utilized the questions that refer to player experience appropriately adjusted so as to record the perceptions of prospective ICT teachers on the anticipated experience of primary school students when playing “Follow the Paws” (RQ2). From the various key criteria for serious game design and the accompanying questionnaire proposed by Sanchez [9], we utilized questions that refer to game acceptance, usability and didactic utility that as a whole would help us draw conclusions on the overall didactic utility of the game. All the aforementioned questions were actually statements and the participants expressed their level of agreement in a 5-point Likert scale, where 1 = totally disagree and 5 = totally agree. Finally, the questionnaire included demographics questions, questions on the participants’ game play habits and open source questions on the games’ positive and negative aspects, as well as problems encountered during game play.

5. Results of the Evaluation

In this section the results of the questionnaire on game acceptance and didactic utility (Section 5.1) and the anticipated player experience (Section 5.2) are presented. Before presenting the main results of the questionnaire, we would like to present some results on the participants’ game playing habits and the play-testing process of “Follow the Paws”.

The majority of the participants are game players. Specifically, only 7% of the participants stated that they never play games, 36% of them rarely play games and 57% of them play games at least once a month (with 20% playing daily). Consequently, the vast majority of the participants were still active game players. Moreover, the majority of the participants (55%) finished the game “Follow the Paws” in a time range of 10 to 45 min. The time devoted by the majority of the participants (61%) on play testing (no matter if they finished the game or not) was approximately 15 min, while 30% of them played for approximately 30 min and 7% for 45 min.

5.1. Game Acceptance and Didactic Utility (RQ1)

In Table 2, the results on game acceptance and didactic utility of “Follow the Paws” are presented. The participants consider that the educational content of the game (median = 4) is relevant to its purpose and without errors; fits the characteristics of its target group; and the main principles of safe Internet use. What is more important is that the participants agree that there is a balance between game elements and the achievement of the educational objective of the game (median = 4), which is widely accepted to be a great challenge for educational games [16,17]. The tasks within the game are considered to be appropriate to the educational background of primary school students (median = 4), while the game provides scaffolding features. The majority of the participants agree that the game provides both guidance and adequate help (median = 4) and clear and relevant feedback to students about their choices during game playing (median = 4). As a consequence, prospective ICT teachers believe that the game can improve students’ knowledge on safe Internet use (median = 4) and, as such, can be used as a supplement in the educational process (median = 4).

5.2. Anticipated Player Experience (RQ2)

In Table 3, the results on the anticipated player experience are summarized. The results are quite encouraging, since 26 out of the 29 questions had a median value of 4 and 3 questions a median value of 3. Next, we briefly analyze the results for each dimension of the MEEGA+ model that refers to player experience.

Aesthetics. For the first dimension, the evaluation indicated that the game design is quite attractive for young students (median = 4) and that the text font and colors are well blended and consistent (median = 4).

Learnability. The participants are divided as to whether the students will have to learn a few things before they can play the game (median = 3). However, they agree that learning to play the game will be easy for the students (median = 4) and they will learn to play it very quickly (median = 4).

Operability. Regarding the operability of the game, both questions managed to reach a median of 4 (agree), meaning that the game is easy to play and the rules are clear and easy to understand. However, as will be explained in the results of the open-ended questions, the participants proposed to provide more guidance to the player about the tasks that have to be carried out as the story evolves.

Accessibility. For the accessibility dimension, the participants agree (median = 4) that the fonts used in the game are easy to read and the colors are meaningful.

Error prevention and recovery. The participants agree (median = 4) that the game prevents students from making mistakes, and in case of a mistake, it is easy to recover from it quickly. Nevertheless, the mean value of the answers received ranges from 3.5 to 3.7 and this might be another alert for providing further guidance to the players in the case of unsuccessful actions.

Confidence. The confidence dimension has been evaluated through questions regarding the impression of the players on how easy the game will be and how confident players are that they will learn with this game. For both questions, the median was 4 (agree).

Challenge. Regarding the challenge dimension, the median is 4 as well (agree) for all its items, while the mean ranges from 3.5 to 3.9 and the standard deviation ranges from 0.88 to 1.02. The participants consider that the game is appropriately challenging for young students and provides new challenges at an appropriate pace, but when it comes on to how monotonous the game can become as it progresses, the mean is 3.5 and the responses were not so close to the median (Std.Dev.: 1.02)

Satisfaction. The participants agree (median = 4) that players will have the feeling of accomplishment by completing the tasks based on their own efforts; they will learn from the game and they would recommend it to other students.

Fun. The participants believe that young students will have fun while playing the game, while there are also some features that will make them smile (median = 4).

Focused attention. In this dimension, the answers ranged from indifferent (median = 3) to agree (median = 4). Although the participants believe that the game will capture the attention of the players at the beginning (median = 4), they are divided as to whether the players will lose track of time (median = 3), or forget their immediate surroundings while playing the game (median = 3). This is a clear indication that effort should be made in order to achieve a higher immersion of players.

Relevance. The participants agree (median = 4) that the game is relevant to the interests of young students, it is related to educating them on digital competencies and DI and it is recommended as a teaching method for correct digital behavior.

5.3. Positive and Negative Aspects of the Game

The questionnaire included three open-ended questions on the positive and negative aspects of the game, as well as the problems encountered during game play.

The most positive aspects of the game according to the participants are the graphics, the colors and the overall environment of the game. The game is considered to be interesting, entertaining and interactive. Another strong aspect of the game is its educational nature and at the same time its simplicity.

The participants’ responses regarding the negative aspects of the game and the problems encountered during game playing were at a high degree similar as expected. We must note that 48% of the participants did not report any problems during game playing. The most prominent negative aspect or else problem refers to the guidance offered to players regarding the next steps that they should follow. Several participants mentioned the interaction with the virtual computer in the game and the hitboxes (e.g., buttons that should be pressed) as problematic, which required restrictive precision from the player. Although the game provides help/hints for the next steps, this feature is not available at all time and some participants proposed guiding the player with other ways as well, such as “highlighting with an arrow image the button that should be pressed in the virtual computer” or “a bright outline rectangle enclosing elements in the game that the player can interact with”. The participants proposed several improvements of this type that will be taken into account for improving the game prior to its usage in the classroom. We have to note, however, that some of the interactions that were characterized as vague were conscious design choices and were designed as puzzles for cultivating the players’ problem solving capabilities. Another problem mentioned was the movement of the protagonist of the game with the mouse instead of the arrow keys or WASD, as well as the fact that the mouse cursor was visible in the game environment. Some participants mentioned as a negative aspect the limited number of levels and the duration of the game and proposed adding more levels for covering other aspects of safe Internet use and DI as well. Finally, the participants mentioned some problems with the colliders of specific game objects, as well as problems that might be attributed to bugs. Taking into account the number of the participants (213), we consider the feedback invaluable and it will be utilized for implementing the final version of the game.

6. Discussion

Based on the results of the evaluation by prospective ICT teachers, “Follow the Paws” received, for all dimensions of the questionnaire, a median of 4 (agree), except for the learnability and the focused attention dimensions that received an average of 3.67 and 3.33, respectively. In general, the participants positively evaluated the didactic utility of the game based on close-typed questions adopted by key criteria for game design and the corresponding questions proposed by Sanchez [9], implying the acceptance of the game as a supplement in the educational process. Furthermore, the questions adopted from the MEEGA+ model on the anticipated player experience indicated that the game’s user interface and experience is suitable for primary school students.

However, it should be noted that through the open-type questions commented on the game, there were some interesting points to be noted for further improvements and expansion of the game. These comments included:

Uncertainties on the use of the game by students unfamiliar with the use of computers and social media.
Improvements on the game controls, in order to simplify player’s movement while playing.
Providing the player with an introduction to the game’s purpose, in order to familiarize him/her with the aim of not receiving direct guidance through the game, but only minor hints to encourage their own thinking and problem solving skills.

Of course, in order to draw safe conclusions, the game should be tested also by primary school students themselves, in order to obtain feedback from the game’s main target group on their experience and also assess the impact of the game on acquiring safe Internet use and related DI competencies. Furthermore, the game might be good to be translated to English (currently it supports only Greek) to help expand the use of the game.

7. Limitations and Future Work

The study investigated the acceptance of “Follow the Paws” by prospective ICT teachers in terms of its didactic utility and the anticipated player experience. The participants evaluated the game positively and pointed out various improvements. Since the majority of the participants were active players and were also knowledgeable about game-based learning and the instruments utilized in the evaluation of the game, the results are considered trustworthy. A major limitation of the study lies in the fact that the game was not evaluated by the actual target group, which is primary school students. Consequently, the actual effectiveness of the game on raising awareness on safe internet use and acquiring DI competencies remains to be investigated in the classroom after revising the game based on the recommendations recorded in its pilot evaluation.

“Follow the Paws” could be further enriched with new missions and intermediate steps between stages in order to include even more areas of DI. Other design frameworks could be utilized in this re-design process, such as the Activity Theory-based Model of Serious Games (ATMSG) [18], for achieving even better results in terms of achieving the pedagogical goals of the game. Additionally, it would be of great interest to measure, display and save the results and the progress of the player in the game. Saving the player’s progress and results in a database will give him/her the opportunity of playing the game at any time by continuing from the last saved point and saving his/her progress for further use.

In order to measure and display the progress of the player in the game, a digital skills “map”, which captures the progress of players and their actions throughout the game, could be added. More specifically, the purpose of this map will be to display the in-game performance of the player, based on the digital goals that have been set in advance, during the stages and sub-missions of the game from the beginning to its end. In the form of a polygonal web-map, the proficiency level of the skills under consideration will be determined, based on the distance from the center of the web-map to its endings, as shown in Figure 11.

Author Contributions

Conceptualization, A.G.; methodology, A.G. and S.X.; software, A.G.; validation, A.G. and S.X.; investigation, A.G. and S.X.; resources, A.G. and S.X.; data curation, A.G. and S.X.; writing—original draft preparation, A.G. and S.X.; writing—review and editing, S.X.; visualization, A.G.; supervision, S.X. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

Jackman, J.A.; Gentile, D.A.; Cho, N.J.; Park, Y. Addressing the digital skills gap for future education. Nat. Hum. Behav. 2021, 5, 542–545. [Google Scholar] [CrossRef] [PubMed]
Park, Y.; DQ Institute. DQ Global Standards Report 2019: Common Framework for Digital Literacy, Skills and Readiness. Available online: https://www.dqinstitute.org/wp-content/uploads/2019/03/DQGlobalStandardsReport2019.pdf (accessed on 5 May 2023).
Zhonggen, Y. A Meta-Analysis of Use of Serious Games in Education over a Decade. Int. J. Comput. Games Technol. 2019, 2019, 4797032. [Google Scholar] [CrossRef]
Corti, K. Games-based Learning; a serious business application. PIXELearning 2006, 34, 1–20. [Google Scholar]
Mitchell, A.; Savill-Smith, C. The Use of Computer and Video Games for Learning; Learning and Skills Development Agency: London, UK, 2004; Volume 88, pp. 1397–1399. ISBN 1-85338-904-8. [Google Scholar]
Mitgutsch, K.; Alvarado, N. Purposeful by design?: A serious game design assessment framework. In Proceedings of the International Conference on the Foundations of Digital Games (FDG ’12), New York, NY, USA, 29 May 2012; pp. 121–128. [Google Scholar]
Petri, G.; von Wangenheim, C.; Borgatto, A. MEEGA+: An Evolution of a Model for the Evaluation of Educational Games; Brazilian Institute for Digital Convergence: Florianópolis, Brazil, 2016. [Google Scholar]
Petri, G.; von Wangenheim, C.G. MEEGA+: A Method for the Evaluation of the Quality of Games for Computing Education. In Proceedings of the SBGames, Rio de Janeiro, Brazil, 28–31 October 2019; pp. 28–31. [Google Scholar]
Sanchez, E. Key Criteria for Game Design: A Framework; IFE/Ecole Normale Supérieure: Lyon, France, 2011. [Google Scholar]
Blumberg, F.C.; Ismailer, S.S. What do children learn from playing digital games? In Serious Games: Mechanisms and Effects; Ritterfeld, U., Cody, M., Vorderer, P., Eds.; Routledge: Oxfordshire, UK, 2009; p. 135. ISBN 978-113-584-891-0. [Google Scholar]
Robertson, D.; Miller, D. Learning gains from using games consoles in primary classrooms: A randomized controlled study. In Proceedings of the World Conference on Educational Sciences: New Trends and Issues in Educational Sciences, Nicosia, North Cyprus, 4–7 February 2009; pp. 1641–1644. [Google Scholar]
Hill, W.A., Jr.; Fanuel, M.; Yuan, X.; Zhang, J.; Sajad, S. A survey of serious games for cybersecurity education and training. In Proceedings of the 2020 KSU Conference on Cybersecurity Education, Research and Practice, Kennesaw, GA, USA, 23–24 October 2020. [Google Scholar]
Weitzman, E.M. Teaching Good Digital Citizenship with Pledge Planets, an Intergalactic Journey from Messenger Kids. Available online: https://messengernews.fb.com/2022/01/20/teaching-good-digital-citizenship-with-pledge-planets-an-intergalactic-journey-from-messenger-kids/ (accessed on 5 May 2023).
McCarthy, D. Minecraft: Education Edition Launches a New World to Teach Students About Internet Safety in Honor of Safer Internet Day. Available online: https://news.xbox.com/en-us/2022/02/07/minecraft-education-edition-safer-internet-day-course/ (accessed on 5 May 2023).
DQ Institute. Digital Citizenship Test: Cyber-Risk and Digital Skills Assessment Launch. Available online: https://www.dqinstitute.org/news-post/digital-citizenship-test-cyber-risk-and-digital-skills-assessment-launch/ (accessed on 5 May 2023).
Silva, F.G.M. Practical Methodology for the Design of Educational Serious Games. Information 2020, 11, 14. [Google Scholar] [CrossRef]
Natucci, G.C.; Borges, M.A. Balancing Pedagogy, Emotions and Game Design in Serious Game Development. In Proceedings of the 20th Brazilian Symposium on Games and Digital Entertainment, Gramado, Brazil, 18–21 October 2021; pp. 1013–1016. [Google Scholar]
Carvalho, M.B.; Bellotti, F.; Berta, R.; De Gloria, A.; Islas Sedano, C.; Baalsrud Hauge, J.; Hu, J.; Rauterberg, M. An Activity Theory-based Model for Serious Games Analysis And-conceptual Design. Comput. Educ. 2015, 87, 166–181. [Google Scholar] [CrossRef]

Figure 1. 24 DQ Competencies as defined in [2].

Figure 2. The environment of the game: (A) Achieved game stage name: Information Collection, (B) Message Box: the text in Greek refers to the information the player receives, (C) Informative messages, (D) Help area: Hints, (E) Information collected.

Figure 3. The use of the search engine and the resulting websites: (A) Search keywords options from top to bottom: Dog search, Lost dog, Found pets, (B) Resulted websites after search: the text in Greek represents sites names. (C) Help area: Hints.

Figure 4. The malicious pop-up window: (A) Pop-up window with terms to be accepted or not: the text in Greek refers to the message body of the terms the player is asked to accept or not, (B) Help area: Hints.

Figure 5. The post options: (A) Post option for the player to choose from: the text in Greek refers to the context of each ad, (B) Help area: Hints.

Figure 6. The communication with other internet users: (A) Message box for internet user Max22: the text in Greek refers to the communication being held between the player and the internet user, (B) Message box for internet user Michelle: the text in Greek refers to the communication being held between the player and the internet user, (C) Help area: Hints, (D) Social media web page: the text in Greek refers to a translation of the web page in Greek.

Figure 7. Returning the dog to its owner: (A) Help area: Hints.

Figure 8. The deletion of the internet post: (A) Post deletion button: Delete Post, (B) Help area: Hints, (C) Social media web page: the text in Greek refers to a translation of the web page in Greek.

Figure 9. Serious Game Design Assessment Framework [6].

Figure 10. Coherence & Cohesion between SGDA elements for “Follow the Paws”.

Figure 11. Results Map.

Table 1. “Follow the Paws” stages (column 1) and the corresponding DQ domains (row 1), DQ Competencies (row 2) & Player’s actions (competency number—DQ level).

	Digital Identity	Digital Safety	Digital Security	Digital Communication		Digital Literacy		Digital Rights
	Digital Co-Creator Identity	Behavioral Cyber-Risk Management	Personal Cyber Security Management	Online Communications and Collaboration	Digital Footprint Management	Content Creation and Computational Literacy	Media and Information Literacy	Privacy Management
Information Collection	Dog’s collar check for contact details. (9—Digital Creativity)			Collect information from surroundings. (14—Digital Creativity)
Internet Search			Avoidance of maliculus pop-up window. (4—Digital Citizenship)				Correct use of keywords and site selection, while browsing the internet. (7—Digital Citizenship)
Ad Posting						Efficiently creating the ad by combining the information gathered with the given options (15—Digital Creativity)		Discreet handling of personal information in the ad creation (8—Digital Citizenship)
Owner Authentication		Handling online behaviors and verification of the real owner. (3—Digital Citizenship)		Information collection through online communication and collaboration with internet users. (14—Digital Creativity)				Discreet handling of third-party personal information online. (8—Digital Citizenship)
Returning the Dog to its Owner	Proper selection of a meeting place, for returning the dog to its owner, based on the transport options available. (9—Digital Creativity)
Deletion of the Post & Completion of the game					Deletion of the post from the internet when it is no longer valid. (5—Digital Citizenship)

Table 2. Game acceptance and didactic utility of the game.

Dimension	Statement	Mean	Std.Dev.	Median
Game acceptance	The content is relevant (no errors)	3.6	0.86	4
	The content fits the characteristics of the students (age, prior knowledge, etc.)	4.1	0.73	4
	The content fits the main principles of safe Internet browsing	4.3	0.7	4
Usability	The game provides guidance and adequate help to the students	3.6	1.05	4
Usability	The game provides clear and relevant feedback to students about their choices during game playing	3.8	0.96	4
Didactic utility	The game can be used as a supplement during the educational process	4.1	0.83	4
	The game is suited to the pedagogical objectives of the teacher	4	0.81	4
	The tasks of the students within the game are relevant with their educational background	3.9	0.83	4
	There is a balance between game elements and the achievement of the educational objective	3.9	0.73	4
	Through this game students improve their knowledge on safe Internet browsing	4.1	0.81	4

Table 3. Anticipated player experience.

Dimension	Statement	Mean	Std.Dev.	Median
Usability: Aesthetics	The game design is attractive for young students (interface. graphics., etc.).	4.1	0.64	4
Usability: Aesthetics	The text font and colors are well blended and consistent.	4	0.92	4
Usability: Learnability	Students will have to learn a few things before they can play the game.	3.2	1.02	3
	Learning to play this game will be easy for students.	3.9	0.87	4
	I think that most students will learn to play this game very quickly.	3.8	0.90	4
Usability: Operability	I think that the game is easy to play for young students.	3.8	0.90	4
Usability: Operability	The game rules are clear and easy to understand.	3.8	0.96	4
Usability: Accessibility	The fonts (size and style) used in the game are easy to read.	4.2	0.78	4
Usability: Accessibility	The colors used in the game are meaningful.	4	0.84	4
Usability: Error prevention and recovery	The game prevents students from making mistakes.	3.5	0.99	4
Usability: Error prevention and recovery	When the student makes a mistake it is easy to recover from it quickly.	3.7	0.92	4
Confidence	When the student first looks at the game, s/he will have the impression that it will be easy for him/her.	3.9	0.89	4
Confidence	The contents and structure will help the student to become confident that he/she will learn with this game.	3.8	0.78	4
Challenge	This game is appropriately challenging for a young student.	3.9	0.91	4
	The game provides new challenges (offers new obstacles. situations or variations) at an appropriate pace.	3.7	0.88	4
	The game does not become monotonous as it progresses (repetitive or boring tasks).	3.5	1.02	4
Satisfaction	Completing the game tasks will give students a satisfying feeling of accomplishment.	4.1	0.84	4
	Managing to advance in the game will be due to the student’s personal effort.	4	0.79	4
	Students will feel satisfied with the things that they learned from the game.	3.8	0.8	4
	I would recommend this game for young students.	4.1	0.85	4
Fun	Students will have fun with the game.	4	0.86	4
Fun	During the game I believe that there are features that would make a young student smile (game elements. competition., etc.).	3.9	0.82	4
Focused Attention	There is something interesting at the beginning of the game that will capture the attention of a young student.	3.8	0.92	4
	A young student could be so involved in the gaming task that would lose track of time.	2.8	1.13	3
	A student could forget about his/her immediate surroundings while playing this game.	2.7	1.08	3
Relevance	The game contents are relevant to the interests of a young student.	3.7	0.84	4
	It is clear to me how the contents of the game are related to the investigation of digital competencies.	4	0.8	4
	This game is an adequate method for investigating digital intelligence.	3.6	0.86	4
	I suggest investigating and teaching students about the correct digital behavior with this game in comparison to learning through other ways (e.g., other teaching methods).	4.1	0.79	4

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Georgiadou, A.; Xinogalos, S. Prospective ICT Teachers’ Perceptions on the Didactic Utility and Player Experience of a Serious Game for Safe Internet Use and Digital Intelligence Competencies. Computers 2023, 12, 193. https://doi.org/10.3390/computers12100193

AMA Style

Georgiadou A, Xinogalos S. Prospective ICT Teachers’ Perceptions on the Didactic Utility and Player Experience of a Serious Game for Safe Internet Use and Digital Intelligence Competencies. Computers. 2023; 12(10):193. https://doi.org/10.3390/computers12100193

Chicago/Turabian Style

Georgiadou, Aikaterini, and Stelios Xinogalos. 2023. "Prospective ICT Teachers’ Perceptions on the Didactic Utility and Player Experience of a Serious Game for Safe Internet Use and Digital Intelligence Competencies" Computers 12, no. 10: 193. https://doi.org/10.3390/computers12100193

APA Style

Georgiadou, A., & Xinogalos, S. (2023). Prospective ICT Teachers’ Perceptions on the Didactic Utility and Player Experience of a Serious Game for Safe Internet Use and Digital Intelligence Competencies. Computers, 12(10), 193. https://doi.org/10.3390/computers12100193

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu