Evaluation of STEM digital educational games using an instrument that integrates technology acceptance and player experience factors
Authors: 
Husna Hafiza R.Azami, Roslina Ibrahim, Suraya Masrom, Rasimah Che Mohd Yusoff, Suraya YaacobAuthors Info & Claims
Article No.: 12, Pages 1 - 11
Abstract
Digital educational games (DEGs) are gaining a lot of recognition in a variety of educational and training contexts, including science, technology, engineering, and mathematics (STEM). Since modern-day children are accustomed to digital and gaming technologies, DEGs are considered to be a progressive and effective learning strategy that appeals to them. However, despite the enormous efforts and financial investments made by game academics and producers, students’ acceptance of this learning tool is still low. There is also a lack of studies that provide an understanding of students’ perceptions and acceptance of STEM DEGs. Furthermore, existing research normally conducts the assessment of game acceptance and player experience (PX) separately even though both are crucial assessments that can be integrated and measured simultaneously in a comprehensive manner. Hence, this paper presents the evaluation of three DEGs related to STEM namely RoboCo, Poly Bridge 2, and Moonbase Alpha using a questionnaire that extensively combines important user acceptance and PX components. The proposed DEGAPX instrument which consisted of 13 factors measured by 64 items had passed the reliability and validity testing. The descriptive statistics of the 281 valid responses received from high school students in Malaysia and the interview results with ten survey participants were discussed in this paper. Students’ views on the games were generally positive and their intention to use the games was higher when more factors of the DEGAPX instrument received high scores. This paper highlights the design criteria that can be considered by game developers for creating DEGs that satisfy the needs of target users. Educators can also utilize the suggestions discussed in this paper for the selection of games and deployment in classrooms.
References
[1]
A. H. Hussain, N. M. Sahar, W. M. Din, Z. Mahadi, and K. Chandru, “Using Space Science as a Tool To Promote STEM Education to High School Students in Malaysia,” in Proceeding of the 2019 6th International Conference on Space Science and Communication (IconSpace), 2019, no. July, pp. 28–30.
[2]
A. Joyce and M. Dzoga, “Science, technology, engineering and mathematics education - overcoming challenges in Europe,” 2012. [Online]. Available: http://www.ingenious-science.eu/c/document_library/get_file?uuid=3252e85a-125c-49c2-a090-eaeb3130737a&groupId=10136
[3]
H. H. R.Azami and R. Ibrahim, “Integration of Player Experience for the Acceptance Model of Science, Technology, Engineering and Mathematics Digital Educational Games,” in 2022 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), 2022, pp. 622–629.
[4]
R. Power, Technology and the Curriculum: Summer 2019. Lethbridge, AB, Canada: Power Learning Solutions, 2019. [Online]. Available: https://techandcurr2019.pressbooks.com/
[5]
F. Dahalan, N. Alias, and M. S. N. Shaharom, Gamification and Game Based Learning for Vocational Education and Training: A Systematic Literature Review, no. 0123456789. Springer US, 2023.
[6]
S. Palma, L. A. Ripamonti, N. A. Borghese, D. Maggiorini, and D. Gadia, “Player behaviour metrics for adjusting content in VR games: The case of fear,” in CHItaly 2021: 14th Biannual Conference of the Italian SIGCHI Chapter (CHItaly ’21), 2021, pp. 1–6.
[7]
H. H. Razami and R. Ibrahim, “Distance Education during COVID-19 Pandemic: The Perceptions and Preference of University Students in Malaysia Towards Online Learning,” Int. J. Adv. Comput. Sci. Appl., vol. 12, no. 4, pp. 118–126, 2021.
[8]
I. Voulgari, K. Lavidas, V. Komis, and S. Athanassopoulos, “Examining Student Teachers’ Perceptions and Attitudes towards Game Based Learning,” in International Conference on the Foundations of Digital Games, 2020.
[9]
E. Kutsch, S. Jagger, and H. Siala, “Cultural influences moderating learners’ adoption of serious 3D games for managerial learning,” Inf. Technol. People, vol. 33, no. 2, pp. 424–455, Jan. 2019.
[10]
G. Petri, C. G. von Wangenheim, and A. F. Borgatto, “Quality of games for teaching software engineering: an analysis of empirical evidences of digital and non-digital games,” Proceedings of the 39th International Conference on Software Engineering: Software Engineering and Education Track. IEEE Press, Buenos Aires, Argentina, pp. 150–159, 2017.
[11]
H. Hafiza Razami and R. Ibrahim, “Models and Constructs to Predict Students’ Digital Educational Games Acceptance: A Systematic Literature Review,” Telemat. Informatics, vol. 73, no. 2022, p. 101874, 2022.
[12]
A. Dillon and M. G. Morris, “User Acceptance of Information Technology,” Inf. Today, vol. 31, no. January 2001, pp. 3–32, 1996.
[13]
V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, “User Acceptance of Information Technology: Toward a Unified View,” Source MIS Q., vol. 27, no. 3, pp. 425–478, 2003.
[14]
K. Wan, V. King, and K. Chan, “Examining flow antecedents in game-based learning to promote self-regulated learning and acceptance,” Electron. J. e-Learning, vol. 19, no. 6, pp. 531–547, 2021.
[15]
Y. Wang, Y. Wang, and S. Jian, “Investigating the Determinants of Students’ Intention to Use Business Simulation Games,” J. Educ. Comput. Res., vol. 58, no. 2, pp. 433–458, 2020.
[16]
R. Ibrahim, S. Masrom, R. C. M. Yusoff, N. M. M. Zainuddin, Z. I. Rizman, and M. Sciences, “Student Acceptance of Educational Games in Higher Education,” J. Fundam. Appl. Sci., vol. 9, no. 3S, pp. 809–829, 2017.
[17]
C. Udeozor, F. Russo-Abegão, and J. Glassey, “Perceptions and factors affecting the adoption of digital games for engineering education: a mixed-method research,” Int. J. Educ. Technol. High. Educ., vol. 20, no. 1, 2023.
[18]
H. H. Razami, R. Ibrahim, and N. M. Aljuaid, “A Review of Models and Factors for Digital Educational Games Acceptance and User Experience,” ISMSIT 2022 - 6th Int. Symp. Multidiscip. Stud. Innov. Technol. Proc., no. 21, pp. 37–42, 2022.
[19]
A. H. Allam, A. Razak, and C. Hussin, “User Experience: Challenges and Opportunities,” J. Res. Innov. Inf. Syst., pp. 28–36, 2009, [Online]. Available: http://seminar.spaceutm.edu.my/jisri/download/F_FinalPublished/Pub20_UserExperienceChallenges.pdf%5Cnhttp://seminar.utmspace.edu.my/jisri/download/F1_FinalPublished/Pub4_UserExperienceChallenges.pdf
[20]
M. H. Phan and J. R. Keebler, “The Development and Validation of the Game User Experience Satisfaction Scale (GUESS),” Hum. Factors Ergon. Soc., vol. 58, no. 8, pp. 1217–1247, 2016.
[21]
V. Nagalingam, R. Ibrahim, and R. Che Mohd Yusoff, “EDUGXQ: User Experience Instrument for Educational Games’ Evaluation,” Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 1, pp. 562–569, 2020.
[22]
F. Fu, R. Su, and S. Yu, “EGameFlow: A scale to measure learners’ enjoyment of e-learning games,” Comput. Educ., vol. 52, no. 152, pp. 101–112, 2009.
[23]
K. Cagiltay and S. Yeni, “A heuristic evaluation to support the instructional and enjoyment aspects of a math game,” Progr. Electron. Libr. Inf. Syst., vol. 51, no. 4, pp. 406–423, Jan. 2017.
[24]
R. F. DeVellis, Scale development: Theory and applications. SAGE Publications, Inc., 1991. [Online]. Available: https://us.sagepub.com/en-us/nam/scale-development/book269114
[25]
M. Castillo-montoya, “Preparing for Interview Research: The Interview Protocol Refinement Framework,” Qual. Rep., vol. 21, no. 5, pp. 811–831, 2016.
[26]
J. C. Nunnally, Psychometric Theory. New York: McGraw-Hill, 1978.
[27]
S. Elo and H. Kyngäs, “The qualitative content analysis process,” J. Adv. Nurs., vol. 62, no. 1, pp. 107–115, 2008.
[28]
B. Luyt, Y. R. Guo, and D. H.-L. Goh, “Tertiary students’ acceptance of a game to teach information literacy,” Aslib J. Inf. Manag., vol. 69, no. 1, pp. 46–63, Jan. 2017.
[29]
G. Sideris and S. Xinogalos, “Py-Rate Adventures: A 2D Platform Serious Game for Learning the Basic Concepts of Programming With Python,” Simul. Gaming, vol. 50, no. 6, pp. 754–770, 2019.
[30]
Y. Tao, C. Cheng, and S. Sun, “What influences college students to continue using business simulation games? The Taiwan experience,” Comput. Educ., vol. 53, no. 3, pp. 929–939, 2009.
[31]
G. Petri, C. G. von Wangenheim, and A. F. Borgatto, “MEEGA+: A Method for the Evaluation of Educational Games for Computing Education,” 2018.
[32]
Y. Huang, “Exploring students’ acceptance of educational computer games from the perspective of learning strategy,” Australas. J. Educ. Technol., vol. 35, no. 3, pp. 132–149, 2019.
[33]
F. Strada and A. Bottino, “Investigating the Design and Evaluation of Educational Games Under the Perspective of Player Experience,” in Lecture Notes in Computer Science, vol. 10653, Cham: Springer, 2017.
[34]
J. Moizer, “An approach to evaluating the user experience of serious games,” Comput. Educ., vol. 136, no. April, pp. 141–151, 2019.
[35]
M. Ashfaq, Q. Zhang, A. U. Zafar, M. Malik, and A. Waheed, “Understanding Ant Forest continuance: effects of user experience, personal attributes and motivational factors,” Ind. Manag. Data Syst., vol. 122, no. 2, pp. 471–498, 2022.
[36]
Y. M. Huang, “Students’ Continuance Intention Toward Programming Games: Hedonic and Utilitarian Aspects,” Int. J. Hum. Comput. Interact., vol. 36, no. 4, pp. 393–402, 2020.
[37]
Y. Wirani, T. Nabarian, and M. S. Romadhon, “Evaluation of continued use on Kahoot! as a gamification-based learning platform from the perspective of Indonesia students,” Procedia Comput. Sci., vol. 197, no. 2021, pp. 545–556, 2022.
[38]
E. Park, J. Han, K. J. Kim, Y. Cho, and A. P. del Pobil, “Student Acceptance Model of Educational Games in University Class,” in Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication, 2018.
Index Terms
- Evaluation of STEM digital educational games using an instrument that integrates technology acceptance and player experience factors
Index terms have been assigned to the content through auto-classification.
Recommendations
Entertaining Education --- Using Games-Based and Service-Oriented Learning to Improve STEM Education
Edutainment '09: Proceedings of the 4th International Conference on E-Learning and Games: Learning by Playing. Game-based Education System Design and DevelopmentThis paper addresses the development of a computer game design and development curriculum at the authors' institution. The basis for curriculum decisions, as well as comparison to the other institutions' curricula is covered. In situating the curriculum ...
Entertaining education – using games-based and service-oriented learning to improve STEM education
Transactions on Edutainment IIIThis paper addresses the development of a computer game design and development curriculum at the authors' institution. The basis for curriculum decisions, as well as comparison to the other institutions' curricula is covered. In situating the curriculum ...
Models and constructs to predict students’ digital educational games acceptance: A systematic literature review
Highlights- Technology acceptance model was commonly adopted to predict digital educational game acceptance among students.
AbstractIn recent years, the growth of digital game technologies has ushered a new paradigm in education. Researchers and practitioners have expressed interest in the potential benefits of digital educational games (DEG) toward improving ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2023
416 pages
ISBN:9798400708060
DOI:10.1145/3605390
Copyright © 2023 ACM.
Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 20 September 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Ministry of Higher Education Malaysia
Conference
CHItaly 2023
CHItaly 2023: 15th Biannual Conference of the Italian SIGCHI Chapter
September 20 - 22, 2023
Torino, Italy
Acceptance Rates
Overall Acceptance Rate 109 of 242 submissions, 45%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 47Total Downloads
- Downloads (Last 12 months)30
- Downloads (Last 6 weeks)7
Reflects downloads up to 13 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format