Abstract
Industrial-grade games, like Space Engineers, must adopt swift development and testing processes to conform to rigorous quality standards. Nevertheless, the testing phase of these extensive and complex games heavily relies on manual effort from play-testers, leading to productivity constraints during development cycles. This experience paper reports a Behavior-Driven-Development (BDD) software development process for automated regression test scenarios that allows complement testers’ work during development cycles. To enable BDD test scripts for the Space Engineers game, we have extended the iv4XR framework into a game plugin to connect and execute game actions. Additionally, we have integrated the Cucumber software to describe game test scenarios using natural language. This approach allows testers to create, maintain, and execute a subset of regression test scenarios by relying on a BDD agent that can autonomously verify Space Engineers game features, enabling seamless integration into the development cycle.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
de Andrade, S.A., Nunes, F.L., Delamaro, M.E.: Exploiting deep reinforcement learning and metamorphic testing to automatically test virtual reality applications. STVR 33(8), e1863 (2023)
Ariyurek, S., Surer, E., Betin-Can, A.: Playtesting: what is beyond personas. IEEE Trans. Games (2022)
Bahaweres, R.B., et al.: Behavior-driven development (BDD) cucumber katalon for automation GUI testing case CURA and swag labs. In: ICIMCIS, pp. 87–92 (2020)
Binamungu, L.P., Maro, S.: Behaviour driven development: a systematic mapping study. J. Syst. Softw. 111749 (2023)
Bucchiarone, A., Cooper, K.M., Lin, D., Melcer, E.F., Sung, K.: Games and software engineering: engineering fun, inspiration, and motivation. ACM SIGSOFT Softw. Eng. Notes 48(1), 85–89 (2023)
Coppola, R., Morisio, M., Torchiano, M.: Mobile GUI testing fragility: a study on open-source android applications. Trans. Reliab. 68(1), 67–90 (2018)
Dees, I., Wynne, M., Hellesoy, A.: Cucumber Recipes: Automate Anything with BDD Tools and Techniques. Pragmatic Bookshelf (2013)
García, B., Gallego, M., Gortázar, F., Munoz-Organero, M.: A survey of the selenium ecosystem. Electronics 9(7), 1067 (2020)
Gordillo, C., Bergdahl, J., Tollmar, K., Gisslén, L.: Improving playtesting coverage via curiosity driven reinforcement learning agents. In: 2021 IEEE Conference on Games (CoG), pp. 1–8. IEEE (2021)
Hammoudi, M., Rothermel, G., Tonella, P.: Why do record/replay tests of web applications break? In: International Conference ICST, pp. 180–190. IEEE (2016)
Nguyen, V., To, T., Diep, G.H.: Generating and selecting resilient and maintainable locators for web automated testing. STVR 31(3), e1760 (2021)
Nie, L., Said, K.S., Ma, L., Zheng, Y., Zhao, Y.: A systematic mapping study for graphical user interface testing on mobile apps. IET Software 17(3), 249–267 (2023)
North, D., et al.: Introducing BDD. Better Softw. 12, 7 (2006)
Paduraru, C., Paduraru, M., Stefanescu, A.: Rivergame-a game testing tool using artificial intelligence. In: International Conference ICST, pp. 422–432. IEEE (2022)
Pereira, L., Sharp, H., de Souza, C., Oliveira, G., Marczak, S., Bastos, R.: Behavior-driven development benefits and challenges: reports from an industrial study. In: ICASD Companion, pp. 1–4 (2018)
Pezze, M., Rondena, P., Zuddas, D.: Automatic GUI testing of desktop applications: an empirical assessment of the state of the art. In: ISSTA, pp. 54–62 (2018)
Pfau, J., Smeddinck, J.D., Malaka, R.: Automated game testing with ICARUS: intelligent completion of adventure riddles via unsupervised solving. In: CHI PLAY: Extended Abstracts, pp. 153–164. ACM (2017)
Politowski, C., Guéhéneuc, Y.G., Petrillo, F.: Towards automated video game testing: still a long way to go. In: 2022 IEEE/ACM 6th International Workshop on Games and Software Engineering (GAS), pp. 37–43. IEEE (2022)
Politowski, C., Petrillo, F., Guéhéneuc, Y.G.: A survey of video game testing. In: International Conference on Automation of Software Test (AST), pp. 90–99. IEEE (2021)
Prasetya, I., Dastani, M., Prada, R., Vos, T., Dignum, F., Kifetew, F.: Aplib: tactical agents for testing computer games. In: 8th EMAS Workshop (2020)
Prasetya, I., et al.: An agent-based approach to automated game testing: an experience report. In: A-TEST, pp. 1–8 (2022)
Ralph, P., Tempero, E.: Construct validity in software engineering research and software metrics. In: 22nd International Conference EASE, pp. 13–23 (2018)
Rodríguez-Valdés, O., Vos, T., Aho, P., Marín, B.: 30 years of automated GUI testing: a bibliometric analysis. In: Paiva, A.C.R., Cavalli, A.R., Ventura Martins, P., Pérez-Castillo, R. (eds.) QUATIC 2021, pp. 473–488. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85347-1_34
Sestini, A., Gisslén, L., Bergdahl, J., Tollmar, K., Bagdanov, A.D.: Automated gameplay testing and validation with curiosity-conditioned proximal trajectories. IEEE Trans. Games 16(1), 113–126 (2022)
Smart, J., Molak, J.: BDD in Action. Simon and Schuster (2023)
Walker, A.M.: Tacit knowledge. Eur. J. Epidemiol. 32(4), 261–267 (2017)
Wang, X.: Vrtest: an extensible framework for automatic testing of virtual reality scenes. In: 2022 IEEE/ACM 44th ICSE-Companion, pp. 232–236. IEEE (2022)
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29044-2
Wu, Y., Chen, Y., Xie, X., Yu, B., Fan, C., Ma, L.: Regression testing of massively multiplayer online role-playing games. In: 2020 IEEE ICSME, pp. 692–696. IEEE (2020)
Zheng, Y., et al.: Wuji: automatic online combat game testing using evolutionary deep reinforcement learning. In: 34th International Conference ASE, pp. 772–784. IEEE (2019)
Acknowledgment
This work has been partially funded by: H2020 EU iv4XR grant nr. 856716 and ENACTEST ERASMUS+ grant nr. 101055874.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pastor Ricós, F., Marín, B., Prasetya, I.S.W.B., Vos, T.E.J., Davidson, J., Hovorka, K. (2024). An Industrial Experience Leveraging the iv4XR Framework for BDD Testing of a 3D Sandbox Game. In: Araújo, J., de la Vara, J.L., Santos, M.Y., Assar, S. (eds) Research Challenges in Information Science. RCIS 2024. Lecture Notes in Business Information Processing, vol 513. Springer, Cham. https://doi.org/10.1007/978-3-031-59465-6_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-59465-6_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-59464-9
Online ISBN: 978-3-031-59465-6
eBook Packages: Computer ScienceComputer Science (R0)