research-article

Open access

Exploring the Impact of Interactive Content on User Experiences within a Metaverse Learning Platform

Authors: Paolo Mengoni, Haerim Hwang, Daniel Jiandong Shen, Yuanxi Li, Paulina Pui Yun WongAuthors Info & Claims

ICSLT '24: Proceedings of the 2024 10th International Conference on e-Society, e-Learning and e-Technologies (ICSLT)

Pages 31 - 37

https://doi.org/10.1145/3678610.3678623

Published: 03 December 2024 Publication History

All formats PDF

Abstract

This study delves into the educational possibilities and user experiences offered by a Metaverse platform, utilizing immersive Virtual Reality (VR) to bolster engagement and learning in communication and grammar. In the wake of advancing digital technologies, there's an imperative to reevaluate conventional educational practices for their effectiveness in engaging students and fostering in-depth learning. Despite VR's potential in education, integrating these immersive experiences effectively into pedagogical frameworks and ensuring their accessibility and usability poses significant challenges. Our research employed a mixed-method approach, blending quantitative analysis to gauge user engagement, learning effectiveness, and acceptance of technology with qualitative feedback for deeper insight into user experiences. Participants navigated specially designed Metaverse scenarios, engaging with AI avatars through interactive tasks and narrative explorations across two distinct modules focused on grammar intricacies and the evolution of search technologies. The quantitative findings underscore a substantial correlation between users' interest and their intent to utilize the platform, accounting for 59% of the variance in intent. Qualitative responses highlight a robust engagement level and favorable perceptions of the Metaverse experience, particularly driven by the method being interesting. Nonetheless, identified areas for improvement in technology and ease of use underscore the necessity for a holistic approach that melds pedagogical efficacy with technological advancement to fully leverage the educational potential of immersive learning environments.

1 Introduction

With the rise of digital technologies, traditional educational methods are being reevaluated for their effectiveness in engaging students and fostering deep learning. The OECD [12] underscores this transition, advocating for the integration of ICT in classrooms to not only personalize instruction but also to expand access to information, encourage collaboration, and promote active learning approaches. This shift towards digital facilitation aligns with the transformative potential of immersive environments, particularly those enabled by Extended Reality (XR) and Virtual Reality (VR), as highlighted by Pringle et al. [14]. These VR and XR platforms offer accessible, scenario-based opportunities that overcome traditional educational limitations, such as resource constraints and environmental barriers, thereby enhancing the learning experience with engaging, interactive, and contextually rich settings that significantly bolster the comprehension and retention of complex subjects. Maddox & Fitzpatrick [9] delve into the neuroscience of VR, showing its impact on various brain centers. This interaction not only streamlines the learning process by spreading cognitive demands but also lessens the cognitive load linked to memorizing and deciphering complex tasks.

Moreover, the cognitive benefits of engaging with digital environments, as discussed by Bavelier & Green [1], extend to critical functions like attention, memory, and problem-solving skills. Active participation and interaction inherent in VR environments promote a dynamic and effective learning process that nurtures higher-order cognitive abilities. Van Laer et al. [19] highlight the technique of weaving captivating narratives within VR to draw students into the content, enhancing engagement. In this context, the Metaverse can be defined as a collective virtual shared space, created by the convergence of virtually enhanced physical and digital reality, where users interact with a computer-generated environment and other users [5].

Transitioning VR to educational settings poses several challenges. Cipresso et al. [3] highlight the complexity and time-intensive nature of customizing these digital spaces for learning, which extends beyond immersive design to ensuring educational effectiveness and accessibility for both teachers and learners. Omori et al. [13] identify multifaceted obstacles, including technical limitations and financial constraints, alongside the necessity for content that integrates seamlessly with existing teaching methodologies. Radianti et al. [15] also note a significant lack in the exchange of effective pedagogical strategies within the digital learning community. Adding to this, Sarıtaş & Topraklıkoğlu [16] point out the absence of engaging, active learning opportunities in such environments, indicating that the full educational potential of immersive technologies might not yet be fully leveraged. Furthermore, Checa & Bustillo [2] argue that only a subset of studies has demonstrated a definitive advantage of immersive technologies over traditional learning approaches, suggesting that their incorporation into the wider educational framework often falls short of expectations.

This points to a significant gap in the current application of VR in education, where despite the technology's capabilities, the pedagogical approaches employed do not always facilitate active, deep engagement with the content, underscoring the need for more dynamic and interactive experiences.

1.1 Theoretical Background

In VR and the Metaverse, concepts such as presence, immersion, narrative transportation, and social connection shape educational experiences. Presence, as in de Regt, Plangger, & Barnes [4], is the sensation of being in a virtual environment with engagement and focus. This sense of "being there" is crucial for immersive learning, allowing users to interact with the virtual space realistically, deepening their experience. Immersion extends presence, embodying engagement and involvement within a virtual environment.

Narrative transportation, highlighted by Van Laer et al. [19] places learners within engaging storylines, making complex concepts more approachable. This enriches educational content, making it more impactful and significantly improving learning outcomes. The learners become active participants in the narrative, employing their creativity to shape the storyline and strengthen their connection to the subject matter. Muthmainnah et al. [11] further investigate digital storytelling for language learning, demonstrating the effective use of creative and interactive techniques in the Metaverse. This strategy utilizes the natural appeal of storytelling to motivate learners, immersing them in educational and entertaining narratives. Embedding language practice within a narrative context provides learners with a structured environment that boosts understanding and memory.

Shin [17] further elaborates on the multifaceted nature of immersion, emphasizing the importance of user engagement, preferences, and social dynamics within the Metaverse. This perspective underscores the need for immersive environments to be designed with user-centric considerations, ensuring that they accommodate diverse preferences and foster rich social interactions. Sinatra et al. [18] further explore the dimension of social fidelity in virtual agents, emphasizing the significant role of social behaviors such as emotions, voice modulation, and facial expressions in enhancing the sense of social presence. Hwang & Chien [8] propose the innovative use of Non-Player Characters (NPCs) embedded with AI technologies, such as GPT, to act as tutors or mentors within the Metaverse. This approach aims to guide learning through simulated social interactions, providing learners with a dynamic educational experience.

Farrell [6] brings into focus experiential learning and authentic assessment, emphasizing the creation of meaningful educational experiences using simulations which include also the Metaverse. This approach underscores the importance of designing engaging virtual environments deeply resonating the learners' experiences and challenges. Further enriching the pedagogical toolbox, Mochizuki et al. [10] illustrate the potential of simulation games as a catalyst for social learning. They suggest that the principles of game-based learning can be effectively applied to Metaverse fostering collaborative and interactive experiences. This highlights the versatility of VR for both individual and collective learning journeys, enabling educators to craft engaging and socially enriching educational simulations.

To evaluate the efficacy of these pedagogical strategies within the Metaverse, Fussell & Truong [7] introduce an expanded technology acceptance model tailored for dynamic virtual environments. This model offers a robust framework for assessing user acceptance and the educational value of Metaverse platforms, providing insights into how these technologies can be optimized for education.

1.2 Research Question

Given these insights and challenges, how can the Metaverse, with its immersive and interactive capabilities, be effectively utilized to enhance the learning experience? Furthermore, how can educational content and interactions within the Metaverse be optimized to promote not only engagement but also active learning and deeper understanding.

2 Method

This study adopted a mixed method approach, where an educational metaverse platform designed with interactive learning tasks - such as 2D and 3D interactions supported by avatars, was used and evaluated on technological and pedagogical factors based on students’ intent to use the method for learning.

2.1 Participants

Our study was structured around two distinct student groups: one from the fields of Communication and Computer Science, with a focus on search engines, comprising 23 students, and another from the Linguistics domain, concentrating on grammar topics, consisting of 18 students. Each participant engaged with the Metaverse learning environment utilizing their personal computing devices, ensuring each had a reliable internet connection.

2.2 Assessments and Measures

Grammar Module: In the Grammar module, users embark on a journey within a Metaverse village, where they tackle linguistic challenges aimed at reviving lost language elements. The module employs conversational prompts, selection tasks, and scenario-based problem-solving to deepen understanding of grammatical structures, idioms, and sentence crafting, all within a compelling narrative framework.

Communication and Computer Science Module: The Communication and Computer Science module presents an exploration of search technology's evolution, from manual indexing to contemporary digital systems. Participants engage with historical insights, practical tasks simulating early data organization, and interactive discussions that highlight key milestones like Project Xanadu's influence, the role of automated web crawlers, and the significance of Google's PageRank, blending education with immersive storytelling.

In the Metaverse platform, performance is achieved through the embodiment of roles by avatars, simulating real-life tasks within a structured set of game rules and mechanics, facilitating practical, hands-on learning experiences (see Table 1).

Table 1:

Authentic Learning	Definition
Performance	Learners complete tasks through avatars, embodying roles that mirror real-life functions, within the game's rules and mechanics, to achieve realistic task completion.
Realism	Realistic, practice-based scenarios are crafted, incorporating problem-solving within the game's narrative, to offer learners practical challenges that mimic real-world issues.
Varying Experiences	Diverse situational and quest-based learning experiences are provided, governed by game mechanics and rules, offering a variety of contexts and challenges.
Challenge	Progressive, adaptable quests introduce scalable challenges, with game rules defining the framework for these challenges, ensuring a continuous learning curve.
Feedback	Immediate, context-specific feedback is given through interactive game elements and avatars, facilitating a responsive and guided learning environment.

Table 1: Integration of Authentic Learning Principles with Game-Based Learning Concepts

For this study, we utilized a Metaverse platform - Classlet, designed for immersive learning experiences. This platform integrates storytelling and interactive elements through the use of Non-Player Characters (NPCs), enabling a more engaging educational journey. It allows for the customization of content, where educators can map uploaded materials to NPCs, providing varied tasks like answering questions or making choices based on varying scenarios. The study focused on two distinct educational themes: grammar, set within a virtual village, and search engines, explored through a time-travel narrative. Each theme was brought to life in the Metaverse environment, offering students a unique blend of learning and engagement (Figure 1). The grammar module placed students in a village setting where they interacted with NPCs to tackle grammar-related tasks. Conversely, the search engine module took students on a time-travel adventure, challenging them to apply search engine concepts in various historical contexts. Students accessed the educational content by entering specific scene codes, which transported them to the relevant virtual environments. These environments were carefully designed to support the learning objectives, with tasks ranging from dialogue-based questions to content challenges.

Figure 1:

2.3 Data Collection and Analysis

To adapt the survey for assessing user perceptions and intentions towards the Metaverse as a platform for this study, we focused on specific variables aligned with our objectives. These variables include:

•

Aesthetic Appeal (AEST): Evaluating participants' perceptions of the visual design and overall aesthetic quality of the Metaverse resources.

•

Technical Functionality (TECH): Assessing the operational effectiveness and technical reliability of the Metaverse resources.

•

Ease of Use (EASE): Rating the user-friendliness and navigational ease within the Metaverse learning environment.

•

Engagement and Fun (FUN): Gauging the level of engagement and enjoyment participants experienced while using the Metaverse resources.

•

Accessibility/Usability (USAB): Determining the accessibility and usability of the Metaverse resources from the user's perspective.

•

Learning and Performance Enhancement (PERF): Evaluating the perceived impact of the Metaverse on participants' learning outcomes and performance in the course.

•

Usefulness in Learning (USEF): Assessing the perceived utility of the Metaverse in facilitating learning processes.

•

Interest in Learning (INTE): Measuring the extent to which the Metaverse enhances the interest and motivation towards learning.

•

Intention to Use (INTENT): Capturing participants' willingness to use the Metaverse for future learning endeavors.

Quantitative data were analyzed using descriptive statistics, T-tests, and Spearman's correlation to explore relationships between user engagement, technological functionality, and educational outcomes. Qualitative feedback was subjected to thematic analysis to identify recurring themes related to usability, engagement, and learning effectiveness.

3 Results

The scale's reliability was assessed using Cronbach's alpha within a frequentist framework, yielding a point estimate of 0.890. This indicates a high level of internal consistency among the scale items, suggesting that the scale is a reliable measure of the constructs related to Metaverse.

The descriptive statistics, with responses from 41 participants, indicate an overall positive perception. The highest mean scores were observed in INTE (4.268) and INTENT (4.000), suggesting strong engagement and a clear intention to use the platform. The lowest means were for TECH (3.463) and EASE (3.390), while still reasonably positive (68%), highlighted areas that could be enhanced to improve the user experience. The Shapiro-Wilk test showed significant results, indicating deviations from normality in the data distribution, which was considered when applying parametric statistical analyses.

The first t-test analysis aimed to discern whether the type of device influenced users' experiences with Metaverse resources. The tests did not reveal any significant differences between desktop and mobile users in their ratings of aesthetic, technical functioning, ease of use, enjoyment, or perceived learning improvement from the Metaverse resources.

The Mann-Whitney U test revealed significant differences between the "Comm" and "Grammar" groups in the "TECH" (W=298.000, p=0.005, RBC=0.440) and "EASE" (W=290.500, p=0.009, RBC=0.403) dimensions, indicating moderate positive effects where the "Comm" group scored higher. No significant differences were found in "FUN" and "INTE", while "PERF" showed a positive effect (W=273.500, p=0.031, RBC=0.321), suggesting higher performance scores for the "Comm" group. A separate Mann-Whitney U test for type of device used did not show any significance.

Spearman's correlation analysis of the Metaverse learning platform revealed key relationships among its dimensions. A significant link was found correlating interest (INTE) with users' intent to engage (INTENT) (ρ=0.775, p<.001), highlighting INTE's crucial role in driving platform usage. This analysis also uncovered strong connections of INTE with both aesthetics (AEST) (ρ=0.661, p<.001) and enjoyment (FUN) (ρ=0.634, p<.001), stressing the importance of visual appeal and fun in boosting interest. Additionally, significant correlations emerged linking technology (TECH) with ease of use (EASE) (ρ=0.593, p<.001), and performance (PERF) with usefulness (USEF) (ρ=0.627, p<.001), emphasizing the significance of technological excellence, design quality, and content value in enhancing the Metaverse experience.

Given the intertwined relationships among key features of the Metaverse platform, particularly between Interaction and User Intent, as well as other critical aspects like Aesthetics, Fun, and Usability, we combined the communication and grammar elements for further analysis in regression analysis to capture their collective impact on user engagement and learning outcomes.

Before assessing the regression model's performance, diagnostic checks were performed to ensure that the underlying assumptions of the analysis were met. The Residuals vs. Predicted Values plot indicated consistent variance of residuals, suggesting homoscedasticity, and the Normal Q-Q Plot showed that residuals approximated a normal distribution, with some minor deviations at the extremes. The linear regression analysis for "INTENT" reveals that the model explains approximately 57.9% of the variance (Adjusted R² = 0.557) and is statistically significant F (2, 38) = 26.097, p < .001. The "INTE" variable is a strong positive predictor of "INTENT" (β=0.816, p<.001), while "Type (Grammar)" shows a negative, marginally significant relationship (β=-0.346, p=0.057). The Durbin-Watson statistic indicates no significant autocorrelation in residuals (d=2.028), suggesting the model's assumptions are adequately met.

3.1 Thematic Analysis

Engagement: A prominent theme is the engaging nature of the learning method. Participants describe the method as "interesting," "fun," and "just like playing a game," which suggests that the learning experience captures their attention and makes the process enjoyable.

•

Illustrative Quote: "Different elements can be reacted"

•

Illustrative Quote: "The design is engaging; Content can be very flexible for teachers to update?"

Ease of Use and Clarity: Another theme is the usability and clarity of the method. Responses indicate that the method is "easy to handle," has a "clear guide," and offers "clear stages" that help learners progress. This suggests that the design is user-friendly and supports a smooth learning curve.

•

Illustrative Quote: "Easy to handle for the Qs"

•

Illustrative Quote: "The steps are clearly"

Motivation and Interactive Elements: The method seems to motivate learners through immersive features, which is underscored by descriptions such as "creative," "interactive," and "immersive." It appears to effectively utilize gamification and interaction to encourage active participation.

•

Illustrative Quote: "Learn the skill to ask questions in GPT, creative learning method"

•

Illustrative Quote: "It is colorful. 2. completing the tasks is very satisfying."

Technical Issues and Usability: Concerns are expressed about the platform's functionality, including bugs and difficulties with navigation and control, which could disrupt the learning process.

•

Illustrative Quote: "Difficult to move. Difficult to control the camera. A bit lag."

•

Illustrative Quote: "Hard to control, delay."

Instructional Design and Content Quality: Feedback indicates that instructions and guidance within the method may not be sufficient or clear, potentially leading to confusion.

•

Illustrative Quote: "The instruction is not good enough."

•

Illustrative Quote: "The game can be more smooth. There are still bug… The guide can be more clear."

Hardware Limitations and Learning Effectiveness: Remarks suggest the learning method may require advanced hardware, which not all users may have access to, and there's a possibility of the learning experience leading to surface-level engagement rather than deep understanding.

•

Illustrative Quote: "Requires smart devices."

•

Illustrative Quote: "1. More question types can be provided 2. The visual wasn't smooth enough."

•

Illustrative Quote: "1. too many distractions 2. time-consuming 3. students can complete the quests simply by trial and error without much thinking."

4 Discussion

In our study, the regression analysis highlighted that 'INTE' significantly predicts users' intent to engage with the Metaverse platform, with a beta value of 0.816 (p<.001). This suggests that students found the method interesting and wished to use it more - based on the high intent to use mean. However, the variable 'Type (Grammar)' showed a slight negative association with intent to use, indicating that some content might be less effective in motivating users. These findings, particularly the model's ability to explain 57.9% of the variance in 'INTENT', open discussions on how to better design Metaverse content and interactions to meet learner needs.

The significant correlation between 'INTE'—indicative of the content's interesting nature—and users' intent to use the platform underscores the pivotal role of captivating material in virtual environments like the Metaverse. With 'INTE' scoring a mean of 4.268 and directly influencing the intent to use, it's clear that engaging content can significantly drive user commitment. Despite this strong correlation, areas such as Technology and Ease of Use scored lower. Addressing these aspects is crucial for maximizing the potential of the Metaverse as an educational tool, ensuring that its innovative capabilities are fully leveraged to enhance learning outcomes.

The immersive and scenario-based opportunities provided by Extended Reality (XR), highlighted by Pringle et al. [14], emphasize the critical role of the platform being interesting('INTE') in the Metaverse. This underscores the potential of XR in enhancing user engagement through immersive experiences, suggesting that Metaverse content should be designed to capitalize on this immersive potential to boost user engagement and learning outcomes. Additionally, van Laer et al. [19] explore the impact of narrative transportation in digital storytelling, which can further enrich Metaverse experiences by deeply engaging users in the content, making complex concepts more approachable and memorable. Hwang & Chien's [8] insights into the Metaverse's educational potential, viewed through an AI lens, open avenues for employing AI-driven avatars to enrich learning experiences. In our study, avatars took on multifaceted roles, from providing navigational assistance and imparting knowledge to guiding users through narratives and offering personalized feedback. These interactive elements may have enhanced user engagement by creating a more responsive and immersive environment.

The emphasis on 'INTE' in our regression analysis, which shows a strong positive influence on users' intentions to engage with the learning method, suggests that the pedagogical aspects are of paramount importance to the users. Despite the feedback on technical aspects and usability concerns, such as "Difficult to move. Difficult to control the camera. A bit lag" and "Hard to control, delay," these issues do not significantly detract from users' overall positive perception and willingness to use the platform, as evidenced by the high mean score of 4.268 for 'INTE'. This indicates that users may prioritize the educational value and content interest over technical imperfections. Furthermore, the absence of significant differences in user experience based on the devices used (desktop or mobile) reinforces the idea that, while technical enhancements could improve the user experience, they are not the primary determinant of users' engagement. Instead, the focus on enriching the pedagogical content and making learning interesting and engaging seems to be the key driver for users, highlighting the need for continued innovation in educational content over technical perfection.

The statistical analysis highlighted differences between the "Comm" and "Grammar" groups in both the "TECH" and "EASE" dimensions. The statistical analysis showed the "Comm" group rating the platform higher in both "TECH" (W=298.000, p=0.005) and "EASE" (W=290.500, p=0.009), indicating a more favorable experience with the technology and usability for the Communication module. The "Comm" module, with its time-travel narrative, likely offered a more engaging and dynamic story, possibly leading to higher satisfaction in "TECH" and "EASE" due to the immersive and varied historical contexts. In contrast, the "Grammar" module focused on interactions within a single village setting, which, while educational, might have lacked the same level of narrative engagement. Further, the "Grammar" module's tasks, centered on grammar and linguistic structure, demanded higher cognitive effort, potentially affecting user perceptions of ease and technology. The intricate balance between cognitive load and engagement, as discussed by Bavelier & Green [1] and Maddox & Fitzpatrick [9], highlights the complexity of optimizing user experience in virtual environments, especially when educational content requires significant cognitive effort.

The positive student feedback on interest, perceived usefulness, and enjoyment in our study, although not directly tied to performance metrics, underscores the potential value of the Metaverse approach. This approach, characterized by its authentic and game-based learning methods, not only fosters enthusiasm and engagement but may also facilitate deeper immersion in the process, a crucial aspect often linked to enhanced educational outcomes. The observations by Checa & Bustillo [2] suggest that while immersive technologies offer unique experiences, their full integration into educational systems and their impact on learning effectiveness still require further exploration and development. This aligns with our findings, which, through positive learner feedback, indicate that the Metaverse could be an effective medium for immersive education when designed thoughtfully. Moreover, addressing Sarıtaş & Topraklıkoğlu's [16]concerns about the lack of active learning in VR environments, our study presents a practical example of how immersive technologies can incorporate active learning principles. By leveraging authentic and game-based learning within the Metaverse, our approach suggests a path forward for making VR environments more educationally enriching, potentially overcoming some of the limitations highlighted in current VR applications in education.

5 Conclusion

This study reveals that immersive learning environments within the Metaverse, enriched by Extended Reality (XR) and Virtual Reality (VR), hold substantial potential to transform educational experiences. By leveraging interactive and scenario-based opportunities, these environments can significantly enhance user engagement, comprehension, and retention of complex subjects.

The data analysis indicates a high level of user engagement and a strong intention to use the Metaverse platform for learning, particularly emphasized by the 'INTE' (interest and engagement) dimension. However, the study also highlighted areas for improvement, such as technological aspects and ease of use, to enhance the overall experience. The narrative-driven and interactive nature of the Metaverse was found to be particularly effective in capturing users' attention and motivating their educational journey. One limitation of this study is the absence of direct performance metrics, which constrains the ability to conclusively link the observed engagement and interest to tangible learning outcomes. Additionally, the study's scope was limited to specific modules within the Metaverse, which may not fully represent the vast potential and diversity of scenarios that the platform can offer.

This research underscores the importance of designing immersive environments that prioritize pedagogical effectiveness alongside technological innovation. By aligning the engaging elements of the Metaverse with educational objectives, there is a significant opportunity to foster a more engaging, effective, and enjoyable learning experience for students. Future research should aim to integrate direct performance metrics and broader scenarios within the Metaverse to more comprehensively assess the platform's educational efficacy. Further exploration into the integration of AI-driven avatars and the development of collaborative experiences could also enhance the Metaverse's educational value.

This study's insights into the transformative potential of immersive learning in the Metaverse not only contribute to the academic discourse on digital education but also offer practical guidance for educators and developers looking to harness the power of VR and XR in educational contexts.

Acknowledgments

This work has been funded by the “UGC FITE IICA titled “Beyond Reality: Unleashing Generative Metaverse Avatars in Education” fund, University Grant Council, Hong Kong Baptist University, Hong Kong, China

References

[1]

Bavelier,D., & Green, C. S. (2019). The brain-boosting power of video games. Scientific American, 321(1), 26-31.

Abstract

1 Introduction

1.1 Theoretical Background

1.2 Research Question

2 Method

2.1 Participants

2.2 Assessments and Measures

2.3 Data Collection and Analysis

3 Results

3.1 Thematic Analysis

4 Discussion

5 Conclusion

Acknowledgments

References

Index Terms

Recommendations

The possibilities of changes in learning experiences with Metaverse

Understanding experiences in metaverse: How virtual nature impacts affect, pro-environmental attitudes, and intention to engage with physical nature

Interactive video stories from user generated content: a school concert use case

Comments

Information

Published In

Publisher

Publication History

Check for updates

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

View options

PDF

eReader

HTML Format

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations