6.1 RQ1: How Do the Three Training Modalities Compare in Terms of Presence and Technology Acceptance?
The results on presence paint a mixed picture across the three modalities (MR, VR, and REAL). Starting with Self-Presence, the results revealed a significant difference, with MR and REAL standing out as superior to VR. This aligns well with the qualitative findings where MR was frequently cited for its realistic simulation of audio, visual and haptic information, which likely contributes to a greater feeling of
‘being there’ with their own body in the training scenario. Participants also expressed similar sentiments for REAL training, pointing to the tangible experience and naturalistic human interaction as strengths. Using Skarbez’ framework [
48], the increased immersion due to haptic feedback and increased world knowledge through tangible interaction actually led to higher levels in self-presence. By including highly specific and realistic tangible devices into MR, and thereby increasing the haptic realism [
36], specifically the sense of self-presence could be increased, nearly reaching that of training in reality.
Social Presence scores were also significantly different across the modalities, real training stood out as the best, followed by MR and then VR. This finding is intuitive, as having a tangible manikin or real actor in the scenario does seem to improve the perception of actually interacting with another person. Though MR’s social presence was not as high as in the real training, it nonetheless showed significant improvement over VR, which is in line with previous research about social presence in VR training [
52]. This can be a critical factor in patient treatment training scenarios where social interactions are essential. The qualitative data corroborated these results; participants praised the real training for the naturalness of speech interaction. Social presence is closely related to the coherence of the simulation [
48]. Given that VR and MR used the same speech interaction system, the blend of reality and virtuality in the MR condition apparently makes it more plausible that one is interacting with a living, embodied patient.
One notable aspect was the absence of significant differences in Physical Presence across the modalities, although on a descriptive level MR outperformed the real training, which in turn was rated higher than VR. This is somewhat counter intuitive, as the qualitative data suggests that both VR and MR provide realistic environmental simulations, while real training, despite having physical objects like a bicycle and backpack, falls short in communicating the environmental context as vividly as the digital modalities do. The environmental context is important for contributing towards perceived realism of the training and also can add a stress inducing element [
37].
Nonetheless, the lack of statistical significance suggests that the three modalities are more similar in terms of Physical Presence than one might initially assume, indicating that technological augmentations are not necessarily superior in creating a sense of physical engagement.
The results for technology acceptance were also quite informative. Real training led in Effort Expectancy, which makes sense considering it’s an already established training method and would be easier to integrate as participants are familiar with it. MR also trended higher, although this was not statistically significant. This aligns with participants’ open-ended responses, where real training was often referred to as the established standard in training scenarios.
Hedonic motivation was similar across all modalities, implying that participants found each form of training to be engaging. There was also a trend in Performance Expectancy, with MR and REAL outperforming VR, but again, statistical significance was not reached.
For Facilitating Conditions, both MR and REAL outperformed VR, indicating that they were easier and more intuitive to use. This can be particularly important for training scenarios where ease of use can affect the training outcomes significantly. This is in line with our prior work [
54], where it was observed that using the actual tools in a virtual environment requires very little time to learn and is very intuitive, as the sensorimotor contingencies are preserved. In contrast, the VR environment might introduce an additional layer of complexity due to the need for specialized controllers or gestures, which could interrupt the flow of the training and require additional cognitive load to master. This potentially hampers the effectiveness of the training in the VR modality, making it less conducive for quick and intuitive learning. Interestingly, MR, which blends digital and physical elements, was able to maintain a high level of facilitating conditions, perhaps due to its ability to superimpose digital information directly onto real-world objects, thus reducing the disconnect between the digital and physical worlds. This seamless integration likely makes it easier for users to adapt and perform tasks, contributing to its higher rating in facilitating conditions. These findings suggest that while VR has benefits in terms of immersive experiences, when it comes to ease of use and facilitation of effective training, MR and REAL environments may offer superior advantages.
Regarding Behavioral Intention, there were no significant differences in the quantitative data. Contrasting, qualitative responses partly speak to the intention to use: MR was chosen by 11 of 15 participants as their preferred training method, while the remaining 4 chose real training. Furthermore, MR was often cited as offering
‘the best of both worlds’. This relates to [
13], who state (in regard to medical simulators) that many approaches limit themselves to only mimicking the real world regarding haptic feedback. In the proposed approach, there is no need for mimicking, as real tools are integrated, a notion which was called for since early manikin integrations into VR [
46], who expressed the need of utilizing interaction devices instead of–in their case– data gloves.
This discrepancy between the quantitative and qualitative data might suggest that while the measurable intention to engage with a certain modality may not vary significantly, people’s subjective preferences lean more towards MR. The appeal of MR as ‘the best of both worlds’ likely stems from its ability to provide a highly immersive experience, while still preserving the intuitiveness and ease-of-use found in real-world training.
In summary, our findings suggest that MR offers a balanced training solution, exhibiting higher levels of presence and technology acceptance than VR and coming close to real training in terms of social presence. These findings are supported by qualitative data, which revealed that MR’s integration of real and virtual elements, and its realistic simulations make it a highly promising and effective modality for training, even in its prototypical state. These findings are in line with previous research comparing social agents in virtual vs. real training, e.g. in the context of the workplace [
52]. The results that while REAL training remains the gold standard, MR’s capabilities make it a strong candidate for future training, particularly those that demand a blend of realistic interaction and complex, risk-free simulations.
6.2 RQ2: What Modalities Are Most Beneficial and Best Matching for Which Training Objectives?
Our study indicates that the selection of a training modality–MR, VR, or Real Training–can have a significant impact on the use of emergency medical training programs, depending on the training goals. Consistent with participant perspectives, the data suggests that each modality offers specific advantages that make it particularly well-suited for distinct training contexts.
MR training was consistently highlighted for its realistic environmental portrayal and for skill-specific training. This finding aligns with previous research indicating that MR’s capabilities can offer a blended experience that retains the realism of a physical environment while incorporating virtual elements [
15]. As participants in our study pointed out, MR can be invaluable in scenarios where understanding the context or the environment is crucial. It’s capabilities for rendering details (e.g. bleeding, bystanders or environmental threats), making it ideal for specialized, high-fidelity training scenarios.
VR training was particularly appreciated for its potential in organizational training and continuing education. This supports the notion that VR’s strength lies in abstract scenario training and procedural knowledge development [
41,
59]. Moreover, VR’s advantage in simulating factors like skin color and respiratory rate efficiently adds an extra layer of fidelity in continuous medical education.
However, Real Training still offers a unique ability to facilitate actual physical interactions with real patients. It was also cited as particularly beneficial in the initial stages of training, highlighting its foundational role in building basic skills. These findings are supported by existing literature emphasizing the importance of hands-on experience in medical training [
59] and also the importance of training muscle memory [
26].
The quantitative data further nuanced these perspectives. For instance, the Self Presence and Social Presence scales showed significant differences among the modalities, corroborating participant claims about the immersive capabilities of MR and Real Training. MR scored highest in Self Presence, which might suggest that trainees felt most present or engaged when training in a mixed-reality environment. Real Training excelled in Social Presence, possibly capturing the irreplaceable value of human interaction in medical training scenarios.
Similarly, Effort Expectancy showed a highly significant difference, indicating that Real Training is perceived as less effortful compared to MR and VR. This aligns with the qualitative data suggesting that Real Training offers a more ’natural’ or intuitive interface, especially in contexts that require the manipulation of actual medical equipment.
Interestingly, no significant differences were observed in technology acceptance variables like Behavioral Intention, Hedonic Motivation, and Performance Expectancy, suggesting that acceptance of these modalities may be relatively uniform. It may also imply that the ’novelty’ factor associated with technological training modalities like MR and VR is not a dominant influence on trainees’ willingness to use them.
6.3 Limitations
There are certain limitations in this work that need to be discussed. First, the relatively small sample size of 15 first responders left our study statistically underpowered, meaning that only large deviations between the modalities reached statistical significance, making it harder to detect smaller, yet potentially meaningful, differences between the training modalities. Also, the study was conducted with first responders in one country from one institution. As procedures and training curricula might differ between countries or institutions, the generalizability of results needs to be confirmed in future studies involving first responders from other institutions and/or countries.
Despite this limitation, the study provides a exploration into an under-researched area within the HCI community. Given the specialized nature of this field, even a small sample can offer valuable insights into the practicality and usability of MR tools like the Green Manikin.
Second, the fidelity of the three training modalities is crucial when comparing them, as not to measure artefacts. In the case of real training it could be argued, that our variant represents a quite low-level training. Higher fidelity real training would involve make-up and silicone wounds attached to the role-player for example. Still, this low-level form of training is much more common in practice due to its simplicity - a goal that the MR and VR alternatives want to enable as well.
Lastly, a limitation of this study lies in its primary focus on Human-Computer Interaction (HCI) contributions, particularly in the evaluation and comparison of different training modalities. While this emphasis offers important insights into the interface and user experience aspects of medical training technologies, it may not comprehensively address other critical dimensions such as psychological factors, educational pedagogy, or training effectiveness.
6.4 Future Work
Future research in the human-computer interaction (HCI) domain has several future directions to explore, based our findings.
One next step is to conduct longitudinal studies. While our research provides an initial snapshot of user experiences across MR, VR, and REAL modalities, a longitudinal approach could track these experiences over time. This would offer a more comprehensive understanding of how user adaptability and the sustained effectiveness of these interactive technologies evolve in the context of medical training.
Another critical research direction pertains to the concept of presence, which yielded mixed results across the modalities in our study. It would be beneficial for HCI research to delve deeper into the specific elements that contribute to presence in these environments in this special context. Identifying the factors, be it realism or interaction capabilities, that influence self, social, and physical presence, could greatly inform the design and application of future interactive systems for training.
The issue of technology acceptance also extends to the type of interfaces and tools used in the training modalities. Specialized controllers and tangible input devices, for example, could significantly impact the training experience in MR and VR scenarios. Future work in HCI could explore how these specialized tools compare to generic ones in terms of engagement and training effectiveness.
Lastly, understanding how these new modalities can be integrated into existing medical training curricula is an essential question. As educational institutions often rely on well-established teaching methods, the seamless integration of emerging interactive technologies like MR and VR could offer a substantial advantage. Future research should therefore also focus on the practical aspects of such integration, potentially guiding institutions on how to best complement traditional methods with interactive systems.