Open AccessArticle

A Modular XR Collaborative Platform for Occupational Safety and Health Training: A Case Study in Circular Logistics Facilities

Ali Vatankhah Barenji

^1,2,*

Jorge E. Garcia

^2,3

and

Benoit Montreuil

^2,3

Department of Technology, Illinois State University, Normal, IL 61790, USA

Physical Internet Center, Georgia Tech, Atlanta, GA 30332, USA

School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA

Author to whom correspondence should be addressed.

Information 2024, 15(9), 570; https://doi.org/10.3390/info15090570

Submission received: 27 July 2024 / Revised: 3 September 2024 / Accepted: 11 September 2024 / Published: 18 September 2024

(This article belongs to the Special Issue Extended Reality and Cybersecurity)

Download

Browse Figures

Figure 1
Representative operational workflow of a CLF. "> Figure 2
Occupational injuries and illnesses by type of case [<a href="#B10-information-15-00570" class="html-bibr">10</a>]. "> Figure 3
Overview of the proposed architecture. "> Figure 4
Network properties and topology of the MXC-P. "> Figure 5
Key Training and Characteristics of PPE Module. "> Figure 6
Key training and characteristics of the PH module. "> Figure 7
Key training and characteristics of the PR module. "> Figure 8
Kinematic chain of the hand and the body model. Joints defined as revolute-type. "> Figure 9
Evaluation of the MXC-P based on the heuristic evaluation method. "> Figure 10
Results of full body movement with ergonomic risk evaluation for five main joints. "> Figure 11
Results of hand joint movement with ergonomic risk evaluation. ">

Versions Notes

Abstract

Over the past few years, safety and health have become major concerns in the warehouse and logistics sectors. Each year, warehouse fatalities, injuries, and accidents cause unrecoverable losses and huge financial costs. In spite of all the advancements in methods, tools, equipment, and regulations, the number of accidents, especially fatal ones, has not subsided significantly. As a result, safety professionals and researchers have explored new and innovative ways to combat this problem. In the circular logistics facility (CLF) industry, located inside warehouses and providing human muscle-oriented services to maintain pallets, both short-term safety incidents and long-term health concerns are present. Long-term health training is rarely discussed in the literature compared to short-term safety training. This is because health issues are more complex than safety issues, since biological outcomes may take time to develop, are affected by multiple resources, and cumulative injuries may occur. This paper contributes to warehouse health and safety by designing and developing a modular XR collaborative training and testing platform (MXC-P). The co-design process is applied to design each module in the MXC-P. Three main modules related to health and safety training for CLF were considered, namely personal protection equipment, pallet handling, and pallet repairing. On this platform, a virtual interactive world provides a solid hands-on training environment and generates syntactic data for evaluating long-term health risks. On the other hand, collaborative and modular environments provide a solution to geographically distributed systems, allowing employees to connect and train remotely. The effectiveness of the MXC-P is compared with traditional safety training in a pilot study. Based on the results, we can establish that the MXC-P is effective in teaching and testing hazard identification situations, especially those relating to short-term health. The results also indicate that trainees’ recall of knowledge would improve with the MXC-P. In addition to this, the MXC-P can also be used to test and evaluate a new system and generate syntactic data for evaluating long-term health.

Keywords:

extended reality; training; risk perception; safety; occupational health and safety; co-design process

1. Introduction

Pallets are a common part of any warehouse, and the handling, repairing, and recycling of them are mostly performed within the warehouse by a third party or a special section that is commonly referred to as a circular logistics facility (CLF). The development of CLFs provides an efficient way for companies to manage their pallets [1]. Third-party management companies combine pallet manufacturers with pallet repair and recycling companies to provide a complete pallet management system. Third-party companies either manage a user’s existing pool of pallets or rent pallets to the user on a per-trip basis. The CLF tracks pallets throughout their usage and retrieves them for reuse or refurbishing. The CLF uses higher-quality pallets, increasing the number of trips per pallet by an average of 20 to 40 times compared to 5 times when managed by the user. This style of pallet management reduces the amount of lumber used to make new pallets, promotes reuse and repair, reduces pallet waste via recycling, and saves the pallet user money and time [2,3]. Due to the nature of the warehouse and pallets, CLFs are primarily a human-originated and muscle-based industry. In recent years, the issue of warehouse safety, specifically regarding CLFs, has become more frequent in practice and more important because of the high hiring rate of non-expert workers during the high seasonal periods for CLFs [4,5].

The rates of illness and injury at warehouses were described by the U.S. Department of Labor Office of Inspector General (OIG) as “consistently high”. In 2021, the rate was 5.5 per 100 employees, more than double the rate across all U.S. industries. About 1.7 million people work in the nation’s roughly 20,000 warehouses, and one-third of these warehouses have an active CLF. A total of 19,709 facilities are classified as warehouses; an additional 38,785 are listed as electronic shopping and mail-order houses, which the OIG classified as “online retailers”. Online retailers employed 447,059 workers in 2021 [6,7]. The OIG noted that warehouse workers can face hazards resulting in serious injury due to powered industrial trucks, loaded pallets, and repetitive movements. These dangers can be compounded when time-based delivery quotas and production goals increase because of the rise in e-commerce. According to the U.S. Census Bureau, during the first year of the COVID-19 pandemic in 2020, there was a 43% increase in e-commerce sales, with sales rising from USD 571.2 billion in 2019 to USD 815.4 billion in 2020 [8]. This causes the hiring of non-expert workers, which dramatically increases the rate of injury in the warehouse industry.

Figure 1 illustrates the representative operational workflow of a CLF in a warehouse, which is started by unloading pallets from a truck via a forklift in groups of ten or eight. Once the pallets are stacked in the buffer zone, workers manually sort them into three categories indicating that they need repair, should be recycled, or are ready to use. The pallets belonging to the repair category move to the repair table to undergo the repair process, the pallets to be recycled move to the recycle station, and the ready-to-use pallets move to the buffer zone to be sent to customers. The key factors that contribute to injury are awkward lifting, reaching, or twisting, the pace of work, and repetitive motion. Regarding this point, Amazon recently published the main factors that contribute to injury in its warehouses [9]. Awkward lifting, reaching, or twisting is the highest factor at 28%, followed by heavy packages at 27%, pace of work/workload at 21%, and repetitive motion at 19%. In addition, the U.S. Department of Labor reported CLF occupational injuries, illnesses, and fatal injuries from 2016 to 2018 [10]. Figure 2 shows occupational injuries and illnesses by type of case for the warehouse and CLF industries. Overexertion and bodily reactions (OBR), contact with objects (CB), and slips, trips, and falls (STP) are the main factors involved in occupational injuries and illness.

Improved safety performance in the industry can best be achieved by preventing incidents from occurring in the first place [11]. A better education and training program for CLF workers is one way to achieve this goal. Historically, different approaches have been used to train workers on safety, such as video recordings, handouts, and hands-on training. Each of these methods has its advantages and disadvantages. For example, while video recordings and handouts are easy to produce and relatively inexpensive, they often fail to fully engage workers during the training process. Additionally, in all of the previous cases, keeping the information updated is a time-consuming process. Hands-on education, with the highest impact on training, has the highest cost and risk associated with it [12]. In particular, because CLFs are geographically dispersed, sending officers to conduct hands-on safety education is very costly, and due to a limited number of available officers, this could reduce the educational impact [13]. This current scenario, where the absence of modern technology is notorious, also leads to missed opportunities. In this setting, it is challenging to obtain key information that can be used to improve training methods, such as capturing typical errors and pain points.

In this context, immersive technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR) are becoming increasingly common in industrial applications under the umbrella of extended reality (XR). The increasing availability of XR devices has led to the possibility of various forms of employee training. This training includes safety training, machine and equipment operator training, and maintenance training, as well as remote repair training in collaboration with outside experts [14]. Recently many researchers and industries have highlighted the advantages of XR in safety training, such as Rafael Sacks et al. [15], who examined the hypothesis that safety training in VR construction sites would be more effective in terms of workers’ learning and recall than equivalent training with conventional methods in identifying and assessing construction safety risks. Arachchige, Chander et al. [16] provided a brief overview of the design and development of a VR tool for physiological and subjective measures of anxiety with repeated exposure on construction sites. An industry-based expert flow was used to design and develop the VR application. Ankit Shringi et al. [17] proposed an interval type-2 fuzzy Delphi method to understand the effectiveness of different XR technologies in imparting important construction safety training to construction workers in a virtual environment compared to conventional classroom training sessions. Zuzhen Ji et al. [18] utilized VR and diminished quality of life (DQL) to develop a training method for occupational health and safety (OSHA). Both hazard identification and risk perception were improved through the development of the platform.

Safety training using VR, AR, and XR provides significant improvements in occupational injuries and illnesses and has been widely studied by many researchers. These training programs are mainly focused on educating about sequential operations, standardizing working procedures, and short-term safety risk assessment [19,20,21]. The majority of developed platforms seem promising for specific industrial aims without adapting training materials for hands-on and practical learning in the XR environment with a geographically distributed system. Table 1 synthesizes the key findings mentioned previously and introduces additional insights from other relevant studies. This table’s structure is designed to facilitate the comparison of the specific XR technology used with traditional alternatives when training workers in different domains of operations and across several industries.

This paper presents the development of a modular virtual collaborative training and testing platform (MXC-P) utilizing a co-design process for designing each module for CLFs which can be transferable to other industrial cases. It provides a methodology to create an effective safety learning environment and scenarios in collaboration with safety professionals, CLF managers, and a team of researchers. The MXC-P is an educational software package which runs on the cloud and can be used by people with little prior computer or VR experience. It is composed of three modules, namely the personal protection equipment (PPE) module, the pallet handling (PH) module, and the pallet repairing (PR) module, which utilize a VR-based software capability to assist the workers in “visualizing” the concepts and to provide an immediate collaborative virtual environment between workers and safety officers, giving users the opportunity to graphically provide feedback during the learning or evaluating processes.

Through our study, we aim to make several theoretical and practical contributions. First, we propose a modular virtual collaborative platform for CLFs aimed at overcoming the challenges of OSHA compliance training. This platform offers cost-effective and hands-on learning experiences. Second, we design and develop each module through a co-design process, which ensures better educational content for end-users and addresses customization issues. Third, the proposed platform is implemented through a cloud-based and collaborative environment, enabling the capture of synaptic data for subsequent analysis of long-term health concerns.

The rest of this paper is organized as follows. Section 2 provides an overview of the proposed MXC-P platform. Section 3 presents the implementation approach. Section 4 explains the evaluation method and the results, while Section 5 provides the limitations and discussion. Finally, Section 6 concludes this paper and outlines future research directions.

2. The Overview of the Proposed Architecture

The proposed MXC-P was designed and developed as a software package. Figure 3 shows the overall proposed architecture of the MXC-P. It consists of four main parts, namely an educational content engine, which is used to co-design and create educational content, the users and developers, which include all players in the system, long- and short-term analysis engines, which provide data for analysis and evaluation, and a cloud-based immersive interactive environment, which is the main location of the MXC-P. During the development of the system, a set of fundamental functionalities and a set of software requirements for the system were identified to ensure coherent software generation, resulting in a highly modular and editable system. A new module can be added to the system by following sets of defined requirements in the development of new modules. There are some technical challenges involved in MXC-P development, such as the exchange of data among modules and users, which is resolved by assigning a database to each module and user profile and considering a data server with direct access to all the databases through an authentication and verification process to manage all the shared data in the program. Each layer is explained as follows.

2.1. Users and Developers

The user and developer parts include all direct and indirect individuals who engage with a CLF. This encompasses workers responsible for performing processes such as unloading, sorting, and repairing pallets; they are the main users of training materials, as well as the MXC-P. Managers are responsible for managing the CLF and facilitating the training materials. Officers are responsible for providing and administering training materials to workers. Safety officers are responsible for evaluating and validating the training materials. Researchers are working on improving training materials, in conjunction with policy makers.

2.2. Educational Content Engine

The educational content engine (ECE) is responsible for providing educational content for each module, such as objectives and scenarios. To design each module, a step-by-step co-design process is applied. A co-design approach is defined as a process of collaborative design thinking, involving joint inquiry and imagination in which diverse people jointly explore and define a problem and jointly develop and evaluate solutions [31]. Co-design is emerging as the best methodological approach for designing health services and educational games [32], and the solutions designed through this process appear more likely to be successful and sustainable. Following the co-design approach, the ECE has four sequential main steps, namely to empathize, define, ideate, and prototype. Each step is defined as follows.

Empathize: This step involved a mixed-method study to describe and capture the existing safety training methods and materials for CLFs. It helped to capture data from files by applying a time study method and an interview with workers and domain expert. This approach provided information regarding the process, needs, and safety index related to each activity. This step helped us to define proper requirements for safety education based on the existing activities and process. It had three main phases. The preparation phase captured data from the field by applying the time study method and the interview process with workers and experts in the field and provided data regarding the process and safety index for each process. The investigative phase aimed to provide insights on the data and identify short-term and long-term safety aspects related to each activity through data which were delivered by the preparation phase as well as existing training content and materials. In addition, it was responsible for understanding the experiences and empowerment needs of people. The development phase aimed to achieve three objectives: first, to prioritize addressable safety needs based on short- and long-term aspects; second, to propose examples and possible scenarios for hands-on training of content for each module; and third, to assess the acceptability and usability of each module.
Define: This step assisted us in discussing and analyzing the information we obtained from the empathize step to create actionable problem and objective statements. It enabled us to prioritize the safety training needs for a specific audience. All end users, such as workers, managers, and safety officers, played direct roles in formulating priorities and problem statements. Table 2 shows the key actionable problem statements.
Ideate: Within the context of the problem statements and prioritizing the safety training needs, we generated conceptual design alternatives [12]. In this process, we utilized brainstorming and mind-mapping exercises, followed by convergent thinking, to synthesize and refine collections of ideas into cohesive module concepts. We shared the generated concepts with officers, workers, researchers, and policymakers to improve the modules based on their feedback. The entire ideate step led to the development of a conceptual model of modules and training content for each module. In this step, we proposed three main modules to represent key concepts and training materials for CLFs, namely the PPE module, the PH module, and the PR module.
Prototype: In this step, the model of each module was validated for its conceptualization and appropriateness and subsequently refined. The objective of this stage was to initiate evaluation, reflection, and learning and typically to develop a single prototype of each module, which was required for the testing or implementation phase. Based on the ideate step, three modules were defined and, in the prototype step, each module was designed and developed.

2.3. Modular Virtual Collaborative Platform

The MXC-P was designed and developed as a software package in order to satisfy multiple users in a geographically distributed system. To enable users to connect to the platform remotely, we adopted a cloud network-based approach. Figure 4 shows the network properties and topology. To implement the proposed network topology, we utilized the Fusion API, which is designed to resemble regular Unity MonoBehaviour code [33]. Developers and officers are defined as authorized users on the platform who can play key roles in modifying the modules or activating specific modules for workers, who are defined as normal users on the platform.

The MXC-P has three main modules which were developed in the ECE. Each module is explained as follows.

The PPE module is focused on providing immersive, audio-based training regarding gloves, shoes, glasses, high-visibility vests, and hearing protection to CLF workers. Training materials related to each piece of personal equipment were designed based on the first three steps of the co-design method: empathize, define, and ideate. This approach allowed us to fully customize this information for CLF workers. Each user has the capability to immerse themselves and interact with each personal protection resource in order to improve their understanding of them. Figure 5 shows the key training materials and the characteristics of the PPE module.

Figure 6 illustrates the key characteristics of the PH module, which serves as a conductor for safety training spanning short-term and long-term dimensions, leveraging immersive and experiential pedagogical methodologies. The core instructional content encompasses pallet lifting, stacking, and sorting techniques. This module comprises three principal instructional modules, employing a hybrid instructional approach for knowledge dissemination. Initially, trainees engage with instructional videos pertinent to each task, subsequently transitioning to practical application within a virtual reality milieu, thus facilitating heightened cognitive assimilation and skill mastery.

Figure 7 depicts the PR module, concentrating on both tool and process safety in the context of pallet repair. This module is entirely hands-on, with activities such as nailing, board removal, and pallet cleaning being conducted within a virtual reality environment. Its primary objective is to facilitate worker comprehension of the repair processes while adhering to safety regulations. This module adopts an approach that integrates short-term and long-term safety considerations, thereby offering an optimal method for tool handling. Each tool is designed and developed to emulate real-world mechanisms, providing users with an authentic experience conducive to skill acquisition and safety awareness.

2.4. Analysis Engine

The analysis engine (AE) interfaces with the MXC-P to furnish training data to the CLF and a researcher in order to create the user portal, which includes the result of each module and personal progress. The AE gives CLF managers the ability to access the training data of workers across geographically distributed CLFs. Researchers could utilize these data to assess long-term safety aspects, particularly focusing on the PH and PR modules, within each process. Officers can use this engine to follow up training for work or evaluate the work based on the training performance.

3. Implementation

To implement the proposed platform, Unity3D was used as the main platform to create the interactive environment, Blender open source 3D modeling software was utilized to create 3D assets and textures, and we utilized the XR Interaction Toolkit framework, which is a high-level, component-based interactive system for creating VR and AR experiences. Photon Fusion 2 was used for networking and the C# language was used as the main programing language. The choice of C# as the primary programming language ensured robust control over the development process, allowing us to implement complex interaction systems, user interfaces, and data management functionalities. The platform was developed for OpenXR API, which supports both Windows and Android devices.

Avatars are one of the most important components of tools. They are virtual representations of humans, performing social roles in the MXC-P, corresponding to their roles and personas. Based on different topologies and abstractions, each service user can have different types of avatars. For example, Figure 8 shows the main factors of avatars in the MXC-P. There is more emphasis on ergonomics and human factors in MXC-P avatars than on facial representation. This is because through full-body avatars, we can capture long-term health information. In this study, we developed two different avatars based on abstraction and topology. The first one is based on high abstraction and low topology, which is used by officers or managers. The second one has low abstraction and high topology. This avatar contains 82 joints, Figure 8 shows the kinematic configuration of the avatar’s body and hands. Kinematic equations are used to obtain the positions of each joint according to the joint angles [34]. The kinematic equations applied followed the Denavit–Hartenberg (D-H) parameters [35]. This convention is commonly used for mechanism and robotics modeling. In this research, we based the human body kinematics on two main equations, the kinematic equation for the hand and the four-segment human body kinematic equation. Figure 8 shows the hand and body models which we used in the avatar.

Regarding the analysis of long-term health risks, the MXC-P platform incorporates an analysis engine (AE) designed to capture and process real-time data from users during their interaction with the virtual environment. The AE is integrated with a cloud-based system, allowing for the aggregation and analysis of data from multiple sessions and users. The ergonomic evaluation within the platform is conducted using a full-body avatar model, which simulates human kinematics with a focus on critical joints such as the knees, ankles, and shoulders. These data are then processed to generate ergonomic risk scores, identifying areas where the user may be at risk of long-term injury due to repetitive or awkward movements. For the analysis of these data, statistical methods are employed to assess the potential for cumulative injuries. The ergonomic risk scores are plotted against operation sequences, allowing for the identification of tasks that pose a higher risk to specific joints or muscle groups. In the case study presented, it was observed that certain pallet handling tasks led to “urgent to change” scores for the knees and ankles, indicating a significant risk for long-term health issues in these areas. This insight allows us to propose adjustments to training modules to focus on safer handling techniques and reduce the risk of chronic injuries.

4. Evaluation

4.1. Learning Evaluation

For the evaluation and verification of the proposed architecture, we utilized a quasi-experimental study [36]. We recruited 24 volunteers for this case study, aged 21 to 35. None of the participants had any prior scaling experience related to CLFs. For the purpose of comparative case studies, all the participants filled out an online questionnaire (see Appendix A), then we divided them into two separate groups, namely control and experimental groups. Each group comprised 12 participants. The control group were trained through existing safety training materials which were provided by our industry collaborator.

The participants in the experimental group were asked to use MXC-P to complete their safety training. First, they received guidance and experienced the operations of virtual reality devices to ensure that they were familiar with the virtual environment and virtual operations. Training started first with the PPE module, then the PH module and finally the PR module. Additional evaluation based on defined VR-specific scales was performed separately using the survey results, and the results are presented in Figure 9. We utilized an heuristic evaluation method [37] to identify design problems in a user interface. The applied approach involved gathering input from users to evaluate their experience using a Likert scale, ranging from strongly agree to strongly disagree, across various categories relevant to the immersive experience on the platform. Figure 9 presents the mean score of the sample for each category. Notice that in all categories, the score was 4 or above, meaning that on average, the group agreed that they felt conformable interacting in the proposed user interface. While this is a positive result, it also provides vital insights into what can be improved for a future design of this experiment. For example, enhancements could be applied to the visual design of objects and task, as well as to the interaction with these objects in the task, to improve the categories of ‘faithful viewpoint’ and ‘realistic feedback’. By ensuring a coherent standard across the platform with a clear interface that helps the user to complete the task, we will address the categories of ‘consistent departures’ and ‘navigation and orientation support.’

After training, both groups were asked to fill out the same Likert-type questionnaire with responses from strongly agree to strongly disagree (see Appendix B) to evaluate their own learning outcomes.

Table 3 shows the demographic information of the participants; the majority of participants were between 19 and 25 years old, with minimum CLF experience, which is highly representation of the majority of workers in CLFs.

In order to evaluate our participants’ learning outcomes using the MXC-P, statistical software was used to analyze our data. In the first step, an independent sample t-test was conducted to compare the learning outcomes between the control and experimental groups. The results indicated statistically significant differences between the two groups (p-value < 0.05), indicating that the MXC-P provides a more effective learning environment for the experimental group compared to the control group. Table 4 summarizes our results. It is worth mentioning that participants’ understanding of long-term safety (Q8) displayed the lowest mean within the experimental group (M = 3.83), although this is still higher than the control group (M = 1.17). This suggests that while the MXC-P approach improves understanding of long-term safety, there is still room for improvement in this area. In the control group, the questions Q2 (1.5), Q3 (1.58), Q6 (1.48), Q7 (1.58), Q8 (1.17), Q9 (1.42), and Q10 (1.5) had mean scores less than the acceptable level, indicating that learning in the control group is not at an acceptable level.

Cohen’s d effect size for all questions was greater than the medium effect size (d = 0.5), indicating that the effect sizes for these questions are substantial.

4.2. Long-Term Safety Evaluation

The real-time ergonomic evaluation of the PH module via the full body avatar is shown in Figure 10, plotted with the operation sequence on the horizontal axis and the ergonomic evaluation score on the vertical axis. As Figure 10 shows, there are potential long-term health issues with the knees and ankles. During pallet handling, the knees twice reached the “urgent to change” area in the ergonomic evaluation score, indicating the need to propose better pallet handling approaches to reduce the long-term health impact on the knees. This occurred once for the ankles, indicating that the ankles in the pallet handling module could develop long-term issues in the future. We can also conclude that the shoulders would experience minimal impact regarding long-term health issues and would mostly remain in the acceptable range, which is a good sign for shoulder health. Figure 11 shows the ergonomic evaluation of the PH module based on the left and right hands. We can conclude that the right hand performed more movements than the left hand, which might cause long-term health issues, as the score crossed into the “urgent to change” area.

While the initial results from our study demonstrate the effectiveness of the MXC-P platform in improving safety training outcomes, it is crucial to consider the long-term efficacy of the training provided. The effectiveness of safety training can diminish over time if the skills and knowledge acquired are not regularly reinforced. Therefore, a refresher course is mandatory every six months at CLFs. The MXC-P platform, developed as a cloud-based system with a user portal, has the capacity to incorporate these refresher courses and continuous learning modules.

These refresher courses can be scheduled at regular intervals based on the specific needs of CLF workers and the nature of the tasks they perform. By integrating periodic training updates, the platform ensures that workers’ knowledge remains current and that safety practices are consistently reinforced. Additionally, the platform can track individual progress and identify areas where further training may be required, allowing for a personalized approach to long-term training efficacy.

The long-term health risks identified through the ergonomic evaluation of tasks, such as those related to knee and ankle strain during pallet handling, can also be mitigated by incorporating targeted training modules that focus on proper body mechanics and safe handling techniques. These modules can be revisited periodically to reinforce safe practices and reduce the risk of cumulative injuries.

5. Limitations of the Work

This study has several limitations. One significant limitation is the need for a larger and more diverse sample size, as the current number of participants and their range are limited. Existing research in the field [16,22] demonstrates that small sample sizes are often accepted as sufficient for preliminary stages of evaluation, even when the sample is divided into several groups [29]. Furthermore, in [15], it is stated that for VR training, it is desirable to use small samples so that all of the trainees can benefit from the high degree of customization that XR technologies allow. This acceptance of small samples in the literature is explained, since the goal is to explore feasibility or gather initial insights [38,39]. In early-stage studies like ours, the primary focus is on identifying potential trends, testing methodologies, and refining experimental designs, rather than achieving broad generalizability. This approach allows researchers to make informed decisions about whether to pursue larger-scale studies and to finetune the study design for subsequent, more comprehensive research. The data capturing method and avatar enrichment could be enhanced by incorporating more sensitive sensors and advanced video analysis algorithms. The bias in data regarding joint movements could be mitigated by using a more sophisticated video analysis system. The case study and platform were developed with industrial workers in mind, especially in CLFs, and thus how well the method might apply to other areas has not been determined. The long-term safety evaluation can be improved by integrating predication methods using effective machine learning approaches.

6. Conclusions

This study addresses the critical need for improved safety and health training in the warehouse and logistics sectors, particularly within CLFs. Despite advancements in safety protocols and training methods, warehouse accidents and injuries remain significant concerns, necessitating innovative solutions. Our research presents the development of an MXC-P designed to enhance safety training through immersive and interactive virtual environments. The MXC-P is structured into three primary modules: personal protection equipment (PPE), pallet handling (PH), and pallet repairing (PR). These modules were developed using a co-design process involving safety professionals, CLF managers, and researchers to ensure relevance and effectiveness. The platform provides a collaborative and remote training environment, offering hands-on experience and generating data for long-term health risk evaluation.

A pilot study comparing the MXC-P with traditional safety training methods demonstrated the platform’s effectiveness in improving safety learning and knowledge recall. Trainees using the MXC-P showed better engagement and comprehension, indicating that virtual collaborative training can significantly enhance learning outcomes. The MXC-P also offers the potential for widespread application across geographically distributed systems, making it a cost-effective solution for training large workforces. The ability to generate and analyze data on long-term health risks further underscores the platform’s value in promoting occupational health.

The results from the quasi-experimental study indicate that the MXC-P significantly enhances the learning outcomes for participants compared to traditional training methods. This was evidenced by the statistically significant differences observed in the independent sample t-tests between the experimental and control groups. The experimental group, which used the MXC-P platform, showed higher scores in all evaluated dimensions, including the understanding of short-term safety practices, engagement with training materials, and knowledge retention. Specifically, the data demonstrated that the MXC-P was particularly effective in areas such as the proper use of personal protective equipment (PPE) and pallet handling techniques. This suggests that immersive and interactive environments can better reinforce critical safety practices.

Another important point that emerges from the results mentioned above is that the use of immersive environments for safety training opens up encouraging avenues of research for other areas within the company, in particular how the MXC-P can be used in different scenarios, such as the risk assessment of load manipulation while loading/unloading a container, the appropriate and safe use of forklifts and other vehicles within a warehouse, and proper crane manipulation. This will allow researchers to create more studies for the logistics industry, a sector where the literature for safety training and virtual training is scant. Moreover, introducing a well-functioning modular, virtual, and collaborative platform, with promising preliminary results, represents a step forward in how companies, from different industries, can think, design, and deliver safety training and training in general and leverage MXC platforms.

However, the conclusions drawn from this study must be considered in the context of its limitations. The small sample size and the preliminary nature of the study mean that further research is necessary to confirm these findings. Future studies should involve larger and more diverse participant groups, as well as a longer follow-up period to assess the long-term impact of the training.

In conclusion, the MXC-P represents a significant advancement in safety training for warehouse environments. By leveraging virtual reality and a collaborative design process, the platform provides a robust and scalable solution to address both immediate safety concerns and long-term health risks. Future research will focus on expanding the dataset and refining the platform with advanced sensors and video analysis to enhance training efficacy and health monitoring.

As future work, we plan to address this limitation by increasing the sample size in subsequent studies. This will enable more robust statistical analyses and enhance the reliability and generalizability of our findings, ultimately strengthening the validity of our proposed approach.

Author Contributions

Conceptualization, A.V.B. and B.M.; methodology, A.V.B. and J.E.G.; software, A.V.B.; validation, A.V.B., J.E.G. and B.M.; formal analysis, A.V.B.; investigation, A.V.B. and J.E.G.; resources, A.V.B. and J.E.G.; data curation, A.V.B.; writing—original draft preparation, A.V.B., J.E.G. and B.M.; writing—review and editing, A.V.B. and J.E.G.; visualization, A.V.B. and J.E.G.; supervision, B.M.; project administration, A.V.B. and B.M.; funding acquisition, B.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Relogistics Industry grant number 5787125. NO APC.

Data Availability Statement

The data presented in this study are available on request from the corresponding author option.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

What is your name?
What is your age?
What is your gender?
What is your occupation?
What is your educational background?
Level of knowledge about warehouse safety (Novice, Intermediate, Advanced)
Experience with virtual reality devices. (Novice, Some, Intermediate, Advanced)

Appendix B

I am satisfied with this training. (Strongly disagree, Disagree, Neutral, Agree and Strongly agree)
I am satisfied with the level of hands-on learning that I experienced through this training.
I believe this training prepared me enough for working in the real CLF environments.
Training motivated me to learn more about the safety aspect of CLF.
Training helped me to understand the key safety equipment.
Training helped me to learn the right pallet handling method.
Training provided a better visualized level of detail to meet the objectives.
Training helped me gain a better understanding of long-term safety.
Training provided a more effective way to learn how to repair pallets.
Training provided me enough knowledge and experience about CLF safety.

Appendix C

https://www.youtube.com/watch?v=K9Hd9qCQ_Co, accessed on 3 September 2024.

References

Roy, D.; Carrano, A.L.; Pazour, J.A.; Gupta, A. Cost-effective pallet management strategies. Transp. Res. Part E Logist. Transp. Rev. 2016, 93, 358–371. [Google Scholar] [CrossRef]
Tornese, F.; Gnoni, M.G.; Thorn, B.K.; Carrano, A.L.; Pazour, J.A. Management and logistics of returnable transport items: A review analysis on the pallet supply chain. Sustainability 2021, 13, 12747. [Google Scholar] [CrossRef]
Tornese, F.; Pazour, J.A.; Thorn, B.K.; Roy, D.; Carrano, A.L. Investigating the environmental and economic impact of loading conditions and repositioning strategies for pallet pooling providers. J. Clean. Prod. 2018, 172, 155–168. [Google Scholar] [CrossRef]
de Koster, R. Warehousing 2030. In Global Logistics and Supply Chain Strategies for the 2020s: Vital Skills for the Next Generation; Springer: Berlin/Heidelberg, Germany, 2022; pp. 243–260. [Google Scholar]
Lin, Y.-S.; Chai, C.-W.; Chao, T.-W. Case study on the safety and disaster prevention system of factory intelligent warehouse. In Proceedings of the 2022 IEEE 5th Eurasian Conference on Educational Innovation (ECEI), Taipei, Taiwan, 10–12 February 2022; IEEE: Piscataway, NJ, USA, 2022. [Google Scholar]
McGarity, T.; Duff, M.C.; Shapiro, S.A. Center for Progressive Reform Report: Protecting Workers in A Pandemic—What The Federal Government Should Be Doing (17 June 2020); Center for Progressive Reform Report; Saint Louis University School of Law: St. Louis, MS, USA, 2020. [Google Scholar]
LaRocco, L.A. Federal Report Says Lack of OSHA Inspections Puts Warehouse Workers at Risk; Warehouse: Balzac, AB, Canada, 2023. [Google Scholar]
Brewster, M. Annual Retail Trade Survey Shows Impact of Online Shopping on Retail Sales During COVID-19 Pandemic. In E-Commerce Sales Surged During the Pandemic; Census, Ed.; United States Census Bureau: Suitland-Silver Hill, MD, USA, 2023. Available online: https://www.census.gov/library/stories/2022/04/ (accessed on 3 September 2024).
Gutelius, B.; Pinto, S. Pain Points: Data on Work Intensity, Monitoring, and Health at Amazon Warehouses; Center for Urban Economic Development: Chicago, IL, USA, 2023. [Google Scholar]
U.S. Department of Labor. Employer-Reported Workplace Injuries and Illnesses—2021–2022. In Survey of Occupational Injuries and Illnesses, in Cooperation with Participating State Agencies; U.S. Department of Labor, Bureau of Labor Statistics: Washington, DC, USA, 2023; p. 1. [Google Scholar]
Feng, Z.; González, V.A.; Mutch, C.; Amor, R.; Rahouti, A.; Baghouz, A.; Li, N.; Cabrera-Guerrero, G. Towards a customizable immersive virtual reality serious game for earthquake emergency training. Adv. Eng. Inform. 2020, 46, 101134. [Google Scholar] [CrossRef]
Altass, P.; Wiebe, S. Re-imagining education policy and practice in the digital era. J. Can. Assoc. Curric. Stud. 2017, 15, 48–63. [Google Scholar] [CrossRef]
Rokooei, S.; Shojaei, A.; Alvanchi, A.; Azad, R.; Didehvar, N. Virtual reality application for construction safety training. Saf. Sci. 2023, 157, 105925. [Google Scholar] [CrossRef]
Lacko, J. Health safety training for industry in virtual reality. In Proceedings of the 2020 Cybernetics & Informatics (K&I), Velke Karlovice, Czech Republic, 29 January–1 February 2020; IEEE: Piscataway, NJ, USA, 2020. [Google Scholar]
Sacks, R.; Perlman, A.; Barak, R. Construction safety training using immersive virtual reality. Constr. Manag. Econ. 2013, 31, 1005–1017. [Google Scholar] [CrossRef]
Arachchige, S.N.K.; Chander, H.; Turner, A.J.; Shojaei, A.; Knight, A.C.; Griffith, A.; Burch, R.F.; Chen, C.-C. Physiological and Subjective Measures of Anxiety with Repeated Exposure to Virtual Construction Sites at Different Heights. Saf. Health Work. 2023, 14, 303–308. [Google Scholar] [CrossRef]
Shringi, A.; Arashpour, M.; Golafshani, E.M.; Dwyer, T.; Kalutara, P. Enhancing Safety Training Performance Using Extended Reality: A Hybrid Delphi–AHP Multi-Attribute Analysis in a Type-2 Fuzzy Environment. Buildings 2023, 13, 625. [Google Scholar] [CrossRef]
Ji, Z.; Wang, Y.; Zhang, Y.; Gao, Y.; Cao, Y.; Yang, S.-H. Integrating diminished quality of life with virtual reality for occupational health and safety training. Saf. Sci. 2023, 158, 105999. [Google Scholar] [CrossRef]
Stefan, H.; Mortimer, M.; Horan, B.; Kenny, G. Evaluating the preliminary effectiveness of industrial virtual reality safety training for ozone generator isolation procedure. Saf. Sci. 2023, 163, 106125. [Google Scholar] [CrossRef]
Man, S.S.; Wen, H.; So, B.C.L. Are virtual reality applications effective for construction safety training and education? A systematic review and meta-analysis. J. Saf. Res. 2023, 88, 230–243. [Google Scholar] [CrossRef] [PubMed]
Gauthier, S.; Leduc, M.; Perfetto, S.J.; Godwin, A. Use of virtual reality to increase awareness of line-of-sight hazards around industrial equipment. Safety 2022, 8, 52. [Google Scholar] [CrossRef]
Stefan, H.; Mortimer, M.; Horan, B.; McMillan, S. How effective is virtual reality for electrical safety training? Evaluating trainees’ reactions, learning, and training duration. J. Saf. Res. 2024, 90, 48–61. [Google Scholar] [CrossRef]
Alaker, M.; Wynn, G.R.; Arulampalam, T. Virtual reality training in laparoscopic surgery: A systematic review & meta-analysis. Int. J. Surg. 2016, 29, 85–94. [Google Scholar]
Scorgie, D.; Feng, Z.; Paes, D.; Parisi, F.; Yiu, T.; Lovreglio, R. Virtual reality for safety training: A systematic literature review and meta-analysis. Saf. Sci. 2024, 171, 106372. [Google Scholar] [CrossRef]
Gong, P.; Lu, Y.; Lovreglio, R.; Lv, X.; Chi, Z. Applications and effectiveness of augmented reality in safety training: A systematic literature review and meta-analysis. Saf. Sci. 2024, 178, 106624. [Google Scholar] [CrossRef]
Wilkins, H.V.; Spikmans, V.; Ebeyan, R.; Riley, B. Application of augmented reality for crime scene investigation training and education. Sci. Justice 2024, 64, 289–296. [Google Scholar] [CrossRef] [PubMed]
Steven, L.; Hauw, J.K.; Keane, M.B.; Gunawan, A.A.S. Empowering military in tactical and warfare area with virtual reality technology: A systematic literature review. Procedia Comput. Sci. 2023, 227, 892–901. [Google Scholar] [CrossRef]
Piñal, O.; Arguelles, A. Mixed reality and digital twins for astronaut training. Acta Astronaut. 2024, 219, 376–391. [Google Scholar] [CrossRef]
Bolierakis, S.N.; Kostovasili, M.; Karagiannidis, L.; Amditis, A. Training on LSA lifeboat operation using Mixed Reality. Virtual Real. Intell. Hardw. 2023, 5, 201–212. [Google Scholar] [CrossRef]
Jain, S.; Timofeev, I.; Kirollos, R.W.; Helmy, A. Use of mixed reality in neurosurgery training: A single centre experience. World Neurosurg. 2023, 176, e68–e76. [Google Scholar] [CrossRef] [PubMed]
Zamenopoulos, T.; Alexiou, K. Co-Design as Collaborative Research; Bristol University/AHRC Connected Communities Programme: Bristol, UK, 2018. [Google Scholar]
Koren, I.; Hensen, B.; Klamma, R. Co-design of gamified mixed reality applications. In Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Munich, Germany, 16–20 October 2018; IEEE: Piscataway, NJ, USA, 2018. [Google Scholar]
Photon. Fusion 2 Introduction; Photon Engine: Hamburg, Germany, 2023. [Google Scholar]
Cobos, S.; Ferre, M.; Uran, M.S.; Ortego, J.; Pena, C. Efficient human hand kinematics for manipulation tasks. In Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France, 22–26 September 2008; IEEE: Piscataway, NJ, USA, 2008. [Google Scholar]
Rocha, C.; Tonetto, C.; Dias, A. A comparison between the Denavit–Hartenberg and the screw-based methods used in kinematic modeling of robot manipulators. Robot. Comput.-Integr. Manuf. 2011, 27, 723–728. [Google Scholar] [CrossRef]
Campbell, D.T.; Stanley, J.C. Experimental and Quasi-Experimental Designs for Research; Ravenio Books: London, UK, 2015. [Google Scholar]
Yeratziotis, A.; Zaphiris, P. A heuristic evaluation for deaf web user experience (HE4DWUX). Int. J. Hum.-Comput. Interact. 2018, 34, 195–217. [Google Scholar] [CrossRef]
Bujang, M.A.; Omar, E.D.; Foo, D.H.P.; Hon, Y.K. Sample size determination for conducting a pilot study to assess reliability of a questionnaire. Restor. Dent. Endod. 2024, 49, e3. [Google Scholar] [CrossRef]
Hertzog, M.A. Considerations in determining sample size for pilot studies. Res. Nurs. Health 2008, 31, 180–191. [Google Scholar] [CrossRef]

Figure 1. Representative operational workflow of a CLF.

Figure 2. Occupational injuries and illnesses by type of case [10].

Figure 3. Overview of the proposed architecture.

Figure 4. Network properties and topology of the MXC-P.

Figure 5. Key Training and Characteristics of PPE Module.

Figure 6. Key training and characteristics of the PH module.

Figure 7. Key training and characteristics of the PR module.

Figure 8. Kinematic chain of the hand and the body model. Joints defined as revolute-type.

Figure 9. Evaluation of the MXC-P based on the heuristic evaluation method.

Figure 10. Results of full body movement with ergonomic risk evaluation for five main joints.

Figure 11. Results of hand joint movement with ergonomic risk evaluation.

Table 1. Comparative studies of XR technologies and traditional training methods by domain and industry.

XR-Technologies vs. Alternatives	Domain of Operation	Industry	Key Results	Reference
VR vs. lecture-based training	Safety training	Construction	VR has provided statistically significant results, being at least as effective as or better than traditional methods for immediate learning and over time. A key element for an effective learning experience in VR is maintaining smaller groups of trainees. Further research is needed to determine the optimal group size.	[15]
VR study only	Safety training	Construction	Stressful environmental conditions can negatively affect cognitive processing. In such cases, VR has been demonstrated to be an excellent tool to reduce the affected cognition.	[16]
VR vs. AR vs. lecture-based training	Safety training	Construction	Conventional methods of training are, by far, an inefficient method of training compared to XR alternatives (VR or AR).	[17]
VR vs. lecture-based training	Safety training	Chemical	The study shows that, for the homogeneous sample used, VR training outperforms conventional methods in both short-term and long-term hazard identification and risk perception.	[18]
VR vs. lecture-based training	Safety training	Water treatment	The VR alternative took significantly less time to share the same content without compromising the knowledge acquired. Other positive findings of the VR alternative are an overwhelming positive reaction of trainees and the easier accessibility that it provides.	[19]
VR vs. video training VR vs. paper-based training VR vs. lecture-based training	Safety training	Construction	This work showed that VR safety training is better than traditional methods in three dimensions: behavior, experience, and skills. Young workers with fewer years of experience benefit more from VR training than experienced workers.	[20]
VR vs. desktop training (verbal + presentation)	Safety training	Mining	While the obtained knowledge was comparable for both methods, VR training significantly improves the user confidence to perform the evaluated task.	[21]
VR vs. lecture-based training	Safety training	Industrial Electrical	VR improves engagement and enjoyment of training, strengthening the learning experience. Score evaluations immediately after and four weeks later were significantly better compared to traditional methods.	[22]
VR vs. video training	Operation training and best practices	Medical	VR simulation enhances operative performance and shortens operative times. Immediate feedback also boosts training quality	[23]
VR vs. video training VR vs. paper-based training VR vs. lecture-based training	Safety training	Construction, Fire Safety, Aviation, Mining	Construction and fire safety training are the most studied industries in the literature. VR safety training methods are more effective than traditional methods for both knowledge acquisition and retention.	[24]
AR vs. video training AR vs. paper-based training AR vs. lecture-based training	Safety training	Construction, Manufacturing, Transportation	AR outperforms traditional methods in providing safety training and demonstrates equivalent efficacy in knowledge acquisition.	[25]
AR vs. on-site training	Operation training and best practices	Forensic science	Easy accessibility to training and re-training material is fundamental for information retention, a feature that AR offers but on-site training cannot. AR is a highly customizable tool that enables the generation of multiple training scenarios in virtually no time.	[26]
AR vs. on-site training	Tactical and warfare operations training	Military	AR can serve as a platform to train military forces without compromising the effectiveness of the training. While the results seem promising, more AR studies are required, as only a few countries are currently exploring this area.	[27]
MR vs. simulated environments	Operation training and best practices	Aerospace	A multi-module approach has shown that MR can create suitable environments for astronauts to conduct their training at a fraction of the actual cost and with high customization. Results also suggest that combining MR with digital twins can help to obtain relevant KPIs immediately.	[28]
MR vs. on-site training	Operation and maintenance training	Maritime	MR technology enables the feasible and accessible generation of virtual training scenarios as well as the enhancement of physical settings. No significant differences in the gained knowledge, but MR has been proven to close the gap in training accessibility, providing an ‘everyone, everywhere’ experience.	[29]
MR study only	Operation training and best practices	Medical	MR technology is able to provide a significant training experience, even in complex scenarios like neurosurgery. MR has been proven to help identify relationships between complex variables that were difficult to grasp with 2D and 3D images	[30]

Table 2. Key actionable problem statements.

Level of Action	Key Actionable Problem Statements
Worker level	Inadequate hand-on learning materials Poor training material and delivery method Limited confidence of workers in communication with safety officers and managers Poor self-learning practices
Manager level	Poor communication between officers and managers Limited resources and contents Lack of integration between learning materials and manager’s level High cost of training Low satisfaction
Safety officer level	There is a limited number of officers available. Unavailability of training long term health issues Lack of information, educational, communication training materials

Table 3. Demographic information of the participants.

Category	Details	Male	Female	Total (n)
Age	19–25	8	4	12
	26–30	4	4	8
	31–35	2	2	4
Occupation	Full-time	2	2	4
	Part-time	8	4	12
	Student	4	4	8
Level of Knowledge on Warehouse Safety	Novice	6	8	14
	Intermediate	5	2	7
	Advanced	3	0	3
Experience with Virtual Reality	Novice	6	4	10
	Intermediate	5	5	10
	Advanced	4	0	4

Table 4. Descriptive statistics for both the experimental and control group.

Question	Group	Mean	Std. Deviation	t-Value	p-Value	Cohen’s d/Standardizer
Q1	Experimental	4.170	0.835	3.331	0.003	0.919
	Control	2.920	0.996
Q2	Experimental	4.670	0.492	13.140	<0.001	0.590
	Control	1.500	0.674
Q3	Experimental	3.580	0.793	6.680	<0.001	0.733
	Control	1.580	0.669
Q4	Experimental	4.330	0.778	7.088	<0.001	0.749
	Control	2.170	0.718
Q5	Experimental	4.500	0.674	6.780	<0.001	0.696
	Control	2.600	0.750
Q6	Experimental	4.170	0.718	9.120	<0.001	0.694
	Control	1.480	0.669
Q7	Experimental	4.670	0.492	3.083	<0.001	0.587
	Control	1.580	0.669
Q8	Experimental	3.830	0.835	2.600	<0.001	0.651
	Control	1.170	0.389
Q9	Experimental	4.080	0.669	2.667	<0.001	0.597
	Control	1.420	0.515
Q10	Experimental	4.500	0.522	3.000	<0.001	0.522
	Control	1.500	0.522

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Vatankhah Barenji, A.; Garcia, J.E.; Montreuil, B. A Modular XR Collaborative Platform for Occupational Safety and Health Training: A Case Study in Circular Logistics Facilities. Information 2024, 15, 570. https://doi.org/10.3390/info15090570

AMA Style

Vatankhah Barenji A, Garcia JE, Montreuil B. A Modular XR Collaborative Platform for Occupational Safety and Health Training: A Case Study in Circular Logistics Facilities. Information. 2024; 15(9):570. https://doi.org/10.3390/info15090570

Chicago/Turabian Style

Vatankhah Barenji, Ali, Jorge E. Garcia, and Benoit Montreuil. 2024. "A Modular XR Collaborative Platform for Occupational Safety and Health Training: A Case Study in Circular Logistics Facilities" Information 15, no. 9: 570. https://doi.org/10.3390/info15090570

APA Style

Vatankhah Barenji, A., Garcia, J. E., & Montreuil, B. (2024). A Modular XR Collaborative Platform for Occupational Safety and Health Training: A Case Study in Circular Logistics Facilities. Information, 15(9), 570. https://doi.org/10.3390/info15090570

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu