Search Results (302)

Telerobotics involves the operation of robots from a distance, often using advanced communication technologies combining wireless and wired technologies and a variety of protocols. This application domain is crucial because it allows humans to interact with and control robotic systems safely and from a distance, often performing activities in hazardous or inaccessible environments. Thus, by enabling remote operations, telerobotics not only enhances safety but also expands the possibilities for medical and industrial applications. In some use cases, telerobotics bridges the gap between human skill and robotic precision, making the completion of complex tasks requiring high accuracy possible without being physically present. With the growing availability of high-speed networks around the world, especially with the advent of 5G cellular technologies, applications of telerobotics can now span a gamut of scenarios ranging from remote control in the same room to robotic control across the globe. However, there are a variety of factors that can impact the control precision of the robotic platform and user experience of the teleoperator. One such critical factor is latency, especially across large geographical areas or complex network topologies. Consequently, military telerobotics and remote operations, for example, rely on dedicated communications infrastructure for such tasks. However, this creates a barrier to entry for many other applications and domains, as the cost of dedicated infrastructure would be prohibitive. In this paper, we examine the network latency of robotic control over shared network resources in a variety of network settings, such as a local network, access-controlled networks through Wi-Fi and cellular, and a remote transatlantic connection between Finland and the United States. The aim of this study is to quantify and evaluate the constituent latency components that comprise the control feedback loop of this telerobotics experience—of a camera feed for an operator to observe the telerobotic platform’s environment in one direction and the control communications from the operator to the robot in the reverse direction. The results show stable average round-trip latency of 6.6 ms for local network connection, 58.4 ms when connecting over Wi-Fi, 115.4 ms when connecting through cellular, and 240.7 ms when connecting from Finland to the United States over a VPN access-controlled network. These findings provide a better understanding of the capabilities and performance limitations of conducting telerobotics activities over commodity networks, and lay the foundation of our future work to use these insights for optimizing the overall user experience and the responsiveness of this control loop. Full article

Background: As the Internet of Things (IoT) expands, it enables new forms of communication, including interactions mediated by teleoperated robots like avatars. While extensive research exists on the effects of these devices on communication partners, there is limited research on the impact on the operators themselves. This study aimed to objectively assess the psychological and physiological effects of operating a teleoperated robot, specifically Telenoid, on its human operator. Methods: Twelve healthy participants (2 women and 10 men, aged 18–23 years) were recruited from Osaka University. Participants engaged in two communication sessions with a first-time partner: face-to-face and Telenoid-mediated. Telenoid is a minimalist humanoid robot teleoperated by a participant. Blood samples were collected before and after each session to measure hormonal and oxidative markers, including cortisol, diacron reactive oxygen metabolites (d-ROMs), and the biological antioxidat activity of plasma (BAP). Psychological stress was assessed using validated questionnaires (POMS-2, HADS, and SRS-18). Results: A trend of a decrease in cortisol levels was observed during Telenoid-mediated communication, whereas face-to-face interactions showed no significant changes. Oxidative stress, measured by d-ROMs, significantly increased after face-to-face interactions but not in Telenoid-mediated sessions. Significant correlations were found between oxytocin and d-ROMs and psychological stress scores, particularly in terms of helplessness and total stress measures. However, no significant changes were observed in other biomarkers or between the two conditions for most psychological measures. Conclusions: These findings suggest that cortisol and d-ROMs may serve as objective biomarkers for assessing psychophysiological stress during robot-mediated communication. Telenoid’s minimalist design may help reduce social pressures and mitigate stress compared to face-to-face interactions. Further research with larger, more diverse samples and longitudinal designs is needed to validate these findings and explore the broader impacts of teleoperated robots. Full article

Currently, teleoperated robots, with the operator’s input, can fully perceive unknown factors in a complex environment and have strong environmental interaction and perception abilities. However, physiological tremors in the human hand can seriously affect the accuracy of processes that require high-precision control. Therefore, this paper proposes an EEMD-IWOA-LSTM model, which can decompose the physiological tremor of the hand into several intrinsic modal components (IMF) by using the EEMD decomposition strategy and convert the complex nonlinear and non-stationary physiological tremor curve of the human hand into multiple simple sequences. An LSTM neural network is used to build a prediction model for each (IMF) component, and an IWOA is proposed to optimize the model, thereby improving the prediction accuracy of the physiological tremor and eliminating it. At the same time, the prediction results of this model are compared with those of different models, and the results of EEMD-IWOA-LSTM presented in this study show obvious superior performance. In the two examples, the MSE of the prediction model proposed are 0.1148 and 0.00623, respectively. The defibrillation model proposed in this study can effectively eliminate the physiological tremor of the human hand during teleoperation and improve the control accuracy of the robot during teleoperation. Full article

In this study, a Wi-Fi system with a tethered balloon is proposed for the teleoperation of vehicular forestry machines. This system was developed to establish a Wi-Fi communication for stable teleoperation in a timber harvesting site. This system consisted of a helium balloon, Wi-Fi nodes, a measurement system, a global navigation satellite system (GNSS) antenna, and a wind speed sensor. The measurement system included a GNSS module, an inertial measurement unit (IMU), a data logger, and an altitude sensor. While the helium balloon with the Wi-Fi system was 60 m in the air, the received signal strength indicator (RSSI) was measured by moving a Wi-Fi receiver on the ground. Another GNSS set was also utilized to collect the latitude and longitude data from the Wi-Fi receiver as it traveled. The developed Wi-Fi system with a tethered balloon can create a Wi-Fi zone of up to 1.9 ha within an average wind speed range of 2.2 m/s. It is also capable of performing the teleoperation of vehicular forestry machines with a maximum latency of 185.7 ms. Full article

Medical ultrasound is a widely used diagnostic imaging modality that provides real-time imaging at a relatively low cost. However, its widespread application is hindered by the need for expert operation, particularly in remote regional areas where trained sonographers are scarce. This paper presents the development of HaptiScan, a state-of-the-art telerobotic ultrasound system equipped with haptic feedback. The system utilizes a commercially available robotic manipulator, the UR5 robot from Universal Robots, integrated with a force/torque sensor and the Phantom Omni haptic device. This configuration enables skilled sonographers to remotely conduct ultrasound procedures via an internet connection, addressing both the geographic and ergonomic limitations faced in traditional sonography. Key innovative features of the system include real-time force feedback, ensuring that sonographers can precisely control the ultrasound probe from a remote location. The system is further enhanced by safety measures such as over-force sensing, patient discomfort monitoring, and emergency stop mechanisms. Quantitative indicators of the system’s performance include successful teleoperation over long distances with time delays, as demonstrated in simulations. These simulations validate the system’s control methodologies, showing stable performance with force feedback under varying time delays and distances. Additionally, the UR5 manipulator’s precision, kinematic, and dynamic models are mathematically formulated to optimize teleoperation. The results highlight the effectiveness of the proposed system in overcoming the technical challenges of remote ultrasound procedures, offering a viable solution for real-world telemedicine applications. Full article

This study aims to experimentally determine the most suitable control interface for different subtasks in the teleoperation of construction robots in a simulation environment. We compare a conventional lever-based rate control interface (“Rate-lever”) with two alternative methods: rate control (“Rate-3D”) and position control (“Position-3D”), both using a 3D positional input device. In the experiments, participants operated a construction machine in a virtual environment and evaluated the control interfaces across three tasks: sagittal plane excavation, turning, and continuous operation. The results revealed that “Position-3D” outperformed others for sagittal excavation, while both “Rate-lever” and “Rate-3D” were more effective for turning. Notably, “Position-3D” and “Rate-3D” can be implemented on the same input device and are easily integrated. This feature offers the possibility of a hybrid-type interface suitable for operators to obtain optimized performance in sagittal and horizontal tasks. Full article

This real-world investigation aimed to quantify the human–machine interaction between remote drivers of teleoperation systems and the Level 4 automated vehicle in a real-world setting. The primary goal was to investigate the effects of disengagement and distraction on remote driver performance and behaviour. Key findings revealed that mental disengagement, achieved through distraction via a reading task, significantly slowed the remote driver’s reaction time by an average of 5.309 s when the Level 4 automated system required intervention. Similarly, disengagement resulted in a 4.232 s delay in decision-making time for remote drivers when they needed to step in and make critical strategic decisions. Moreover, mental disengagement affected the remote drivers’ attention focus on the road and increased their cognitive workload compared to constant monitoring. Furthermore, when actively controlling the vehicle remotely, drivers experienced a higher cognitive workload than in both “monitoring” and “disengagement” conditions. The findings emphasize the importance of designing teleoperation systems that keep remote drivers actively engaged with their environment, minimise distractions, and reduce disengagement. Such designs are essential for enhancing safety and effectiveness in remote driving scenarios, ultimately supporting the successful deployment of Level 4 automated vehicles in real-world applications. Full article

Disassembly is a key step in remanufacturing, especially for end-of-life (EoL) products such as electric vehicle (EV) batteries, which are challenging to dismantle due to uncertainties in their condition and potential risks of fire, fumes, explosions, and electrical shock. To address these challenges, this paper presents a robotic teleoperation system that leverages augmented reality (AR) and digital twin (DT) technologies to enable a human operator to work away from the danger zone. By integrating AR and DTs, the system not only provides a real-time visual representation of the robot’s status but also enables remote control via gesture recognition. A bidirectional communication framework established within the system synchronises the virtual robot with its physical counterpart in an AR environment, which enhances the operator’s understanding of both the robot and task statuses. In the event of anomalies, the operator can interact with the virtual robot through intuitive gestures based on information displayed on the AR interface, thereby improving decision-making efficiency and operational safety. The application of this system is demonstrated through a case study involving the disassembly of a busbar from an EoL EV battery. Furthermore, the performance of the system in terms of task completion time and operator workload was evaluated and compared with that of AR-based control methods without informational cues and ‘smartpad’ controls. The findings indicate that the proposed system reduces operation time and enhances user experience, delivering its broad application potential in complex industrial settings. Full article

The demand for remote rehabilitation is increasing, opening up convenient and effective home-based therapy for the sick and elderly. In this study, we use AnyBody simulations to analyze muscle activity and determine key parameters for designing a rehabilitation exoskeleton, as well as selecting the appropriate motor torque to assist patients during rehabilitation sessions. The exoskeleton was designed with a PID control mechanism for the precise management of motor positions and joint torques, and it operates in both automated and teleoperation modes. Hip and knee movements are monitored via smartphone-based IMU sensors, enabling real-time feedback. Bluetooth communication ensures seamless control during various training scenarios. Our study demonstrates that remotely controlled rehabilitation systems can be implemented effectively, offering vital support not only during global health crises such as pandemics but also in improving the accessibility of rehabilitation services in remote or underserved areas. This approach has the potential to transform the way physical therapy can be delivered, making it more accessible and adaptable to the needs of a larger patient population. Full article

Improving the autonomy of underwater interventions by remotely operated vehicles (ROVs) can help mitigate the impact of communication delays on operational efficiency. Currently, underwater interventions for ROVs usually rely on real-time teleoperation or preprogramming by operators, which is not only time-consuming and increases the cognitive burden on operators but also requires extensive specialized programming. Instead, this paper uses the intuitive learning from demonstrations (LfD) approach that uses operator demonstrations as inputs and models the trajectory characteristics of the task through the dynamic movement primitive (DMP) approach for task reproduction as well as the generalization of knowledge to new environments. Unlike existing applications of DMP-based robot trajectory learning methods, we propose the underwater DMP (UDMP) method to address the problem that the complexity and stochasticity of underwater operational environments (e.g., current perturbations and floating operations) diminish the representativeness of the demonstrated trajectories. First, the Gaussian mixture model (GMM) and Gaussian mixture regression (GMR) are used for feature extraction of multiple demonstration trajectories to obtain typical trajectories as inputs to the DMP method. The UDMP method is more suitable for the LfD of underwater interventions than the method that directly learns the nonlinear terms of the DMP. In addition, we improve the commonly used homomorphic-based teleoperation mode to heteromorphic mode, which allows the operator to focus more on the end-operation task. Finally, the effectiveness of the developed method is verified by simulation experiments. Full article

We envision a telepresence system that enhances remote work by facilitating both physical and immersive visual interactions between individuals. However, during robot teleoperation, communication often lacks realism, as users see the robot’s body rather than the remote individual. To address this, we propose a method for overlaying a digital human model onto a humanoid robot using XR visualization, enabling an immersive 3D telepresence experience. Our approach employs a learning-based method to estimate the 2D poses of the humanoid robot from head-worn stereo views, leveraging a newly collected dataset of full-body poses for humanoid robots. The stereo 2D poses and sparse inertial measurements from the remote operator are optimized to compute 3D poses over time. The digital human is localized from the perspective of a continuously moving observer, utilizing the estimated 3D pose of the humanoid robot. Our moving camera-based pose estimation method does not rely on any markers or external knowledge of the robot’s status, effectively overcoming challenges such as marker occlusion, calibration issues, and dependencies on headset tracking errors. We demonstrate the system in a remote physical training scenario, achieving real-time performance at 40 fps, which enables simultaneous immersive and physical interactions. Experimental results show that our learning-based 3D pose estimation method, which operates without prior knowledge of the robot, significantly outperforms alternative approaches requiring the robot’s global pose, particularly during rapid headset movements, achieving markerless digital human augmentation from head-worn views. Full article

Force-feedback devices enhance task performance in most robot teleoperations. However, their increased size with additional degrees of freedom can limit the robot’s applicability. To address this, an interface that visually presents force feedback is proposed, eliminating the need for bulky physical devices. Our telepresence system renders robotic hands transparent in the camera image while displaying virtual hands. The forces applied to the robot deform these virtual hands. The deformation creates an illusion that the operator’s hands are deforming, thus providing pseudo-haptic feedback. We conducted a weight comparison experiment in a virtual reality environment to evaluate force sensitivity. In addition, we conducted an object touch experiment to assess the speed of contact detection in a robot teleoperation setting. The results demonstrate that our method significantly surpasses conventional pseudo-haptic feedback in conveying force differences. Operators detected object touch 24.7% faster using virtual hand deformation compared to conditions without feedback. This matches the response times of physical force-feedback devices. This interface not only increases the operator’s force sensitivity but also matches the performance of conventional force-feedback devices without physically constraining the operator. Therefore, the interface enhances both task performance and the experience of teleoperation. Full article

Teleoperation services are expected to operate on-road and often in urban areas. In current teleoperation applications, teleoperators gain a higher viewpoint of the environment from a camera on the vehicle’s roof. However, it is unclear how this viewpoint compares to a conventional viewpoint in terms of safety, efficiency, and mental workload. In the current study, teleoperators (n = 148) performed driving tasks in a simulated urban environment with a conventional viewpoint (i.e., the simulated camera was positioned inside the vehicle at the height of a driver’s eyes) and a higher viewpoint (the simulated camera was positioned on the vehicle roof). The tasks required negotiating road geometry and other road users. At the end of the session, participants completed the NASA-TLX questionnaire. Results showed that participants completed most tasks faster with the higher viewpoint and reported lower frustration and mental demand. The camera position did not affect collision rates nor the probability of hard braking and steering events. We conclude that a viewpoint from the vehicle roof may improve teleoperation efficiency without compromising driving safety, while also lowering the teleoperators’ mental workload. Full article

Wearable technologies represent a significant advancement in facilitating communication between humans and machines. Powered by artificial intelligence (AI), human gestures detected by wearable sensors can provide people with seamless interaction with physical, digital, and mixed environments. In this paper, the foundations of a gesture-recognition framework for the teleoperation of infrared consumer electronics are established. This framework is based on force myography data of the upper forearm, acquired from a prototype novel soft pressure-based force myography (pFMG) armband. Here, the sub-processes of the framework are detailed, including the acquisition of infrared and force myography data; pre-processing; feature construction/selection; classifier selection; post-processing; and interfacing/actuation. The gesture recognition system is evaluated using 12 subjects’ force myography data obtained whilst performing five classes of gestures. Our results demonstrate an inter-session and inter-trial gesture average recognition accuracy of approximately 92.2% and 88.9%, respectively. The gesture recognition framework was successfully able to teleoperate several infrared consumer electronics as a wearable, safe and affordable human–machine interface system. The contribution of this study centres around proposing and demonstrating a user-centred design methodology to allow direct human–machine interaction and interface for applications where humans and devices are in the same loop or coexist, as typified between users and infrared-communicating devices in this study. Full article

Manipulating a flexible ureteroscope is difficult, due to its bendable body and hand–eye coordination problems, especially when exploring the lower pole of the kidney. Though robotic interventions have been adopted in various clinical scenarios, they are rarely used in ureteroscopy. This study proposes a teleoperation system consists of a soft robotic endoscope together with a Guidance Virtual Fixture (GVF) to help users explore the kidney’s lower pole. The soft robotic arm was a cable-driven, 3D-printed design with a helicoid structure. GVF was dynamically constructed using video streams from an endoscopic camera. With a haptic controller, GVF can provide haptic feedback to guide the users in following a trajectory. In the user study, participants were asked to follow trajectories when the soft robotic arm was in a retroflex posture. The results suggest that the GVF can reduce errors in the trajectory tracking tasks when the users receive the proper training and gain more experience. Based on the NASA Task Load Index questionnaires, most participants preferred having the GVF when manipulating the robotic arm. In conclusion, the results demonstrate the benefits and potential of using a robotic arm with a GVF. More research is needed to investigate the effectiveness of the GVFs and the robotic endoscope in ureteroscopic procedures. Full article