Search Results (110)

Task-oriented grasp planning poses complex challenges in modern robotics, requiring the precise determination of the grasping pose of a robotic arm to grasp objects with a high level of manipulability while avoiding hardware constraints, such as joint limits, joint over-speeds, and singularities. This paper introduces a novel manipulability-aware (M-aware) grasp planning and motion control system for seven-degree-of-freedom (7-DoF) redundant dual-arm robots to achieve task-oriented grasping with optimal manipulability. The proposed system consists of two subsystems: (1) M-aware grasp planning; and (2) M-aware motion control. The former predicts task-oriented grasp candidates from an RGB-D image and selects the best grasping pose among the candidates. The latter enables the robot to select an appropriate arm to perform the grasping task while maintaining a high level of manipulability. To achieve this goal, we propose a new manipulability evaluation function to evaluate the manipulability score (M-score) of a given robot arm configuration with respect to a desired grasping pose to ensure safe grasping actions and avoid its joint limits and singularities. Experimental results demonstrate that our system can autonomously detect the graspable areas of a target object, select an appropriate grasping pose, grasp the target with a high level of manipulability, and achieve an average success rate of about 98.6%. Full article

Humanoid robots are becoming a global research focus. Due to the limitations of bipedal walking technology, mobile humanoid robots equipped with a wheeled chassis and dual arms have emerged as the most suitable configuration for performing complex tasks in factory or home environments. To address the high redundancy issue arising from the wheeled chassis and dual-arm design of mobile humanoid robots, this study proposes a whole-body coordinated motion control algorithm based on arm potential energy optimization. By constructing a gravity potential energy model for the arms and a virtual torsional spring elastic potential energy model with the shoulder-wrist line as the rotation axis, we establish an optimization index function for the arms. A neural network with variable stiffness is introduced to fit the virtual torsional spring, representing the stiffness variation trend of the human arm. Additionally, a posture mapping method is employed to map the human arm potential energy model to the robot, enabling realistic humanoid movements. Combining task-space and joint-space planning algorithms, we designed experiments for single-arm manipulation, independent object retrieval, and dual-arm carrying in a simulation of a 23-degree-of-freedom mobile humanoid robot. The results validate the effectiveness of this approach, demonstrating smooth motion, the ability to maintain a low potential energy state, and conformity to the operational characteristics of the human arm. Full article

Robotic harvesting has become an urgent need for the development of the apple industry, due to the sharp decline in agricultural labor. At present, harvesting apples using robots in unstructured orchard environments remains a significant challenge. This paper focuses on addressing the challenges of perception, localization, and dual-arm coordination in harvesting robots and presents a dual-arm apple harvesting robot system. First, the paper introduces the integration of the robot’s hardware and software systems, as well as the control system architecture, and describes the robot’s workflow. Secondly, combining a dual-vision perception system, the paper adopts a fruit recognition method based on a multi-task network model and a frustum-based fruit localization approach to identify and localize fruits. Finally, to improve collaboration efficiency, a multi-arm task planning method based on a genetic algorithm is used to optimize the target harvesting sequence for each arm. Field experiments were conducted in an orchard to evaluate the overall performance of the robot system. The field trials demonstrated that the robot system achieved an overall harvest success rate of 76.97%, with an average fruit picking time of 7.29 s per fruit and a fruit damage rate of only 5.56%. Full article

In order to improve the target visual recognition and localization accuracy of robotic arms in complex scenes with similar targets, hybrid recognition and localization methods based on an industrial camera and depth camera are proposed. First, according to the speed and accuracy requirements of target recognition and localization, YOLOv5s is introduced as the basic algorithm model for target hybrid recognition and localization. Then, in order to improve the accuracy of target recognition and coarse localization based on an industrial camera (eye-to-hand), the AFPN feature fusion module, simple and parameter-free attention module (SimAM), and soft non-maximum suppression (Soft NMS) are introduced. In order to improve the accuracy of target recognition and fine localization based on a depth camera (eye-in-hand), the SENetV2 backbone network structure, dynamic head module, deformable attention mechanism, and chain-of-thought prompted adaptive enhancer network are introduced. After that, on the basis of constructing a dual camera platform for target hybrid recognition and localization, the hand–eye calibration, collection and production of image datasets required for model training are completed. Finally, for the docking of the oil filling port, the hybrid recognition and localization experimental tests are completed in sequence. The test results show that in target recognition and coarse localization based on the industrial camera, the recognition accuracy of the designed model reaches 99%, and the average localization errors in the horizontal and vertical directions are 2.22 mm and 3.66 mm, respectively. In target recognition and fine localization based on the depth camera, the recognition accuracy of the designed model reaches 98%, and the average errors in depth, horizontal, and vertical directions are 0.12 mm, 0.28 mm, and 0.16 mm, respectively. These not only verify the effectiveness of the target hybrid recognition and localization methods based on dual cameras, but also demonstrate that they meet the high-precision recognition and localization requirements in complex scenes. Full article

The impedance control of a dual-arm space robot in orbit auxiliary docking operation is studied. First, for the closed-chain hybrid system formed by the dual-arm space robot after capture operation, the dynamic equation of position uncontrolled and attitude controlled is established. The second-order linear impedance model and second-order approximate environment model are established for the problem of simultaneous output force/pose control of the end of the manipulator. Then, aiming at the transient performance control requirements of the dual-arm space robot auxiliary docking operation in orbit, a sliding mode controller with equivalent replacement of tracking errors is designed by introducing Prescribed Performance Control (PPC) theory. Next, Radial Basis Function Neural Networks (RBFNN) are used to accurately compensate for the modeling uncertainties of the system. Finally, the stability of the system is verified by Lyapunov stability determination. The simulation results show that the attitude control accuracy is better than $0.5°$, the position control accuracy is better than ${10}^{-3}$ m, and the output force control accuracy is better than 0.5 N when it reaches 30 N. It also indicated that the proposed control algorithm can limit the transient performance of the controlled system within the preset range and achieve high-precision force/pose control, which ensures a more stable on-orbit auxiliary docking operation of the dual-arm space robot. Full article

Landscape maintenance is essential for ensuring agricultural productivity, promoting sustainable land use, and preserving soil and ecosystem health. Pruning is a labor-intensive task among landscaping applications that often involves repetitive pruning operations. To address these limitations, this paper presents the development of a dual-arm holonomic robot (called the KOALA robot) for precision plant pruning. The robot utilizes a cross-functionality sensor fusion approach, combining light detection and ranging (LiDAR) sensor and depth camera data for plant recognition and isolating the data points that require pruning. The You Only Look Once v8 (YOLOv8) object detection model powers the plant detection algorithm, achieving a 98.5% pruning plant detection rate and a 95% pruning accuracy using camera, depth sensor, and LiDAR data. The fused data allows the robot to identify the target boxwood plants, assess the density of the pruning area, and optimize the pruning path. The robot operates at a pruning speed of 10–50 cm/s and has a maximum robot travel speed of 0.5 m/s, with the ability to perform up to 4 h of pruning. The robot’s base can lift 400 kg, ensuring stability and versatility for multiple applications. The findings demonstrate the robot’s potential to significantly enhance efficiency, reduce labor requirements, and improve landscape maintenance precision compared to those of traditional manual methods. This paves the way for further advancements in automating repetitive tasks within landscaping applications. Full article

As gestures play an important role in human communication, there have been a number of service robots equipped with a pair of human-like arms for gesture-based human–robot interactions. However, the arms of most human companion robots are limited to slow and simple gestures due to the low maximum velocity of the arm actuators. In this work, we present the JF-2 robot, a mobile home service robot equipped with a pair of torque-controlled anthropomorphic arms. Thanks to the low inertia design of the arm, responsive Quasi-Direct Drive (QDD) actuators, and active compliant control of the joints, the robot can replicate fast human dance motions while being safe in the environment. In addition to the JF-2 robot, we also present the JF-mini robot, a scaled-down, low-cost version of the JF-2 robot mainly targeted for commercial use at kindergarten and childcare facilities. The suggested system is validated by performing three experiments, a safety test, teaching children how to dance along to the music, and bringing a requested item to a human subject. Full article

Future humanoid robots will be widely deployed in our daily lives. Motion planning and control in an unstructured, confined, and human-centered environment utilizing dexterity and a cooperative ability of dual-arm robots is still an open issue. We propose a globally guided dual-arm reactive motion controller (GGDRC) that combines the strengths of global planning and reactive methods. In this framework, a global planner module with a prospective task horizon provides feasible guidance in a Cartesian space, and a local reactive controller module addresses real-time collision avoidance and coordinated task constraints through the exploitation of dual-arm redundancy. GGDRC extends the start-of-the-art optimization-based reactive method for motion-restricted dynamic scenarios requiring dual-arm cooperation. We design a pick–handover–place task to compare the performances of these two methods. Results demonstrate that GGDRC exhibits accurate collision avoidance precision (5 mm) and a high success rate ($84.5\%$). Full article

We focus on dual-user operation, where two users control a single remote robot equipped with a force sensor using haptic interface devices. We employ a cooperative work in which the two users control the remote robot to collaborate with remote robot systems with force feedback to carry an object. By measuring the force acting upon the object, we aim to better understand the underlying mechanisms by which the user with lower network latency can help the other user, as observed in our previous work. We notice that with increasing network delays, the force exerted on the object tends to intensify, indicating that it becomes more challenging for users to operate the remote robot effectively as network delays increase. We also measure the force applied to the object by changing the network delays between the remote robot and the two users to clarify why the user with the lower network delay can assist the other user. We find that when the total network delay is the same, the average force magnitude and the average maximum force magnitude remain nearly identical. This is because, despite the challenges faced by the user with the larger network delay, the user with the smaller delay can operate the remote robot more easily and assist the other user. In order to reduce the force acting upon the object, we propose an enhancement method for the robot position control, which determines the position of the remote robot arm while accounting for network delay, and investigate the effects by experiment. Experimental results demonstrate that our proposed method is effective and can reduce the applied force. This is because the proposed method adjusts the ratio between the user with the lower delay and the user with the higher delay. The user with the lower delay can operate the remote robot more easily and respond to it more quickly. Our findings and proposed method can be useful in improving work accuracy and operability when designing a remote robot system with force feedback for applications. Full article

Robotic systems are crucial in modern manufacturing. Complex assembly tasks require the collaboration of multiple robots. Their orchestration is challenging due to tight tolerances and precision requirements. In this work, we set up two Franka Panda robots performing a peg-in-hole insertion task of 1 mm clearance. We structure the control system hierarchically, planning the robots’ feedback-based trajectories with a central policy trained with reinforcement learning. These trajectories are executed by a low-level impedance controller on each robot. To enhance training convergence, we use reverse curriculum learning, novel for such a two-armed control task, iteratively structured with a minimum requirements and fine-tuning phase. We incorporate domain randomization, varying initial joint configurations of the task for generalization of the applicability. After training, we test the system in a simulation, discovering the impact of curriculum parameters on the emerging process time and its variance. Finally, we transfer the trained model to the real-world, resulting in a small decrease in task duration. Comparing our approach to classical path planning and control shows a decrease in process time, but higher robustness towards calibration errors. Full article

This paper introduces a novel approach to robotic assistance in bottle opening using the dual-arm robot TIAGo++. The solution enhances accessibility by addressing the needs of individuals with injuries or disabilities who may require help with common manipulation tasks. The aim of this paper is to propose a method involving vision, manipulation, and learning techniques to effectively address the task of bottle opening. The process begins with the acquisition of bottle and cap positions using an RGB-D camera and computer vision. Subsequently, the robot picks the bottle with one gripper and grips the cap with the other, each by planning safe trajectories. Then, the opening procedure is executed via a position and force control scheme that ensures both grippers follow the unscrewing path defined by the cap thread. Within the control loop, force sensor information is employed to control the vertical axis movements, while gripper rotation control is achieved through a Deep Reinforcement Learning (DRL) algorithm trained to determine the optimal angle increments for rotation. The results demonstrate the successful training of the learning agent. The experiments confirm the effectiveness of the proposed method in bottle opening with the TIAGo++ robot, showcasing the practical viability of the approach. Full article

In order to improve quality assurance in wafer manufacturing, there are strict process requirements. Besides the well-known residency time constraints (RTCs), a dual-arm cluster tool also requires each robot arm to execute a specific set of tasks. We call such a tool an arm task-constrained dual-arm cluster tool (ATC-DACT). To do this, one of the arms is identified as the dirty one and the other as the clean one. The dirty one can deal with raw wafers, while the clean one can deal with processed wafers. This requirement raises a new problem for scheduling a cluster tool. This paper discusses the scheduling problem of ATC-DACTs with RTCs. Due to the arm task constraints, the proven, effective swap strategy is no longer applicable to ATC-DACTs, making the scheduling problem difficult. To address this problem, we explicitly describe the robot waiting as an event and build a Petri net (PN) model. Then, we propose a hybrid task sequence (HTS) as an operation strategy by combining the swap and backward strategies. Based on the HTS, the necessary and sufficient conditions for schedulability are established; also, a linear programming model is developed. We then develop an algorithm using these results to optimally schedule the system. Industrial case studies demonstrate the application of this method. Full article

The increasing utilization of upper limb rehabilitation robots in rehabilitation therapy has brought to light significant safety concerns regarding their mechanical structures and control systems. This study focuses on a six degrees of freedom (DOF) upper limb rehabilitation robot, which has been designed with an emphasis on safety through careful consideration of its mechanical structure and trajectory planning. Various parameter schemes for the shoulder joint angles were proposed, and the robotic arm’s structure was developed by analyzing the spatial motion trajectories of the shoulder joint motor. This design successfully achieves the objective of minimizing the installation space while maximizing the range of motion. Additionally, an enhanced artificial field method is introduced to facilitate the planning of self-collision avoidance trajectories for dual-arm movements. This approach effectively mitigates the risk of collisions between the robotic arm and the human body, as well as between the two robotic arms, during movement. The efficacy of this method has been validated through experimental testing. Full article

In the Industry 4.0 scenario, human–robot collaboration (HRC) plays a key role in factories to reduce costs, increase production, and help aged and/or sick workers maintain their job. The approaches of the ISO 11228 series commonly used for biomechanical risk assessments cannot be applied in Industry 4.0, as they do not involve interactions between workers and HRC technologies. The use of wearable sensor networks and software for biomechanical risk assessments could help us develop a more reliable idea about the effectiveness of collaborative robots (coBots) in reducing the biomechanical load for workers. The aim of the present study was to investigate some biomechanical parameters with the 3D Static Strength Prediction Program (3DSSPP) software v.7.1.3, on workers executing a practical manual material-handling task, by comparing a dual-arm coBot-assisted scenario with a no-coBot scenario. In this study, we calculated the mean and the standard deviation (SD) values from eleven participants for some 3DSSPP parameters. We considered the following parameters: the percentage of maximum voluntary contraction (%MVC), the maximum allowed static exertion time (MaxST), the low-back spine compression forces at the L4/L5 level (L4Ort), and the strength percent capable value (SPC). The advantages of introducing the coBot, according to our statistics, concerned trunk flexion (SPC from 85.8% without coBot to 95.2%; %MVC from 63.5% without coBot to 43.4%; MaxST from 33.9 s without coBot to 86.2 s), left shoulder abdo-adduction (%MVC from 46.1% without coBot to 32.6%; MaxST from 32.7 s without coBot to 65 s), and right shoulder abdo-adduction (%MVC from 43.9% without coBot to 30.0%; MaxST from 37.2 s without coBot to 70.7 s) in Phase 1, and right shoulder humeral rotation (%MVC from 68.4% without coBot to 7.4%; MaxST from 873.0 s without coBot to 125.2 s), right shoulder abdo-adduction (%MVC from 31.0% without coBot to 18.3%; MaxST from 60.3 s without coBot to 183.6 s), and right wrist flexion/extension rotation (%MVC from 50.2% without coBot to 3.0%; MaxST from 58.8 s without coBot to 1200.0 s) in Phase 2. Moreover, Phase 3, which consisted of another manual handling task, would be removed by using a coBot. In summary, using a coBot in this industrial scenario would reduce the biomechanical risk for workers, particularly for the trunk, both shoulders, and the right wrist. Finally, the 3DSSPP software could be an easy, fast, and costless tool for biomechanical risk assessments in an Industry 4.0 scenario where ISO 11228 series cannot be applied; it could be used by occupational medicine physicians and health and safety technicians, and could also help employers to justify a long-term investment. Full article

Arm and hand function play a critical role in the successful completion of everyday tasks. Lost function due to neurological impairment impacts millions of lives worldwide. Despite improvements in the ability to assess and rehabilitate arm deficits, knowledge about underlying sources of impairment and related sequela remains limited. The comprehensive assessment of function requires the measurement of both biomechanics and neuromuscular contributors to performance during the completion of tasks that often use multiple joints and span three-dimensional workspaces. To our knowledge, the complexity of movement and diversity of measures required are beyond the capabilities of existing assessment systems. To bridge current gaps in assessment capability, a new exoskeleton instrument is developed with comprehensive bilateral assessment in mind. The development of the BiLateral Upper-limb Exoskeleton for Simultaneous Assessment of Biomechanical and Neuromuscular Output (BLUE SABINO) expands on prior iterations toward full-arm assessment during reach-and-grasp tasks through the development of a dual-arm and dual-hand system, with 9 active degrees of freedom per arm and 12 degrees of freedom (six active, six passive) per hand. Joints are powered by electric motors driven by a real-time control system with input from force and force/torque sensors located at all attachment points between the user and exoskeleton. Biosignals from electromyography and electroencephalography can be simultaneously measured to provide insight into neurological performance during unimanual or bimanual tasks involving arm reach and grasp. Design trade-offs achieve near-human performance in exoskeleton speed and strength, with positional measurement at the wrist having an error of less than 2 mm and supporting a range of motion approximately equivalent to the 50th-percentile human. The system adjustability in seat height, shoulder width, arm length, and orthosis width accommodate subjects from approximately the 5th-percentile female to the 95th-percentile male. Integration between precision actuation, human–robot-interaction force-torque sensing, and biosignal acquisition systems successfully provide the simultaneous measurement of human movement and neurological function. The bilateral design enables use with left- or right-side impairments as well as intra-subject performance comparisons. With the resulting instrument, the authors plan to investigate underlying neural and physiological correlates of arm function, impairment, learning, and recovery. Full article