Search Results (150)

Robotic harvesting has become an urgent need for the development of the apple industry, due to the sharp decline in agricultural labor. At present, harvesting apples using robots in unstructured orchard environments remains a significant challenge. This paper focuses on addressing the challenges of perception, localization, and dual-arm coordination in harvesting robots and presents a dual-arm apple harvesting robot system. First, the paper introduces the integration of the robot’s hardware and software systems, as well as the control system architecture, and describes the robot’s workflow. Secondly, combining a dual-vision perception system, the paper adopts a fruit recognition method based on a multi-task network model and a frustum-based fruit localization approach to identify and localize fruits. Finally, to improve collaboration efficiency, a multi-arm task planning method based on a genetic algorithm is used to optimize the target harvesting sequence for each arm. Field experiments were conducted in an orchard to evaluate the overall performance of the robot system. The field trials demonstrated that the robot system achieved an overall harvest success rate of 76.97%, with an average fruit picking time of 7.29 s per fruit and a fruit damage rate of only 5.56%. Full article

In recent decades, the development of surgical systems which minimize patient impact has been a major focus for surgeons and researchers, leading to the advent of robotic systems for minimally invasive surgery. These technologies offer significant patient benefits, including enhanced outcome quality and accuracy, reduced invasiveness, lower blood loss, decreased postoperative pain, diminished infection risk, and shorter hospitalization and recovery times. Surgeons benefit from the elimination of human tremor, ergonomic advantages, improved vision systems, better access to challenging anatomical areas, and magnified 3DHD visualization of the operating field. Since 2000, Intuitive Surgical has developed multiple generations of master-slave multi-arm robots, securing over 7000 patents, which created significant barriers for competitors. This monopoly resulted in the widespread adoption of their technology, now used in over 11 million surgeries globally. With the expiration of key patents, new robotic platforms featuring innovative designs, such as modular systems, are emerging. This review examines advancements in robotic surgery within the fields of general, urological, and gynecological surgery. The objective is to analyze the current robotic surgical platforms, their technological progress, and their impact on surgical practices. By examining these platforms, this review provides insights into their development, potential benefits, and future directions in robotic-assisted surgery. Full article

In industrial robotic arm gripping operations within disordered environments, the loss of physical information on the object’s surface is often caused by changes such as varying lighting conditions, weak surface textures, and sensor noise. This leads to inaccurate object detection and pose estimation information. A method for industrial object pose estimation using point cloud data is proposed to improve pose estimation accuracy. During the feature extraction process, both global and local information are captured by integrating the appearance features of RGB images with the geometric features of point clouds. Integrating semantic information with instance features effectively distinguishes instances of similar objects. The fusion of depth information and RGB color channels enriches spatial context and structure. A cross-entropy loss function is employed for multi-class target classification, and a discriminative loss function enables instance segmentation. A novel point cloud registration method is also introduced to address re-projection errors when mapping 3D keypoints to 2D planes. This method utilizes 3D geometric information, extracting edge features using point cloud curvature and normal vectors, and registers them with models to obtain accurate pose information. Experimental results demonstrate that the proposed method is effective and superior on the LineMod and YCB-Video datasets. Finally, objects are grasped by deploying a robotic arm on the grasping platform. Full article

To address the technical challenges, production quality issues, and inefficiencies caused by the heavy reliance on traditional manual processing of small assembly plates in the shipbuilding industry, this paper presents the design and analysis of a track-based automatic welding device. This equipment provides a solution for achieving batch and continuous welding in the field of automatic welding technology. The design section includes the mechanical design of the equipment’s core mechanisms, the design of the operating systems, the development of visual scanning strategies under working conditions, and the formulation of multi-layer and multi-pass welding processes. The analysis section comprises the static analysis of the equipment’s mechanical structure, kinematic analysis of the robotic arm, and inspection analysis of the device. Compared with manual welding, multi-layer and multi-pass welding experiments conducted using the equipment demonstrated stabilized welding quality for small assembly plates. Under the conditions of single plates with different groove positions and gaps, when the gap was 4 mm, processing efficiency increased by 7.35%, and processing time was reduced by 10.2%; when the gap was 5 mm, processing efficiency increased by 10.7%, and processing time decreased by 7.39%. The welding formation rate for the overall processing of single plate panels and web grooves increased by 11.48%, total material consumption decreased by 13.4%, and unit material consumption decreased by 13.5%. For mass production of small assembly plates of the same specifications, processing time was reduced by 16.7%, and there was a 41.4% reduction in costs. The equipment effectively addresses the low level of automation and heavy dependence on traditional manual processing in the shipbuilding industry, contributing to cost reduction and efficiency improvement. Full article

This study proposes a fault diagnosis method for harmonic drive systems based on multi-sensor data fusion and the gravitational search algorithm (GSA). As a critical component in robotic arms, harmonic drives are prone to failures due to wear, less grease, or improper loading, which can compromise system stability and production efficiency. To enhance diagnostic accuracy, the research employs wavelet packet decomposition (WPD) and empirical mode decomposition (EMD) to extract multi-scale features from vibration signals. These features are subsequently fused, and GSA is used to optimize the high-dimensional fused features, eliminating redundant data and mitigating overfitting. The optimized features are then input into a support vector machine (SVM) for fault classification, with K-fold cross-validation used to assess the model’s generalization capabilities. Experimental results demonstrate that the proposed diagnosis method, which integrates multi-sensor data fusion with GSA optimization, significantly improves fault diagnosis accuracy compared to methods using single-sensor signals or unoptimized features. This improvement is particularly notable in multi-class fault scenarios. Additionally, GSA’s global search capability effectively addresses overfitting issues caused by high-dimensional data, resulting in a diagnostic model with greater reliability and accuracy across various fault conditions. Full article

Planar underactuated mechanical systems have been a popular research issue in the area of mechanical systems and nonlinear control. This paper reviews the current research status of control methods for a class of planar underactuated manipulator (PUM) systems containing a single passive joint. Firstly, the general dynamics model and kinematics model of the PUM are given, and its control characteristics are introduced; secondly, according to the distribution position characteristics of the passive joints, the PUM is classified into the passive first joint system, the passive last joint system, and the passive intermediate joint system, and the analysis and discussion are carried out in respect to the existing intelligent control methods. Finally, in response to the above discussion, we provide a brief theoretical analysis and summarize the challenges faced by PUM, i.e., uncertainty and robustness of the system, unified control methods and research on underactuated systems with uncontrollable multi-passive joints; at the same time, the practical applications have certain limitations that need to be implemented subsequently, i.e., anti-jamming, multi-planar underactuated robotic arm co-control and spatial underactuated robotic arm system development. Aiming at the above challenges and problems in the control of PUM systems, we elaborate on them in points, and put forward the research directions and related ideas for future work, taking into account the contributions of the current work. Full article

This paper outlines a conceptual design for a multi-level natural language-based planning system and describes a demonstrator. The main goal of the demonstrator is to serve as a proof-of-concept by accomplishing end-to-end execution in a real-world environment, and showing a novel way of interfacing an LLM-based planner with open-set semantic maps. The target use-case is executing sequences of tabletop pick-and-place operations using an industrial robot arm and RGB-D camera. The demonstrator processes unstructured user prompts, produces high-level action plans, queries a map for object positions and grasp poses using open-set semantics, then uses the resulting outputs to parametrize and execute a sequence of action primitives. In this paper, the overall system structure, high-level planning using language models, low-level planning through action and motion primitives, as well as the implementation of two different environment modeling schemes—2.5 or fully 3-dimensional—are described in detail. The impacts of quantizing image embeddings on object recall are assessed and high-level planner performance is evaluated using a small reference scene data set. We observe that, for the simple constrained test command data set, the high-level planner is able to achieve a total success rate of 96.40%, while the semantic maps exhibit maximum recall rates of 94.69% and 92.29% for the 2.5d and 3d versions, respectively. Full article

With the current demand for automation in the agro-food industry, accurately detecting and localizing relevant objects in 3D is essential for successful robotic operations. However, this is a challenge due the presence of occlusions. Multi-view perception approaches allow robots to overcome occlusions, but a tracking component is needed to associate the objects detected by the robot over multiple viewpoints. Multi-object tracking (MOT) algorithms can be categorized between two-stage and single-stage methods. Two-stage methods tend to be simpler to adapt and implement to custom applications, while single-stage methods present a more complex end-to-end tracking method that can yield better results in occluded situations at the cost of more training data. The potential advantages of single-stage methods over two-stage methods depend on the complexity of the sequence of viewpoints that a robot needs to process. In this work, we compare a 3D two-stage MOT algorithm, 3D-SORT, against a 3D single-stage MOT algorithm, MOT-DETR, in three different types of sequences with varying levels of complexity. The sequences represent simpler and more complex motions that a robot arm can perform in a tomato greenhouse. Our experiments in a tomato greenhouse show that the single-stage algorithm consistently yields better tracking accuracy, especially in the more challenging sequences where objects are fully occluded or non-visible during several viewpoints. Full article

The current harvesting arms used in harvesting robots are developed based on standard products. Due to design constraints, they are unable to effectively avoid obstacles while harvesting tomatoes in tight spaces. To enhance the robot’s capability in obstacle-avoidance picking of tomato bunches with various postures, this study proposes a geometric parameter optimization method for a 7 degree of freedom (DOF) robotic arm. This method ensures that the robot can reach a predetermined workspace with a more compact arm configuration. The optimal picking posture for the end-effector is determined by analyzing the spatial distribution of tomato bunches, the main stem position, and peduncle posture, enabling a quantitative description of the obstacle-avoidance workspace. The denavit–hartenberg (D-H) model of the harvesting arm and the expected collision-free workspace are set as constraints. The compactness of the arm and the accessibility of the harvesting space serve as the optimization objectives. The Non-dominated Sorting Genetic Algorithm II (NSGA-II) multi-objective genetic algorithm is employed to optimize the arm length, and the results were validated through a virtual experiment using workspace traversal. The results indicate that the optimized structure of the tomato harvesting arm is compact, with a reachability of 92.88% in the workspace, based on the collision-free harvesting criteria. This study offers a reference for structural parameter optimization of robotic arms specialized in fruit and vegetable harvesting. Full article

This study presents the design and implementation of a wire-driven, multi-joint robotic arm equipped with a cutting and gripping mechanism for harvesting delicate strawberries, with the goal of reducing labor and costs. The arm is mounted on a lifting mechanism and linked to a laterally movable module, which is affixed to the tube cultivation shelf. The trained deep learning model can instantly detect strawberries, identify optimal picking points, and estimate the contour area of fruit while the mobile platform is in motion. A two-stage fuzzy logic control (2s-FLC) method is employed to adjust the length of the arm and bending angle, enabling the end of the arm to approach the fruit picking position. The experimental results indicate a 90% accuracy in fruit detection, an 82% success rate in harvesting, and an average picking time of 6.5 s per strawberry, reduced to 5 s without arm recovery time. The performance of the proposed system in harvesting strawberries of different sizes under varying lighting conditions is also statistically analyzed and evaluated in this paper. Full article

To address the issue of accurately recognizing and locating picking points for tea-picking robots in unstructured environments, a visual positioning method based on RGB-D information fusion is proposed. First, an improved T-YOLOv8n model is proposed, which improves detection and segmentation performance across multi-scale scenes through network architecture and loss function optimizations. In the far-view test set, the detection accuracy of tea buds reached 80.8%; for the near-view test set, the mAP_0.5 values for tea stem detection in bounding boxes and masks reached 93.6% and 93.7%, respectively, showing improvements of 9.1% and 14.1% over the baseline model. Secondly, a layered visual servoing strategy for near and far views was designed, integrating the RealSense depth sensor with robotic arm cooperation. This strategy identifies the region of interest (ROI) of the tea bud in the far view and fuses the stem mask information with depth data to calculate the three-dimensional coordinates of the picking point. The experiments show that this method achieved a picking point localization success rate of 86.4%, with a mean depth measurement error of 1.43 mm. The proposed method improves the accuracy of picking point recognition and reduces depth information fluctuations, providing technical support for the intelligent and rapid picking of premium tea. Full article

To increase productivity, more sophisticated cluster tools are developed. To achieve this, one of the ways is to increase the number of spaces in a process module (PM) and the number of fingers on a robot arm as well, leading to a cluster tool with multi-space PMs and a multi-finger-arm robot. This paper discusses the scheduling problem of cluster tools with four-space PMs and a four-finger-arm robot, a typical tool with multi-space PMs and a multi-finger-arm robot adopted in modern fabs. With two arms in such a tool, one is used as a clean one, while the other is used as a dirty one. In this way, wafer quality can be improved. However, scheduling such cluster tools to ensure the residency time constraints is very challenging, and there is no research report on this issue. This article conducts an in-depth analysis of the steady-state scheduling for this type of cluster tools to explore the effect of different scheduling strategies. Based on the properties, four robot task sequences are presented as scheduling strategies. With them, four linear programming models are developed to optimize the cycle time of the system and find feasible schedules. The performance of these strategies is dependent on the activity parameters. Experiments are carried out to test the effect of different parameters on the performance of different strategies. It shows that, given a group of parameters, one can apply all the strategies and choose the best result obtained by one of the strategies. Full article

As the adoption of large-scale model-based AI grows, the field of robotics is undergoing significant changes. The emergence of cloud robotics, where advanced tasks are offloaded to fog or cloud servers, is gaining attention. However, the widely used Robot Operating System (ROS) does not support communication between robot software across different networks. This paper introduces ROS Gateway, a middleware designed to improve the usability and extend the communication range of ROS in multi-network environments, which is important for processing sensor data in cloud robotics. We detail its structure, protocols, and algorithms, highlighting improvements over traditional ROS configurations. The ROS Gateway efficiently handles high-volume data from advanced sensors such as depth cameras and LiDAR, ensuring reliable transmission. Based on the rosbridge protocol and implemented in Python 3, ROS Gateway is compatible with rosbridge-based tools and runs on both x86 and ARM-based Linux environments. Our experiments show that the ROS Gateway significantly improves performance metrics such as topic rate and delay compared to standard ROS setups. We also provide predictive formulas for topic receive rates to guide the design and deployment of robotic applications using ROS Gateway, supporting performance estimation and system optimization. These enhancements are essential for developing responsive and intelligent robotic systems in dynamic environments. Full article

As the global fruit growing area continues to increase and the population aging problem continues to intensify, fruit and vegetable production is constrained by the difficulties of labor shortages and high costs. Single-arm harvesting robots are inefficient, and in order to balance harvesting accuracy and efficiency, research on multi-arm harvesting robots has become a hot topic. This paper summarizes the performance of multi-arm harvesting robots in indoor and outdoor environments from the aspects of automatic navigation technology, fruit and vegetable identification and localization, multi-arm workspace optimization, and multi-arm harvesting task planning and analyzes their advantages and challenges in practical applications. The results show that the lack of application of automatic field navigation for multi-arm harvesting robots, the low harvesting rate in non-structured environments, and the complexity of algorithms for multi-arm harvesting robots’ task planning are the main challenges hindering their wide-scale application. Future studies need to focus on building a standardized growing environment to control the amount of information acquired by the robots and optimize the multi-arm control strategy of these challenges, which is an important direction for research on multi-arm harvesting robots. Full article

This paper investigates the problem of optimal base placement in collaborative robotic car painting. The objective of this problem is to find the optimal fixed base positions of a collection of given articulated robotic arms on the factory floor/ceiling such that the possibility of vehicle paint coverage is maximized while the possibility of robot collision avoidance is minimized. Leveraging the inherent two-dimensional geometric features of robotic car painting, we construct two types of cost functions that formally capture the notions of paint coverage maximization and collision avoidance minimization. Using these cost functions, we formulate a multi-objective optimization problem, which can be readily solved using any standard multi-objective optimizer. Our resulting optimal base placement algorithm decouples base placement from motion/trajectory planning. In particular, our computationally efficient algorithm does not require any information from motion/trajectory planners a priori or during base placement computations. Rather, it offers a hierarchical solution in the sense that its generated results can be utilized within already available robotic painting motion/trajectory planners. Our proposed solution’s effectiveness is demonstrated through simulation results of multiple industrial robotic arms collaboratively painting a Ford F-150 truck. Full article