An Integrated Multi-Sensor Network for Adaptive Grasping of Fragile Fruits: Design and Feasibility Tests
<p>The constituent part and the grasping process view of robotic ripper.</p> "> Figure 2
<p>Mechanical analysis diagram of manipulator. (<b>a</b>) The maximum grasping size. (<b>b</b>) Force and the coordinate of the belts.</p> "> Figure 3
<p>Manipulator grasping control chart. (<b>a</b>) Flow chart of calculation methodology of grasping force (the places marked with numbers correspond to the positions in (<b>b</b>)). (<b>b</b>) Schematic diagram of finger deflection process.</p> "> Figure 4
<p>Distribution of various sensors of manipulator. (<b>a</b>) Diagram of bending sensor application simulation. (<b>b</b>) Dimensional diagram—flex sensor. (<b>c</b>) Experimental setup to evaluate the grasping force. (<b>d</b>) The signal amplification circuit. (<b>e</b>) Variable deflection threshold switch.</p> "> Figure 5
<p>On-line decision-making grasping based on fusion sensory data. (<b>a</b>) Sensor-based control strategy. (<b>b</b>) Primitive shape extraction from point cloud. (<b>c</b>) Force control process.</p> "> Figure 6
<p>The boundary detection algorithm process.</p> "> Figure 7
<p>The calibration and fitting of the force curve of the pressure sensor.</p> "> Figure 8
<p>The calibration and fitting of the bending sensor.</p> "> Figure 9
<p>Linear fitting of experimental results.</p> "> Figure 10
<p>Grasping objects of different shapes in daily life (plastic bottle, milk box, tomato, green pepper, and so on).</p> "> Figure 11
<p>The relationship between force and bending angle under different conditions. (<b>a</b>) Grasp for tomato (bending angle and force). (<b>b</b>) Grasp for potato (bending angle and force). (<b>c</b>) Grasp for plastic bottle (bending angle and force). (<b>d</b>) Grasp for glasses case (bending angle and force). (<b>e</b>) Grasp for tea pot (bending angle and force). (<b>f</b>) Grasp for green pepper (bending angle and force).</p> "> Figure 12
<p>The measurement error of force and bending angle under different conditions.</p> "> Figure 13
<p>The relationship between measurement error and material. (<b>a</b>) The error distribution of bending angle. (<b>b</b>) The error distribution of force.</p> ">
Abstract
:1. Introduction
2. Materials and Methods
2.1. Mechanical Design and Sensor Layout
- (1)
- When a tomato is selected to be grabbed, the flexible gripper driven by the electronic motor approaches the target object; a change in the position is generated by the operation of the motor on the top, thus promoting mechanical gripper to open or close.
- (2)
- Through the process, the tip of fingers moves correctly and contacts the surface of the object easily, and spring 1 and 2 are stretched quickly.
- (3)
- The moving block moves downward continuously under the drive of the motor, causing the finger to close for grabbing. Since the belt itself has a certain amount of elasticity and flexibility, and the existence of the space between the belt and the iron part of the finger, ample room for the adaptive deformation of the belts is provided, helping it to adapt to multiple object contours. The belt’s deformation can be changed with the shape of the object, and the adaptability of this mechanical gripper is guaranteed. The process of grasping and placing the object is successfully completed.
2.2. Model of Grasping Force
2.2.1. Cosserat Theory
2.2.2. Grasping Force
2.3. Analysis and Simulation of Grasping Process
- (1)
- The finger model illustrated above is established and conditions are determined. The force of grasping and the finger’s mechanical deflection angle are increased by 0.01 N and 1 degree, respectively, every time, beginning from zero, and the position of the fingertip P (,) is calculated according to the finger model [26].
- (2)
- When the value is increasing, the position is also changed. However, if the value is less than the value L, the critical distance that we defined earlier, the mechanical deflection angle needs to be increased. When equivalent to L, the two fingers can catch the object just right.
- (3)
- In order to catch the object correctly, the accurate value of the grasping force is bound to be adjusted. Once the value exceeds the value L, this means that the grasping force is not enough. Otherwise, the value is much less than the value L by calculation. Considering the importance of the accuracy, the maximum permissible error is set as Eh [26]. Through the adjustment of the two important elements, the condition is updated continuously until the manipulator finally leaves the object, and the grasping process is finished.
2.4. Integrated Multi-Sensor Network
2.4.1. Kinect Sensor
2.4.2. Force Sensor
2.4.3. Bending Sensor
2.5. Control System and Sensor-Based Control Strategy
- (1)
- Controller initialization.
- (2)
- The manipulator grasps the unknown object with the reference value (2N). Force sensors sense the force. The controller uses Equation (14) to calculate the weight of the object.
- (3)
- When the weight calculated is below 0.1N, the actual weight can be judged as more than 4N, which means the grasping process is not successful (the object slides down between fingers), so step (2) should be repeated again according to another reference value of 4N [28]. If the above unsuccessful result repeats again, the reference value should be increased by 2N until the maximum tolerable pressure. If the weight ranges from 0.1N to 4N, the grip force remains 2N; if the weight is greater than 4N, the grip force is calculated by dividing the weight by 2 [28].
- (4)
- The object is grabbed again with the determined force. Move the gripper by operating the motor [28].
3. Results and Discussion
3.1. Primitive Shape Extraction from Point Cloud
- (1)
- The 3D point set is projected onto its local best fitting plane, which is called discrete point set s. Select any point P1 (, ) in S and find point P2 (, ) less than 2r away from point P1 to form a new point set S2. For each point in S2, we can calculate the center O (, ) according to Equations (15) and (16);
- (2)
- Find the distance DP from each point in S2 to O. If , there is no point in the circle, then points P1 and P2 are boundary points. Otherwise, if , go to step (3) and repeat;
- (3)
- For all points in S2, the next point is selected and step (1) is repeated until each point in S2 is completed.
- (4)
3.2. Grasping Force Acquisition and Calibration
3.3. Adaptive Grasping Related to Bending Angle
3.4. Multi-Sensor Feedback Control System
3.4.1. General Grasping Test
3.4.2. Force and Bending Relationship
3.4.3. Analysis of Measurement Error
3.4.4. Measurement Error and Material Relationship
4. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Wang, X.; Zhao, Y.; Xiao, Y.; Zhu, Y.; Wang, Y. Design of a Underactuated Robotic Hand with Three Articulated Fingers. In Proceedings of the 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China, 12–14 October 2018; pp. 2562–2566. [Google Scholar]
- Liu, C.-H.; Chiu, C.-H.; Chen, T.-L.; Pai, T.-Y.; Chen, Y.; Hsu, M.-C. A Soft Robotic Gripper Module with 3D Printed Compliant Fingers for Grasping Fruits. In Proceedings of the 2018 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), Auckland, New Zealand, 9–12 June 2018; pp. 736–741. [Google Scholar]
- Zhang, B.; Zhou, J.; Meng, Y.; Zhang, N.; Gu, B.; Yan, Z.; Idris, S.I. Comparative study of mechanical damage caused by a two-finger tomato gripper with different robotic grasping patterns for harvesting robots. Biosyst. Eng. 2018, 171, 245–257. [Google Scholar] [CrossRef]
- Shi, K.; Li, X. Stiffness improvement of swirl gripper based on gap height and force estimation. Precis. Eng. 2019, 62, 134–142. [Google Scholar] [CrossRef]
- Wang, Y.; Gupta, U.; Parulekar, N.; Zhu, J. A soft gripper of fast speed and low energy consumption. Sci. China Technol. Sci. 2019, 62, 31–38. [Google Scholar] [CrossRef]
- Xiong, Y.; Peng, C.; Grimstad, L.; From, P.J.; Isler, V. Development and field evaluation of a strawberry harvesting robot with a cable-driven gripper. Comput. Electron. Agric. 2019, 157, 392–402. [Google Scholar] [CrossRef]
- Rus, D.; Tolley, M.T. Design, fabrication and control of soft robots. Nature 2015, 521, 467–475. [Google Scholar] [CrossRef] [Green Version]
- Manti, M.; Hassan, T.; Passetti, G.; D’Elia, N.; Laschi, C.; Cianchetti, M. A Bioinspired Soft Robotic Gripper for Adaptable and Effective Grasping. Soft Robot. 2015, 2, 107–116. [Google Scholar] [CrossRef]
- Zhang, D.; Zhang, Z.; Gao, Q.; Xu, D.; Liu, S. Development of a monolithic compliant SPCA-driven micro-gripper. Mechatronics 2015, 25, 37–43. [Google Scholar] [CrossRef]
- Yang, H.; Chen, Y.; Sun, Y.; Hao, L. A novel pneumatic soft sensor for measuring contact force and curvature of a soft gripper. Sens. Actuators A Phys. 2017, 266, 318–327. [Google Scholar] [CrossRef]
- Yang, Y.; Cao, Q.-X. Monocular vision based 6D object localization for service robot’s intelligent grasping. Comput. Math. Appl. 2012, 64, 1235–1241. [Google Scholar] [CrossRef] [Green Version]
- Du, G.; Wang, K.; Lian, S. Vision-based Robotic Grasping from Object Localization, Pose Estimation, Grasp Detection to Motion Planning: A Review. arXiv 2019, arXiv:1905.06658. [Google Scholar]
- Liu, C.; Cheng, J.; Li, Z.; Cheng, C.; Zhang, C.; Zhang, Y.; Zhong, R.Y. Design of a self-adaptive gripper with rigid fingers for Industrial Internet. Robot. Comput. Manuf. 2020, 65, 101976. [Google Scholar] [CrossRef]
- Sun, T.; Chen, Y.; Han, T.; Jiao, C.; Lian, B.; Song, Y. A soft gripper with variable stiffness inspired by pangolin scales, toothed pneumatic actuator and autonomous controller. Robot. Comput. Integr. Manuf. 2020, 61, 101848. [Google Scholar] [CrossRef]
- Baeten, J.; Donné, K.; Boedrij, S.; Beckers, W.; Claesen, E. Autonomous Fruit Picking Machine: A Robotic Apple Harvester. Springer Proc. Adv. Robot. 2008, 42, 531–539. [Google Scholar]
- Zhou, J.; Chen, S.; Wang, Z. A Soft-Robotic Gripper with Enhanced Object Adaptation and Grasping Reliability. IEEE Robot. Autom. Lett. 2017, 2, 2287–2293. [Google Scholar] [CrossRef]
- Wang, Z.; Hirai, S. A 3D printed soft gripper integrated with curvature sensor for studying soft grasping. In Proceedings of the 2016 IEEE/SICE International Symposium on System Integration (SII), Sapporo, Japan, 13–15 December 2016; pp. 629–633. [Google Scholar]
- Zhong, G.; Hou, Y.; Dou, W. A soft pneumatic dexterous gripper with convertible grasping modes. Int. J. Mech. Sci. 2019, 153, 445–456. [Google Scholar] [CrossRef]
- Reddy, P.V.P.; Suresh, V.V.N.S. A review on importance of universal gripper in industrial robot applications. Int. J. Mech. Eng. Robot. Res 2013, 2, 255–264. [Google Scholar]
- Birglen, L.; Schlicht, T. A statistical review of industrial robotic grippers. Robot. Comput. Integr. Manuf. 2018, 49, 88–97. [Google Scholar] [CrossRef]
- Kim, D.H.; Lee, S.W.; Park, H.-S. Sensor evaluation for soft robotic hand rehabilitation devices. In Proceedings of the 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), Singapore, 26–29 June 2016. [Google Scholar]
- Asheber, W.; Lin, C.Y. Mechatronics Design of Intelligent Robotic Gripper. Key Eng. Mater. 2015, 649, 14–21. [Google Scholar] [CrossRef]
- Edan, Y.; Rogozin, D.; Flash, T.; Miles, G. Robotic melon harvesting. IEEE Trans. Robot. Autom. 2000, 16, 831–835. [Google Scholar] [CrossRef]
- Chen, Y.; Guo, S.; Li, C.; Yang, H.; Hao, L. Size recognition and adaptive grasping using an integration of actuating and sensing soft pneumatic gripper. Robot. Auton. Syst. 2018, 104, 14–24. [Google Scholar] [CrossRef]
- Blanes, C.; Mellado, M.; Ortiz, C.; Valera, A. Technologies for robot grippers in pick and place operations for fresh fruits and vegetables. Span. J. Agric. Res. 2011, 9, 1130. [Google Scholar] [CrossRef]
- Haibin, Y.; Cheng, K.; Junfeng, L.; Guilin, Y. Modeling of grasping force for a soft robotic gripper with variable stiffness. Mech. Mach. Theory 2018, 128, 254–274. [Google Scholar] [CrossRef]
- Rajamani, A.S.; Rammohan, A.; Mohan, S.; Srinivasan, P.; Arthanari, S.; Muthusamy, U.; Sivasubramanian, V.; Palaniappan, R. Prospective Evaluation of Innovative Force Assessing Firmware in Simulation to Improve the Technical Competence of Surgical Trainees. World J. Surg. 2015, 40, 773–778. [Google Scholar] [CrossRef] [PubMed]
- Kim, G.-S. Development of a three-axis gripper force sensor and the intelligent gripper using it. Sens. Actuators A Phys. 2007, 137, 213–222. [Google Scholar] [CrossRef]
- Ning, X.; Li, F.; Wang, Y.; Hao, W. Primitive Shape Extraction for Objects in Point Cloud. In Proceedings of the 2017 International Conference on Virtual Reality and Visualization (ICVRV), Zhengzhou, China, 21–22 October 2017. [Google Scholar]
- Tai, K.; El-Sayed, A.-R.; Shahriari, M.; Biglarbegian, M.; Mahmud, S. State of the Art Robotic Grippers and Applications. Robotics 2016, 5, 11. [Google Scholar] [CrossRef] [Green Version]
- Erfani, S.; Jafari, A.; Hajiahmad, A. Comparison of two data fusion methods for localization of wheeled mobile robot in farm conditions. Artif. Intell. Agric. 2019, 1, 48–55. [Google Scholar] [CrossRef]
- Li, J.; Zhang, H.; Zhou, C.; Chen, Z.; Chen, X.; Long, Z.; Liu, X.; Zhu, W. A Multiparameter Numerical Modeling and Simulation of the Dipping Process in Microelectronics Packaging. IEEE Trans. Ind. Inform. 2019, 15, 3808–3820. [Google Scholar] [CrossRef]
- Li, J.; Zhang, X.; Zhou, C.; Zheng, J.; Ge, D.; Zhu, W. New Applications of an Automated System for High-Power LEDs. IEEE/ASME Trans. Mechatron. 2016, 21, 1035–1042. [Google Scholar] [CrossRef]
- Syed, T.N.; Liu, J.; Xin, Z.; Shengyi, Z.; Yan, Y.; Mohamed, S.H.A.; Lakhiar, I.A. Seedling-lump integrated non-destructive monitoring for automatic transplanting with Intel RealSense depth camera. Artif. Intell. Agric. 2019, 3, 18–32. [Google Scholar] [CrossRef]
Parameters | Symbol |
---|---|
Each point on the belt | p |
The position of each point | r(p) |
The infinite elements within a local frame | |
A belt’s center line curve | |
The direction of the local frame relative to the global one | |
The linear rates of changes of the belts position | |
Derivative of | |
A skew-symmetric matrix | |
The angle between the bending direction and Z axis | |
The distributed external force applied to any part of the belts | |
The distributed bending moment applied to any part of the belts | |
The point force or internal force in the global system | n |
The bending moment within the global system | m |
The linear density | |
Cross-sectional area |
Objects | Average | Max | Min | Standard Deviation |
---|---|---|---|---|
Milk box | 5.8 | 7 | 3 | 1.732051 |
Tomato | 4.5 | 7 | 3 | 1.658312 |
Potato | 12 | 13 | 10 | 1.067187 |
Plastic bottle | 6.1 | 7 | 5 | 0.942809 |
Glasses case | 5.4 | 8 | 3 | 1.607275 |
Tea pot | 12 | 13 | 11 | 1 |
Green pepper | 7.1 | 13 | 5 | 2.821203 |
Objects | Average | Max | Min | Standard Deviation |
---|---|---|---|---|
Milk box | 0.781 | 0.827 | 0.747 | 0.029084 |
Tomato | 2.349 | 2.429 | 2.295 | 0.048115 |
Potato | 5.422 | 5.471 | 5.365 | 0.041964 |
Plastic bottle | 6.895 | 7.126 | 6.726 | 0.164283 |
Glasses case | 2.237 | 2.429 | 2.028 | 0.175296 |
Tea pot | 11.156 | 11.210 | 11.129 | 0.026689 |
Green pepper | 4.390 | 4.804 | 4.003 | 0.379874 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xie, Y.; Zhang, B.; Zhou, J.; Bai, Y.; Zhang, M. An Integrated Multi-Sensor Network for Adaptive Grasping of Fragile Fruits: Design and Feasibility Tests. Sensors 2020, 20, 4973. https://doi.org/10.3390/s20174973
Xie Y, Zhang B, Zhou J, Bai Y, Zhang M. An Integrated Multi-Sensor Network for Adaptive Grasping of Fragile Fruits: Design and Feasibility Tests. Sensors. 2020; 20(17):4973. https://doi.org/10.3390/s20174973
Chicago/Turabian StyleXie, Yuanxin, Baohua Zhang, Jun Zhou, Yuhao Bai, and Meng Zhang. 2020. "An Integrated Multi-Sensor Network for Adaptive Grasping of Fragile Fruits: Design and Feasibility Tests" Sensors 20, no. 17: 4973. https://doi.org/10.3390/s20174973
APA StyleXie, Y., Zhang, B., Zhou, J., Bai, Y., & Zhang, M. (2020). An Integrated Multi-Sensor Network for Adaptive Grasping of Fragile Fruits: Design and Feasibility Tests. Sensors, 20(17), 4973. https://doi.org/10.3390/s20174973