CN110509281A - Device and method for pose recognition and grasping based on binocular vision - Google Patents
Device and method for pose recognition and grasping based on binocular vision Download PDFInfo
- Publication number
- CN110509281A CN110509281A CN201910870880.0A CN201910870880A CN110509281A CN 110509281 A CN110509281 A CN 110509281A CN 201910870880 A CN201910870880 A CN 201910870880A CN 110509281 A CN110509281 A CN 110509281A
- Authority
- CN
- China
- Prior art keywords
- target workpiece
- manipulator
- camera
- workpiece
- end effector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 230000008569 process Effects 0.000 claims abstract description 14
- 230000005540 biological transmission Effects 0.000 claims abstract description 7
- 238000012546 transfer Methods 0.000 claims abstract description 5
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 239000012636 effector Substances 0.000 claims description 61
- 241001292396 Cirrhitidae Species 0.000 claims description 8
- 230000033001 locomotion Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 5
- 238000013459 approach Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 3
- 238000003708 edge detection Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000003709 image segmentation Methods 0.000 claims description 3
- 230000008676 import Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005286 illumination Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000009776 industrial production Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000919 ceramic Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
本发明公开了一种基于双目视觉的位姿识别与抓取的装置与方法。包括工件传送模块、图像采集模块和目标工件抓取模块,工件传送模块通过传送带将目标工件传送到双目相机下方,当双目视觉相机支架下方安装的接近传感器接受工件到达信号时传送带停止工作,便于左右工业相机采集图像;图像采集模块采集图像后传输至计算机,通过图像预处理算法和位姿识别算法处理后获得目标工件的位姿信息,再对位姿信息进行处理、轨迹规划,将数据传输至抓取装置;抓取装置中的机械手对工件进行抓取并摆放至指定区域。本发明可以检测目标工件的位姿信息并进行轨迹规划,利用抓取装置对目标工件进行抓取,提升效率。
The invention discloses a device and method for binocular vision-based pose recognition and grasping. It includes a workpiece transmission module, an image acquisition module and a target workpiece grabbing module. The workpiece transmission module transmits the target workpiece to the bottom of the binocular camera through the conveyor belt. When the proximity sensor installed under the binocular vision camera bracket receives the workpiece arrival signal, the conveyor belt stops working. It is convenient for left and right industrial cameras to collect images; the image acquisition module collects images and transmits them to the computer, and obtains the pose information of the target workpiece after processing through the image preprocessing algorithm and pose recognition algorithm, and then processes the pose information, trajectory planning, and the data Transfer to the grabbing device; the manipulator in the grabbing device grabs the workpiece and places it in the designated area. The present invention can detect the position and posture information of the target workpiece and perform trajectory planning, and use the grasping device to grasp the target workpiece, thereby improving the efficiency.
Description
技术领域technical field
本发明属于机器视觉领域,具体涉及了一种基于双目视觉的位姿识别与抓取的装置与方法。The invention belongs to the field of machine vision, and in particular relates to a binocular vision-based pose recognition and grasping device and method.
背景技术Background technique
双目立体视觉是机器视觉的一种重要形式,它是基于视差原理并利用成像设备从不同的位置获取被测物体的两幅图像,通过计算图像对应点间的位置偏差,来获取物体三维几何信息的方法。Binocular stereo vision is an important form of machine vision. It is based on the principle of parallax and uses imaging equipment to obtain two images of the measured object from different positions, and obtains the three-dimensional geometry of the object by calculating the position deviation between the corresponding points of the image. method of information.
随着信息技术和机器人技术飞速的发展,机器人控制技术所涵盖的内容越来越丰富。工业机器人作为一种新兴的数字控制设备,一经问世就得到了广泛的关注,经过国内外的专家学者和科研人员的不断探索,功能得到逐渐完善。其中双目立体视觉技术被广泛运用在物体识别、虚拟现实、工业检测等领域。双目立体视觉技术可以在多种条件下灵活的获得物体的立体信息,相对单目视觉有着不可比拟的优势。With the rapid development of information technology and robot technology, the content covered by robot control technology is becoming more and more abundant. As a new type of digital control equipment, industrial robots have attracted widespread attention since they came out. After continuous exploration by experts, scholars and researchers at home and abroad, their functions have been gradually improved. Among them, binocular stereo vision technology is widely used in object recognition, virtual reality, industrial inspection and other fields. Binocular stereo vision technology can flexibly obtain stereo information of objects under various conditions, and has incomparable advantages over monocular vision.
在工业生产流水线上,传统的供料设备为振动盘,但是对于软磁体、陶瓷片等弱冲击韧性材料,无法使用振动盘上料,否则将导致材料发生破碎、产生裂缝等后果。而在工业机械手使用场合中,基于单目视觉机械手的的一般动作是采用“抓取-放置”的动作,运用单目视觉进行零件的二维测量,通过固定的参数获取零件的旋转角度、位姿,以实现零件的抓取。在利用双目视觉工业机械手的情景中,机械手对零件、产品的抓取仅停留在“定位-抓取-放置”的程度上,对产品的初始、结束姿态没有考虑。所以在双目视觉的基础上,进行对工件进行位姿识别、机械手轨迹规划的研究,对工业生产效率的提高具有很大的积极意义。In the industrial production line, the traditional feeding equipment is a vibrating plate, but for weak impact tough materials such as soft magnets and ceramic sheets, the vibrating plate cannot be used for feeding, otherwise the material will be broken and cracked. In the application of industrial manipulators, the general action of manipulators based on monocular vision is to use the "grab-place" action, using monocular vision to perform two-dimensional measurement of parts, and obtain the rotation angle and position of parts through fixed parameters. attitude to achieve the grabbing of parts. In the scenario of using binocular vision industrial manipulators, the grasping of parts and products by the manipulator only stays at the level of "positioning-grabbing-placement", without considering the initial and final posture of the product. Therefore, on the basis of binocular vision, the research on the pose recognition of the workpiece and the trajectory planning of the manipulator has great positive significance for the improvement of industrial production efficiency.
发明内容Contents of the invention
为了克服现有技术的不足,本发明提出了一种基于双目视觉的位姿识别与抓取的装置与方法,可以完成对无法在振动盘上料的工件进行位姿识别以及抓取,包括通过双目视觉系统及相关图像处理与三维重建算法,利用相关图像算法进行工件的姿态识别,机械手的运动轨迹规划,实现对工件的位姿识别与抓取。In order to overcome the deficiencies of the prior art, the present invention proposes a binocular vision-based pose recognition and grasping device and method, which can complete the pose recognition and grasping of workpieces that cannot be loaded on the vibrating plate, including Through the binocular vision system and related image processing and 3D reconstruction algorithm, the relevant image algorithm is used to recognize the attitude of the workpiece, and the motion trajectory planning of the manipulator is used to realize the recognition and grasping of the workpiece.
本发明采用的技术方案如下:The technical scheme that the present invention adopts is as follows:
一、一种基于双目视觉的位姿识别与抓取的装置1. A device for pose recognition and grasping based on binocular vision
包括工件传送模块、图像采集模块和目标工件抓取模块;Including workpiece transfer module, image acquisition module and target workpiece grabbing module;
工件传送模块包括带式传送机、带式传送机电机驱动装置和接近传感器,带式传送机上安装有用于传送目标工件的传送带,位于传送带一侧的带式传送机机架上安装有接近传感器。The workpiece transmission module includes a belt conveyor, a belt conveyor motor drive and a proximity sensor. A conveyor belt for transmitting target workpieces is installed on the belt conveyor, and a proximity sensor is installed on the belt conveyor frame at one side of the conveyor belt.
图像采集模块包括工业相机、相机支架、条形光源和双目相机夹具,位于传送带两侧的带式传送机机架上固定有相机支架,相机支架两侧之间安装有高度可调的双目相机夹具,双目相机夹具的导轨上滑动安装有两个CCD工业相机,两个工业相机用于拍摄经过接近传感器的目标工件,两个工业相机分别为左相机和右相机;相机支架两侧底部安装有对称布置的两个条形光源,条形光源用于图像采集时提供照明;接近传感器靠近相机支架布置。The image acquisition module includes an industrial camera, a camera bracket, a strip light source and a binocular camera fixture. The camera bracket is fixed on the belt conveyor frame on both sides of the conveyor belt, and a height-adjustable binocular camera is installed between the two sides of the camera bracket. Camera fixture, two CCD industrial cameras are slidingly installed on the guide rail of the binocular camera fixture. The two industrial cameras are used to shoot the target workpiece passing the proximity sensor. The two industrial cameras are the left camera and the right camera respectively; the bottom of both sides of the camera bracket Two strip light sources arranged symmetrically are installed, and the strip light sources are used to provide illumination during image acquisition; the proximity sensor is arranged close to the camera bracket.
目标工件抓取模块包括四轴机械手和控制柜,控制柜位于带式传送机一侧且高度与传送带高度一致,控制柜与传送带高度一致使目标工件与机械手坐标系原点高度保持一致,便于坐标换算;四轴机械手通过其底座固定于控制柜上。The target workpiece grabbing module includes a four-axis manipulator and a control cabinet. The control cabinet is located on the side of the belt conveyor and its height is consistent with the height of the conveyor belt. The height of the control cabinet and the conveyor belt is consistent so that the height of the target workpiece is consistent with the origin of the coordinate system of the manipulator, which is convenient for coordinate conversion. ; The four-axis manipulator is fixed on the control cabinet through its base.
二、采用上述装置的一种基于双目视觉的位姿识别与抓取的方法2. A method of pose recognition and grasping based on binocular vision using the above-mentioned device
包括以下步骤:Include the following steps:
步骤一:调整工业相机的高度、拍摄角度和两个工业相机之间的距离,使得目标工件位于工业相机的焦距范围内;Step 1: Adjust the height of the industrial camera, the shooting angle and the distance between the two industrial cameras so that the target workpiece is within the focal length range of the industrial camera;
双目相机夹具沿相机支架两侧的竖直杆上下移动从而实现工业相机高度的调节,两个工业相机沿双目相机夹具导轨移动从而实现两个工业相机之间的距离调节,通过调节两个工业相机的安装角度实现工业相机拍摄角度的调节。The binocular camera fixture moves up and down along the vertical rods on both sides of the camera bracket to adjust the height of the industrial camera, and the two industrial cameras move along the guide rails of the binocular camera fixture to realize the distance between the two industrial cameras. The installation angle of the industrial camera realizes the adjustment of the shooting angle of the industrial camera.
步骤二:根据张正友标定法对左右相机进行双目标定获得相机的内外参数矩阵,根据参数矩阵对左右相机进行立体校正后得到校正映射图;Step 2: According to the Zhang Zhengyou calibration method, perform double-target calibration on the left and right cameras to obtain the internal and external parameter matrix of the camera, and perform stereo calibration on the left and right cameras according to the parameter matrix to obtain the correction map;
步骤三:对四轴机械手和工业相机选择手到眼(Eye-To-Hand)的手眼标定模式进行手眼标定,获得相机坐标系相对于机械手坐标系的位姿转换关系;Step 3: Select the hand-to-eye (Eye-To-Hand) hand-eye calibration mode for the four-axis manipulator and industrial camera to perform hand-eye calibration, and obtain the pose transformation relationship between the camera coordinate system and the manipulator coordinate system;
其中,相机坐标系由工业相机的左相机为原点建立得到,机械手坐标系由四轴机械手底座为原点建立得到;Among them, the camera coordinate system is established by the left camera of the industrial camera as the origin, and the manipulator coordinate system is established by the four-axis manipulator base as the origin;
步骤四:待抓取的目标工件通过带式传送机传送至双目相机下方位置,且接近传感器接收到目标工件的信号时,带式传送机电机驱动装置停止带式传送机的工作;Step 4: The target workpiece to be grabbed is conveyed to the position below the binocular camera through the belt conveyor, and when the proximity sensor receives the signal of the target workpiece, the belt conveyor motor drive device stops the work of the belt conveyor;
步骤五:左右相机拍摄经过接近传感器的目标工件的图像,并将拍摄到的目标工件图像传输至计算机;Step 5: The left and right cameras capture images of the target workpiece passing through the proximity sensor, and transmit the captured image of the target workpiece to the computer;
步骤六:利用步骤一获取的校正映射图对左右相机采集的目标工件图像进行校正,然后对图像依次进行图像分割、灰度变换、中值滤波和边缘检测的预处理;Step 6: Use the correction map obtained in step 1 to correct the target workpiece images collected by the left and right cameras, and then perform image segmentation, grayscale transformation, median filtering and edge detection preprocessing on the images in sequence;
步骤七:利用NCC相似性度量算法对步骤六预处理后的图像进行立体匹配,通过双目立体视觉的金字塔模型寻找匹配点,将所有匹配点坐标集合后形成的点云图像构成深度图,对深度图进行平均深度计算从而得到目标工件在相机坐标系下的三维坐标;Step 7: Use the NCC similarity measurement algorithm to perform stereo matching on the image preprocessed in step 6, find matching points through the pyramid model of binocular stereo vision, and form a depth map from the point cloud image formed by gathering the coordinates of all matching points. Calculate the average depth of the depth map to obtain the three-dimensional coordinates of the target workpiece in the camera coordinate system;
步骤八:基于Halcon对目标工件进行姿态估计得到目标工件的姿态信息Q;Step 8: Estimating the attitude of the target workpiece based on Halcon to obtain the attitude information Q of the target workpiece;
步骤九:已知四轴机械手末端执行器的初始位置和初始姿态(即已知末端执行器在初始状态的三维坐标和姿态信息),采用笛卡尔位置控制模式进行轨迹规划,笛卡尔位置控制模式根据目标工件相对于末端执行器的方位角和目标工件的三维坐标进行轨迹规划,四轴机械手末端执行器根据规划的轨迹从初始位置移动至目标工件附近,且移动过程中使末端执行器始终指向目标工件,可以更高效地进行抓取;Step 9: Know the initial position and initial attitude of the end effector of the four-axis manipulator (that is, the three-dimensional coordinates and attitude information of the end effector in the initial state are known), use the Cartesian position control mode for trajectory planning, and the Cartesian position control mode Trajectory planning is performed according to the azimuth angle of the target workpiece relative to the end effector and the three-dimensional coordinates of the target workpiece. The end effector of the four-axis manipulator moves from the initial position to the vicinity of the target workpiece according to the planned trajectory, and the end effector always points to the target workpiece during the movement process. The target workpiece can be grabbed more efficiently;
步骤十:四轴机械手末端执行器离目标工件缓冲距离为d时调整四轴机械手末端执行器的姿态信息,然后以渐进的方式靠近目标工件完成抓取;Step 10: When the buffer distance between the end effector of the four-axis manipulator and the target workpiece is d, adjust the posture information of the end effector of the four-axis manipulator, and then approach the target workpiece in a gradual manner to complete the grasping;
步骤十一:四轴机械手抓取到目标工件时,四轴机械手先抓取目标工件沿机械手坐标系的Z轴向上移动设定距离H,之后四轴机械手控制末端执行器移动至初始位置,然后对姿态信息Q和末端执行器在初始状态的姿态信息进行逆运动求解后将机械手末端执行器调整回末端执行器的初始姿态,此时目标工件完成初始姿态至中间姿态的转移过程;Step 11: When the four-axis manipulator grabs the target workpiece, the four-axis manipulator first grabs the target workpiece and moves up the set distance H along the Z axis of the manipulator coordinate system, and then the four-axis manipulator controls the end effector to move to the initial position. Then, the attitude information Q and the attitude information of the end effector in the initial state are solved by inverse motion, and then the end effector of the manipulator is adjusted back to the initial attitude of the end effector. At this time, the target workpiece completes the transfer process from the initial attitude to the intermediate attitude;
步骤十二:四轴机械手继续抓取目标工件至目标堆放台,目标工件处于结束姿态,完成目标工件的定位-抓取-放置过程。Step 12: The four-axis manipulator continues to grab the target workpiece to the target stacking platform, and the target workpiece is in the final posture, and the positioning-grabbing-placement process of the target workpiece is completed.
所述步骤八具体为:首先利用SolidWorks对目标工件进行3D图像绘制,3D图像中的目标工件位姿是理想位姿;将工件的3D图像导入至Halcon中并读取,利用Halcon中的create_model_3d()算子生成3D形状匹配模板,利用算子find_shape_model_3d()对步骤五得到的预处理图像与3D形状匹配模板进行匹配,得到目标工件相对于机械手坐标系的姿态信息Q。Described step eight specifically is: first utilize SolidWorks to carry out 3D image drawing to target workpiece, and the target workpiece pose in 3D image is ideal pose; Import the 3D image of workpiece into Halcon and read, use create_model_3d( ) operator to generate a 3D shape matching template, use the operator find_shape_model_3d() to match the preprocessed image obtained in step 5 with the 3D shape matching template, and obtain the attitude information Q of the target workpiece relative to the manipulator coordinate system.
所述步骤九中目标工件相对于末端执行器的方位角通过下述计算得到:先通过步骤二的位姿转换关系将目标工件在相机坐标系下的三维坐标转换为目标工件在机械手坐标系的三维坐标,根据目标工件和末端执行器在机械手坐标系下的三维坐标计算目标工件相对于末端执行器的方位角。The azimuth angle of the target workpiece relative to the end effector in the step 9 is obtained by the following calculation: first, the three-dimensional coordinates of the target workpiece in the camera coordinate system are converted into the position of the target workpiece in the manipulator coordinate system through the pose conversion relationship in step 2 Three-dimensional coordinates, calculate the azimuth angle of the target workpiece relative to the end effector according to the three-dimensional coordinates of the target workpiece and the end effector in the manipulator coordinate system.
根据参数m、n、q调整步骤十中四轴机械手末端执行器的姿态信息,参数m、n、q的计算方法如下:Adjust the attitude information of the four-axis manipulator end effector in step 10 according to the parameters m, n, and q. The calculation methods of the parameters m, n, and q are as follows:
在机械手坐标系下,将四轴机械手末端执行器绕x轴、y轴、z轴逆时针旋转的角度设为m、n、q,m、n、q分别为控制末端执行器上下、左右、旋转运动的参数;In the manipulator coordinate system, set the counterclockwise rotation angles of the end effector of the four-axis manipulator around the x-axis, y-axis, and z-axis as m, n, and q, where m, n, and q are the up-down, left-right, and Parameters of the rotational movement;
由于末端执行器在抓取过程中始终指向物体,因此参数m=α;Since the end effector always points to the object during the grasping process, the parameter m = α;
其中,m为末端执行器绕机械手坐标系的x轴的旋转角度(旋转的起始位置为z轴),α为目标工件相对于末端执行器沿x轴的方位角;Among them, m is the rotation angle of the end effector around the x-axis of the manipulator coordinate system (the starting position of the rotation is the z-axis), and α is the azimuth angle of the target workpiece relative to the end effector along the x-axis;
参数n与q根据目标工件的姿态信息Q进行计算,目标工件的姿态信息Q包含有目标工件绕机械手坐标系的x、y、z轴旋转的欧拉角θx、θy、θz,从而得到末端执行器绕机械手坐标系y轴的旋转角度n为θy,绕机械手坐标系z轴的旋转角度q为θz:The parameters n and q are calculated according to the attitude information Q of the target workpiece, which includes the Euler angles θ x , θ y , and θ z of the target workpiece rotating around the x, y, and z axes of the manipulator coordinate system, so that The rotation angle n of the end effector around the y-axis of the manipulator coordinate system is θ y , and the rotation angle q around the z-axis of the manipulator coordinate system is θ z :
即n=θy That is, n = θ y
q=θz q = θz
其中工件坐标系是以目标工件为原点建立的坐标系。The workpiece coordinate system is the coordinate system established with the target workpiece as the origin.
四轴机械手1末端执行器抓取到目标工件10时的姿态信息与步骤八目标工件10的姿态信息Q相同。The attitude information when the end effector of the four-axis manipulator 1 grabs the target workpiece 10 is the same as the attitude information Q of the target workpiece 10 in step eight.
所述目标工件的姿态信息Q基于TCP/IP协议SOCKET通讯的方式传输至四轴机械手1的控制系统,从而使四轴机械手完成对目标工件的抓取。The posture information Q of the target workpiece is transmitted to the control system of the four-axis manipulator 1 based on the TCP/IP protocol SOCKET communication, so that the four-axis manipulator completes the grasping of the target workpiece.
本发明的有益效果:Beneficial effects of the present invention:
本发明可以完成对无法在振动盘上料的工件进行位姿识别,利用双目视觉系统以及相关图像处理算法和三维重建算法进行目标工件的位姿识别获取位姿信息,位姿信息包括工件的姿态参数和位置距离,再对位姿信息进行处理、轨迹规划,将数据传输至抓取装置,通过轨迹规划对目标工件进行最优姿态的抓取并摆放至目标堆放台,一定程度上提高了对目标工件抓取的效率。The present invention can complete the pose recognition of workpieces that cannot be loaded on the vibrating plate, and use the binocular vision system, related image processing algorithms and three-dimensional reconstruction algorithms to perform pose recognition of the target workpiece to obtain pose information, and the pose information includes the position and pose of the workpiece. Attitude parameters and position distance, and then process the position and attitude information, trajectory planning, and transmit the data to the grabbing device. Through trajectory planning, the target workpiece is captured with the optimal posture and placed on the target stacking platform, which improves to a certain extent. Improve the efficiency of grabbing the target workpiece.
附图说明Description of drawings
图1是本发明装置的整体结构示意图;Fig. 1 is the overall structural representation of device of the present invention;
图2是本发明中工件传送装置工作的流程图;Fig. 2 is the flow chart of work of workpiece transmission device among the present invention;
图3是本发明中双目视觉系统工作的流程图;Fig. 3 is the flowchart of binocular vision system work among the present invention;
图4是本发明中目标工件的抓取装置的工作流程图;Fig. 4 is the work flowchart of the grasping device of target workpiece among the present invention;
图5是相机坐标系的示意图。FIG. 5 is a schematic diagram of a camera coordinate system.
图6是工件坐标系的示意图。Fig. 6 is a schematic diagram of a workpiece coordinate system.
图7是机械手坐标系的示意图。Fig. 7 is a schematic diagram of the manipulator coordinate system.
图8是四轴机械手末端执行器在机械手坐标系的位置示意图Figure 8 is a schematic diagram of the position of the end effector of the four-axis manipulator in the manipulator coordinate system
图中:1.四轴机械手、2.工业相机、3.相机支架、4.带式传送机电机驱动装置、5.条形光源、6.目标工件堆放台、7.控制柜、8.接近传感器、9.带式传送机、10.目标工件、11.双目相机夹具。In the figure: 1. Four-axis manipulator, 2. Industrial camera, 3. Camera bracket, 4. Belt conveyor motor drive device, 5. Strip light source, 6. Target workpiece stacking platform, 7. Control cabinet, 8. Proximity Sensor, 9. Belt conveyor, 10. Target workpiece, 11. Binocular camera fixture.
具体实施方式Detailed ways
下面结合附图和实施例对本发明作进一步说明。The present invention will be further described below in conjunction with drawings and embodiments.
如图1所示,本发明包括工件传送模块、图像采集模块和目标工件抓取模块;工件传送模块包括带式传送机9、带式传送机电机驱动装置4和接近传感器8,带式传送机9上安装有用于传送目标工件10的传送带,位于传送带一侧的带式传送机9机架上安装有接近传感器8;图像采集模块包括工业相机2、相机支架3、条形光源5和双目相机夹具11,位于传送带两侧的带式传送机9机架上固定有相机支架3,相机支架3两侧之间安装有高度可调的双目相机夹具11,双目相机夹具11的导轨上滑动安装有两个CCD工业相机2,两个工业相机2用于拍摄经过接近传感器8的目标工件10,两个工业相机2分别为左相机和右相机;相机支架3两侧底部安装有对称布置的两个条形光源5,条形光源5用于图像采集时提供照明;接近传感器8靠近相机支架3布置;目标工件抓取模块包括四轴机械手1和控制柜7,控制柜7位于带式传送机9一侧且高度与传送带高度一致;四轴机械手1通过其底座固定于控制柜上。As shown in Figure 1, the present invention comprises workpiece transmission module, image acquisition module and target workpiece grasping module; Workpiece transmission module comprises belt conveyor 9, belt conveyor motor driver 4 and proximity sensor 8, and belt conveyor 9 is equipped with a conveyor belt for transmitting the target workpiece 10, and a proximity sensor 8 is installed on the belt conveyor 9 frame at one side of the conveyor belt; the image acquisition module includes an industrial camera 2, a camera bracket 3, a strip light source 5 and a binocular Camera fixture 11, camera support 3 is fixed on the belt conveyor 9 frame that is positioned at both sides of conveyer belt, and height-adjustable binocular camera fixture 11 is installed between the two sides of camera support 3, on the guide rail of binocular camera fixture 11 Two CCD industrial cameras 2 are slidingly installed, and the two industrial cameras 2 are used to photograph the target workpiece 10 passing the proximity sensor 8. The two industrial cameras 2 are respectively the left camera and the right camera; the bottom of the camera bracket 3 is installed with symmetrical arrangements Two strip light sources 5, the strip light source 5 is used to provide illumination during image acquisition; the proximity sensor 8 is arranged close to the camera support 3; the target workpiece grasping module includes a four-axis manipulator 1 and a control cabinet 7, and the control cabinet 7 is located in the belt type One side of the conveyor 9 and the height is consistent with the height of the conveyor belt; the four-axis manipulator 1 is fixed on the control cabinet through its base.
如图2所示,工件传送装置将目标工件10传送至接近传感器8附近,接近传感器8检测到目标工件10的靠近,产生信号,并且传输信号至带式传送机电机驱动装置4使传送带电机停止工作,制动器工作,最终使传送带停止,工件此时处于左右CCD工业相机2下方附近,可以利于相机工作,对工件进行图像采集。As shown in Figure 2, the workpiece conveying device transmits the target workpiece 10 to the vicinity of the proximity sensor 8, and the proximity sensor 8 detects the approach of the target workpiece 10, generates a signal, and transmits the signal to the belt conveyor motor drive device 4 to stop the conveyor belt motor work, the brake works, and finally the conveyor belt is stopped, and the workpiece is now near the bottom of the left and right CCD industrial cameras 2, which can facilitate the work of the camera and image acquisition of the workpiece.
图像采集模块中,左右CCD工业相机2固定在双目相机夹具11上,夹具上有导轨,可以手动或者利用其它工具调整两个CCD工业双目相机2的左右间距以及角度,本装置中的左右CCD工业相机2以平视双目为准。In the image acquisition module, the left and right CCD industrial cameras 2 are fixed on the binocular camera fixture 11. There are guide rails on the fixture, and the left and right spacing and angle of the two CCD industrial binocular cameras 2 can be adjusted manually or by other tools. CCD industrial camera 2 is subject to binocular vision.
具体实施方式(如图3和图4所示):Specific implementation (as shown in Figure 3 and Figure 4):
如图5所示灰色的坐标系(X,Y,Z)为相机坐标系,黑色的两个二维坐标系分别为左、右相机的像素坐标系(x,y),左相机的像素坐标系坐落于相机坐标系的(0,0,0)处,右相机的像素坐标系坐落为相机坐标系的(Tx,0,0)处。图中的(X,Y,Z)为相机坐标系中的一个点。As shown in Figure 5, the gray coordinate system (X, Y, Z) is the camera coordinate system, the two black two-dimensional coordinate systems are the pixel coordinate system (x, y) of the left and right cameras, and the pixel coordinates of the left camera The frame is located at (0,0,0) of the camera coordinate system, and the pixel coordinate system of the right camera is located at (Tx,0,0) of the camera coordinate system. (X, Y, Z) in the figure is a point in the camera coordinate system.
如图6所示为工件坐标系,以目标工件底部圆心为基点建立。As shown in Figure 6, the workpiece coordinate system is established with the center of the bottom circle of the target workpiece as the base point.
如图7所示为机械手坐标系,以机械手底座中心原点建立。As shown in Figure 7, the coordinate system of the manipulator is established with the origin of the center of the manipulator base.
步骤一:通过调整工业相机2的高度、拍摄角度和两个工业相机2之间的距离,使得目标工件10位于工业相机2的焦距范围内;Step 1: by adjusting the height of the industrial camera 2, the shooting angle and the distance between the two industrial cameras 2, the target workpiece 10 is located within the focal length range of the industrial camera 2;
双目相机夹具11沿相机支架3两侧的竖直杆上下移动从而实现工业相机2高度的调节,两个工业相机2沿双目相机夹具11导轨移动从而实现两个工业相机2之间的距离调节,通过调节两个工业相机2的安装角度实现工业相机2拍摄角度的调节。The binocular camera fixture 11 moves up and down along the vertical rods on both sides of the camera bracket 3 to adjust the height of the industrial camera 2, and the two industrial cameras 2 move along the guide rail of the binocular camera fixture 11 to realize the distance between the two industrial cameras 2 Adjustment, the adjustment of the shooting angle of the industrial camera 2 is realized by adjusting the installation angle of the two industrial cameras 2 .
步骤二:根据张正友标定法对左右相机进行双目标定获得相机的内外参数矩阵,根据参数矩阵对左右相机进行立体校正后得到校正映射图。Step 2: According to the Zhang Zhengyou calibration method, perform double-target calibration on the left and right cameras to obtain the internal and external parameter matrix of the camera, and perform stereo calibration on the left and right cameras according to the parameter matrix to obtain a correction map.
步骤三:对四轴机械手1和工业相机2选择手到眼(Eye-To-Hand)的手眼标定模式进行手眼标定,获得相机坐标系相对于机械手坐标系的位姿转换关系;Step 3: Select the hand-to-eye (Eye-To-Hand) hand-eye calibration mode for the four-axis manipulator 1 and the industrial camera 2 to perform hand-eye calibration, and obtain the pose transformation relationship between the camera coordinate system and the manipulator coordinate system;
其中,相机坐标系由工业相机的左相机为原点建立得到,机械手坐标系由四轴机械手1底座为原点建立得到。Among them, the camera coordinate system is established by the left camera of the industrial camera as the origin, and the manipulator coordinate system is established by the base of the four-axis manipulator 1 as the origin.
步骤四:待抓取的目标工件10通过带式传送机9传送至双目相机下方位置,且接近传感器8接收到目标工件10的信号时,带式传送机电机驱动装置4停止带式传送机9的工作。Step 4: The target workpiece 10 to be grabbed is conveyed to the position below the binocular camera through the belt conveyor 9, and when the proximity sensor 8 receives the signal of the target workpiece 10, the belt conveyor motor drive device 4 stops the belt conveyor 9 jobs.
步骤五:左右相机拍摄经过接近传感器8的目标工件10的图像,并将拍摄到的目标工件图像传输至计算机。Step 5: The left and right cameras capture images of the target workpiece 10 passing the proximity sensor 8, and transmit the captured images of the target workpiece to the computer.
步骤六:利用步骤一获取的校正映射图对左右相机采集的目标工件图像进行校正,然后对图像依次进行图像分割、灰度变换、中值滤波和边缘检测的预处理。Step 6: Use the correction map obtained in step 1 to correct the target workpiece images collected by the left and right cameras, and then perform image segmentation, grayscale transformation, median filtering and edge detection preprocessing on the images in sequence.
步骤七:利用NCC相似性度量算法对步骤六预处理后的图像进行立体匹配;通过双目立体视觉的金字塔模型寻找匹配点,将所有匹配点坐标集合后形成的点云图像构成深度图,对深度图进行平均深度计算从而得到目标工件10在相机坐标系下的三维坐标;Step 7: Use the NCC similarity measurement algorithm to perform stereo matching on the image preprocessed in Step 6; find matching points through the pyramid model of binocular stereo vision, and form a depth map from the point cloud image formed after the coordinates of all matching points are assembled. Calculate the average depth of the depth map to obtain the three-dimensional coordinates of the target workpiece 10 in the camera coordinate system;
NCC相似性度量算法可以克服图像中光照不均匀等影响。The NCC similarity measurement algorithm can overcome the influence of uneven illumination in the image.
步骤八:基于Halcon对目标工件10进行姿态估计得到目标工件10的姿态信息Q;Step 8: Estimating the attitude of the target workpiece 10 based on Halcon to obtain the attitude information Q of the target workpiece 10;
首先利用SolidWorks对目标工件10进行3D图像绘制,3D图像中的目标工件位姿是理想位姿;将工件的3D图像导入至Halcon中并读取,利用Halcon中的create_model_3d()算子生成3D形状匹配模板,利用算子find_shape_model_3d()对步骤五得到的预处理图像与3D形状匹配模板进行匹配,得到目标工件10相对于机械手坐标系的姿态信息Q。First, use SolidWorks to draw a 3D image of the target workpiece 10. The pose of the target workpiece in the 3D image is an ideal pose; import the 3D image of the workpiece into Halcon and read it, and use the create_model_3d() operator in Halcon to generate a 3D shape Matching template, use the operator find_shape_model_3d() to match the preprocessed image obtained in step 5 with the 3D shape matching template, and obtain the attitude information Q of the target workpiece 10 relative to the manipulator coordinate system.
步骤九:已知四轴机械手1末端执行器的初始位置和初始姿态(即已知末端执行器在初始状态的三维坐标和姿态信息),采用笛卡尔位置控制模式进行轨迹规划,四轴机械手1末端执行器根据规划的轨迹从初始位置移动至目标工件10附近,且移动过程中使末端执行器始终指向目标工件10,可以更高效地进行抓取;Step 9: Know the initial position and initial attitude of the end effector of the four-axis manipulator 1 (that is, the three-dimensional coordinates and attitude information of the end effector in the initial state are known), use the Cartesian position control mode for trajectory planning, and the four-axis manipulator 1 The end effector moves from the initial position to the vicinity of the target workpiece 10 according to the planned trajectory, and keeps the end effector always pointing to the target workpiece 10 during the movement, so that the grasping can be performed more efficiently;
笛卡尔位置控制模式根据目标工件12相对于末端执行器的方位角和目标工件12在相机坐标系下的三维坐标进行轨迹规划;目标工件12相对于末端执行器的方位角通过下述计算得到:先通过步骤二的位姿转换关系将目标工件10在相机坐标系下的三维坐标转换为目标工件10在机械手坐标系的三维坐标,根据目标工件10和末端执行器在机械手坐标系下的三维坐标计算目标工件10相对于末端执行器的方位角。The Cartesian position control mode performs trajectory planning according to the azimuth angle of the target workpiece 12 relative to the end effector and the three-dimensional coordinates of the target workpiece 12 in the camera coordinate system; the azimuth angle of the target workpiece 12 relative to the end effector is obtained by the following calculation: First, convert the three-dimensional coordinates of the target workpiece 10 in the camera coordinate system into the three-dimensional coordinates of the target workpiece 10 in the manipulator coordinate system through the pose conversion relationship in step 2, according to the three-dimensional coordinates of the target workpiece 10 and the end effector in the manipulator coordinate system The azimuth of the target workpiece 10 relative to the end effector is calculated.
步骤十:在四轴机械手1在抓取过程中接近目标工件10时,为了防止机械手姿态调整过程中将工件触碰改变其姿态,在此设置一个缓冲距离d,在机械手末端到达d附近时开始进行姿态调整,然后再以渐进的方式到达目标点位置完成抓取,机械手末端抓取时的姿态为Q。目标工件的姿态信息Q基于TCP/IP协议SOCKET通讯的方式传输至四轴机械手1的控制系统,从而使四轴机械手完成对目标工件的抓取。Step 10: When the four-axis manipulator 1 approaches the target workpiece 10 during the grasping process, in order to prevent the manipulator from touching the workpiece to change its posture during the posture adjustment process, set a buffer distance d here, and start when the end of the manipulator reaches near d Adjust the attitude, and then reach the target point in a gradual manner to complete the grasping. The attitude of the end of the manipulator when grasping is Q. The attitude information Q of the target workpiece is transmitted to the control system of the four-axis manipulator 1 based on the TCP/IP protocol SOCKET communication, so that the four-axis manipulator completes the grasping of the target workpiece.
根据参数m、n、q调整四轴机械手1末端执行器的姿态信息,参数m、n、q的计算方法如下:Adjust the attitude information of the end effector of the four-axis manipulator 1 according to the parameters m, n, and q. The calculation methods of the parameters m, n, and q are as follows:
如图8所示,在机械手坐标系下,将四轴机械手1末端执行器绕x轴、y轴、z轴逆时针旋转的角度设为m、n、q,m、n、q分别为控制末端执行器上下、左右、旋转运动的参数;As shown in Figure 8, in the manipulator coordinate system, set the counterclockwise rotation angle of the end effector of the four-axis manipulator 1 around the x-axis, y-axis, and z-axis as m, n, and q, respectively, and m, n, and q are the control The parameters of the end effector's up and down, left and right, and rotational movements;
由于末端执行器在抓取过程中始终指向物体,因此参数m=α;Since the end effector always points to the object during the grasping process, the parameter m = α;
其中,m为末端执行器以z轴为基准绕机械手坐标系的x轴的旋转角度,α为目标工件相对于末端执行器沿x轴的方位角;Among them, m is the rotation angle of the end effector around the x axis of the manipulator coordinate system based on the z axis, and α is the azimuth angle of the target workpiece relative to the end effector along the x axis;
参数n与q根据目标工件的姿态信息Q进行计算,目标工件的姿态信息Q包含有工件坐标系绕机械手坐标系的x、y、z轴旋转的欧拉角θx、θy、θz,从而得到末端执行器绕机械手坐标系y轴的旋转角度n为θy,绕机械手坐标系z轴的旋转角度q为θz:The parameters n and q are calculated according to the attitude information Q of the target workpiece. The attitude information Q of the target workpiece includes the Euler angles θ x , θ y , and θ z of the workpiece coordinate system rotating around the x, y, and z axes of the manipulator coordinate system. Thus, the rotation angle n of the end effector around the y-axis of the manipulator coordinate system is θ y , and the rotation angle q around the z-axis of the manipulator coordinate system is θ z :
即n=θy That is, n = θ y
q=θz q = θz
其中工件坐标系是以目标工件为原点建立的坐标系。The workpiece coordinate system is the coordinate system established with the target workpiece as the origin.
步骤十一:四轴机械手1抓取到目标工件10时,此时目标工件10的姿态信息与步骤八目标工件10的姿态信息Q相同。四轴机械手1先抓取目标工件10沿机械手坐标系的Z轴向上移动设定距离H,之后四轴机械手1控制末端执行器移动至初始位置,然后对姿态信息Q和末端执行器在初始状态的姿态信息进行逆运动求解后将机械手末端执行器调整回末端执行器的初始姿态,此时目标工件10完成初始姿态至中间姿态的转移过程。Step eleven: when the four-axis manipulator 1 grabs the target workpiece 10 , the posture information of the target workpiece 10 is the same as the posture information Q of the target workpiece 10 in step eight. The four-axis manipulator 1 first grabs the target workpiece 10 and moves up the set distance H along the Z-axis of the manipulator coordinate system, and then the four-axis manipulator 1 controls the end effector to move to the initial position, and then checks the attitude information Q and the end effector at the initial position. After the inverse kinematic solution is performed on the attitude information of the state, the end effector of the manipulator is adjusted back to the initial attitude of the end effector. At this time, the target workpiece 10 completes the transfer process from the initial attitude to the intermediate attitude.
步骤十二:四轴机械手1继续抓取目标工件10至目标堆放台6,目标工件10处于结束姿态,完成目标工件10的定位-抓取-放置过程。Step 12: The four-axis manipulator 1 continues to grab the target workpiece 10 to the target stacking platform 6 , the target workpiece 10 is in the final posture, and the positioning-grabbing-placement process of the target workpiece 10 is completed.
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910870880.0A CN110509281B (en) | 2019-09-16 | 2019-09-16 | Binocular vision-based pose recognition and grabbing device and method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910870880.0A CN110509281B (en) | 2019-09-16 | 2019-09-16 | Binocular vision-based pose recognition and grabbing device and method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110509281A true CN110509281A (en) | 2019-11-29 |
CN110509281B CN110509281B (en) | 2024-10-11 |
Family
ID=68632381
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910870880.0A Active CN110509281B (en) | 2019-09-16 | 2019-09-16 | Binocular vision-based pose recognition and grabbing device and method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110509281B (en) |
Cited By (55)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110963298A (en) * | 2019-12-21 | 2020-04-07 | 深圳市华成工业控制有限公司 | Material taking device and method based on visual following |
CN111028231A (en) * | 2019-12-27 | 2020-04-17 | 易思维(杭州)科技有限公司 | Workpiece position acquisition system based on ARM and FPGA |
CN111250409A (en) * | 2020-01-19 | 2020-06-09 | 上海发那科机器人有限公司 | Automatic sorting and feeding system for robot rubber plugs and working method of automatic sorting and feeding system |
CN111267094A (en) * | 2019-12-31 | 2020-06-12 | 芜湖哈特机器人产业技术研究院有限公司 | Workpiece positioning and grabbing method based on binocular vision |
CN111266315A (en) * | 2020-02-20 | 2020-06-12 | 南京工程学院 | On-line sorting system and method for ore materials based on visual analysis |
CN111283679A (en) * | 2020-01-19 | 2020-06-16 | 路邦科技授权有限公司 | A multi-connected voice control automatic guided transportation system and its control method |
CN111300410A (en) * | 2020-02-25 | 2020-06-19 | 江苏理工学院 | Copper pipe transfer positioning device |
CN111359890A (en) * | 2020-03-13 | 2020-07-03 | 浙江华睿科技有限公司 | Mobile phone screen sorting equipment and mobile phone screen sorting method |
CN111482961A (en) * | 2020-03-31 | 2020-08-04 | 广州富港万嘉智能科技有限公司 | Positioning control method for movement of manipulator, computer-readable storage medium, and device with manipulator |
CN111553955A (en) * | 2020-04-30 | 2020-08-18 | 苏州龙抬头智能科技有限公司 | Multi-view camera three-dimensional system and calibration method thereof |
CN111595266A (en) * | 2020-06-02 | 2020-08-28 | 西安航天发动机有限公司 | Spatial complex trend catheter visual identification method |
CN111721202A (en) * | 2020-06-18 | 2020-09-29 | 博众精工科技股份有限公司 | Mechanism for positioning carrier position |
CN111731837A (en) * | 2020-06-17 | 2020-10-02 | 浙江省机电设计研究院有限公司 | A Grabbing System Based on Industrial Robot Visual Recognition |
CN111780628A (en) * | 2020-05-25 | 2020-10-16 | 中煤科工集团淮北爆破技术研究院有限公司 | Binocular stereo medicine height measuring device |
CN112098137A (en) * | 2020-08-05 | 2020-12-18 | 湖南华菱涟源钢铁有限公司 | Automatic steel plate sampling method and automatic steel plate sampling system |
CN112198192A (en) * | 2020-09-07 | 2021-01-08 | 江苏理工学院 | Infrared image detection device and detection method for fan welding assembly |
CN112265000A (en) * | 2020-11-13 | 2021-01-26 | 辽宁科技大学 | Device and method for quickly acquiring grinding track of magnetic particles on inner surface of bent pipe |
CN112288815A (en) * | 2020-11-06 | 2021-01-29 | 山东产研信息与人工智能融合研究院有限公司 | Target mode position measuring method, system, storage medium and equipment |
CN112342331A (en) * | 2020-11-13 | 2021-02-09 | 中冶赛迪工程技术股份有限公司 | Intelligent mud gun mud filling system and gun mud filling method in iron casting field |
CN112459734A (en) * | 2020-11-26 | 2021-03-09 | 湖南三一石油科技有限公司 | Manipulator positioning method and device, manipulator and storage medium |
CN112518748A (en) * | 2020-11-30 | 2021-03-19 | 广东工业大学 | Automatic grabbing method and system of vision mechanical arm for moving object |
CN112605990A (en) * | 2020-12-04 | 2021-04-06 | 广东拓斯达科技股份有限公司 | Robot vision control method and system |
CN112719830A (en) * | 2020-12-18 | 2021-04-30 | 江苏大学 | Mechanical arm flexible assembling equipment and control method |
CN113009878A (en) * | 2019-12-20 | 2021-06-22 | 中国科学院沈阳计算技术研究所有限公司 | Monocular vision-based moving workpiece attitude estimation method and device |
CN113020959A (en) * | 2021-03-11 | 2021-06-25 | 中国科学院自动化研究所 | Binocular vision-based automatic joint tightening angle prediction device and system |
CN113021037A (en) * | 2021-03-09 | 2021-06-25 | 苏州名匠阀门设备有限公司 | Gate valve blank non-contact self-alignment clamping system and self-alignment clamping method |
CN113205561A (en) * | 2021-05-20 | 2021-08-03 | 清华大学 | Rotating shaft pose detection method and device based on vision |
CN113219926A (en) * | 2021-05-13 | 2021-08-06 | 中国计量大学 | Human-machine co-fusion manufacturing unit safety risk assessment method based on digital twin system |
CN113221953A (en) * | 2021-04-14 | 2021-08-06 | 上海交通大学宁波人工智能研究院 | Target attitude identification system and method based on example segmentation and binocular depth estimation |
CN113232015A (en) * | 2020-05-27 | 2021-08-10 | 杭州中为光电技术有限公司 | Robot space positioning and grabbing control method based on template matching |
CN113297221A (en) * | 2021-05-25 | 2021-08-24 | 山东万国云大数据科技有限公司 | Data center monitoring system based on three-dimensional model |
CN113547520A (en) * | 2021-07-21 | 2021-10-26 | 广东拓斯达科技股份有限公司 | Manipulator movement alignment method, device and system and storage medium |
CN113551661A (en) * | 2020-04-23 | 2021-10-26 | 曰轮法寺 | Pose identification and track planning method, device and system, storage medium and equipment |
CN113602799A (en) * | 2021-08-05 | 2021-11-05 | 西南科技大学 | A kind of airport luggage handling system and control method thereof |
CN113689509A (en) * | 2021-09-15 | 2021-11-23 | 集美大学 | Binocular vision-based disordered grabbing method and system and storage medium |
CN113706628A (en) * | 2021-08-17 | 2021-11-26 | 成都信息工程大学 | Intelligent transfer robot cooperation system and method for processing characteristic image by using same |
CN113716323A (en) * | 2021-08-17 | 2021-11-30 | 成都新气象科技实业股份有限公司 | Intelligent container carrying method |
CN113815323A (en) * | 2021-09-30 | 2021-12-21 | 河南埃尔森智能科技有限公司 | Rail marking device and marking method based on three-dimensional vision guidance |
CN113822945A (en) * | 2021-09-28 | 2021-12-21 | 天津朗硕机器人科技有限公司 | Workpiece identification and positioning method based on binocular vision |
CN113954072A (en) * | 2021-11-05 | 2022-01-21 | 中国矿业大学 | A vision-guided intelligent identification and positioning system and method for wooden door workpieces |
CN113977637A (en) * | 2021-12-03 | 2022-01-28 | 深圳市超准视觉科技有限公司 | Robot vision identification grabbing system and method applicable to non-precision work bin |
CN114368153A (en) * | 2021-12-13 | 2022-04-19 | 赫比(上海)家用电器产品有限公司 | Injection molding adhesive assembling equipment and assembling method thereof |
CN114408532A (en) * | 2022-02-25 | 2022-04-29 | 宝钢湛江钢铁有限公司 | Method, device and system for adjusting code bits of impact sample blank |
CN114693798A (en) * | 2020-12-31 | 2022-07-01 | 北京小米移动软件有限公司 | Manipulator control method and device |
CN114838659A (en) * | 2022-04-26 | 2022-08-02 | 深圳市商汤科技有限公司 | Manipulator testing device, testing method, calibration method and storage medium |
CN114852624A (en) * | 2022-05-18 | 2022-08-05 | 珠海格力智能装备有限公司 | Automatic feeding device and production line thereof |
CN115072357A (en) * | 2021-03-15 | 2022-09-20 | 中国人民解放军96901部队24分队 | Robot reprint automatic positioning method based on binocular vision |
CN115106312A (en) * | 2022-06-13 | 2022-09-27 | 中实洛阳重型机械有限公司 | Intelligent ore sorting device based on binocular camera laser guide |
CN115115931A (en) * | 2022-01-06 | 2022-09-27 | 华中科技大学无锡研究院 | Rapid workpiece positioning method for robot machining system |
CN115214135A (en) * | 2022-05-30 | 2022-10-21 | 武汉新威奇科技有限公司 | Full-automatic feeding system and method for forging line based on 3D vision |
CN116216272A (en) * | 2023-03-15 | 2023-06-06 | 沃尔林自动化(苏州)有限公司 | Automatic production line for mechanical parts |
CN116330285A (en) * | 2023-03-20 | 2023-06-27 | 深圳市功夫机器人有限公司 | Mechanical arm control method and device, mechanical arm and storage medium |
CN116985182A (en) * | 2023-05-31 | 2023-11-03 | 齐鲁工业大学(山东省科学院) | Four-axis robot work piece wobble plate system based on machine vision |
CN118062559A (en) * | 2024-02-28 | 2024-05-24 | 石家庄优创科技股份有限公司 | Card finishing device with manipulator |
CN110509281B (en) * | 2019-09-16 | 2024-10-11 | 中国计量大学 | Binocular vision-based pose recognition and grabbing device and method |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103895042A (en) * | 2014-02-28 | 2014-07-02 | 华南理工大学 | Industrial robot workpiece positioning grabbing method and system based on visual guidance |
CN106000904A (en) * | 2016-05-26 | 2016-10-12 | 北京新长征天高智机科技有限公司 | Automatic sorting system for household refuse |
CN107192331A (en) * | 2017-06-20 | 2017-09-22 | 佛山市南海区广工大数控装备协同创新研究院 | A kind of workpiece grabbing method based on binocular vision |
CN107649406A (en) * | 2017-09-30 | 2018-02-02 | 南京航空航天大学 | A kind of efficient more material picking up system of binocular vision and method |
CN107767423A (en) * | 2017-10-10 | 2018-03-06 | 大连理工大学 | A kind of mechanical arm target positioning grasping means based on binocular vision |
CN108161931A (en) * | 2016-12-07 | 2018-06-15 | 广州映博智能科技有限公司 | The workpiece automatic identification of view-based access control model and intelligent grabbing system |
KR20180072020A (en) * | 2016-12-20 | 2018-06-29 | 주식회사 수아랩 | Method, apparatus and computer program stored in computer readable medium for correction of image data |
CN109454638A (en) * | 2018-10-31 | 2019-03-12 | 昆山睿力得软件技术有限公司 | A kind of robot grasping system of view-based access control model guidance |
CN109848998A (en) * | 2019-03-29 | 2019-06-07 | 砚山永盛杰科技有限公司 | One kind being used for 3C industry vision four axis flexible robot |
CN109968310A (en) * | 2019-04-12 | 2019-07-05 | 重庆渝博创智能装备研究院有限公司 | A kind of mechanical arm interaction control method and system |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110509281B (en) * | 2019-09-16 | 2024-10-11 | 中国计量大学 | Binocular vision-based pose recognition and grabbing device and method |
-
2019
- 2019-09-16 CN CN201910870880.0A patent/CN110509281B/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103895042A (en) * | 2014-02-28 | 2014-07-02 | 华南理工大学 | Industrial robot workpiece positioning grabbing method and system based on visual guidance |
CN106000904A (en) * | 2016-05-26 | 2016-10-12 | 北京新长征天高智机科技有限公司 | Automatic sorting system for household refuse |
CN108161931A (en) * | 2016-12-07 | 2018-06-15 | 广州映博智能科技有限公司 | The workpiece automatic identification of view-based access control model and intelligent grabbing system |
KR20180072020A (en) * | 2016-12-20 | 2018-06-29 | 주식회사 수아랩 | Method, apparatus and computer program stored in computer readable medium for correction of image data |
CN107192331A (en) * | 2017-06-20 | 2017-09-22 | 佛山市南海区广工大数控装备协同创新研究院 | A kind of workpiece grabbing method based on binocular vision |
CN107649406A (en) * | 2017-09-30 | 2018-02-02 | 南京航空航天大学 | A kind of efficient more material picking up system of binocular vision and method |
CN107767423A (en) * | 2017-10-10 | 2018-03-06 | 大连理工大学 | A kind of mechanical arm target positioning grasping means based on binocular vision |
CN109454638A (en) * | 2018-10-31 | 2019-03-12 | 昆山睿力得软件技术有限公司 | A kind of robot grasping system of view-based access control model guidance |
CN109848998A (en) * | 2019-03-29 | 2019-06-07 | 砚山永盛杰科技有限公司 | One kind being used for 3C industry vision four axis flexible robot |
CN109968310A (en) * | 2019-04-12 | 2019-07-05 | 重庆渝博创智能装备研究院有限公司 | A kind of mechanical arm interaction control method and system |
Non-Patent Citations (1)
Title |
---|
荆鑫: "基于模板匹配的视觉分拣方法及应用研究", 中国优秀硕士学位论文全文数据库信息科技辑, 15 February 2018 (2018-02-15) * |
Cited By (71)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110509281B (en) * | 2019-09-16 | 2024-10-11 | 中国计量大学 | Binocular vision-based pose recognition and grabbing device and method |
CN113009878A (en) * | 2019-12-20 | 2021-06-22 | 中国科学院沈阳计算技术研究所有限公司 | Monocular vision-based moving workpiece attitude estimation method and device |
CN110963298A (en) * | 2019-12-21 | 2020-04-07 | 深圳市华成工业控制有限公司 | Material taking device and method based on visual following |
CN111028231A (en) * | 2019-12-27 | 2020-04-17 | 易思维(杭州)科技有限公司 | Workpiece position acquisition system based on ARM and FPGA |
CN111267094A (en) * | 2019-12-31 | 2020-06-12 | 芜湖哈特机器人产业技术研究院有限公司 | Workpiece positioning and grabbing method based on binocular vision |
CN111250409B (en) * | 2020-01-19 | 2021-10-01 | 上海发那科机器人有限公司 | Automatic sorting and feeding system for robot rubber plugs and working method of automatic sorting and feeding system |
CN111283679A (en) * | 2020-01-19 | 2020-06-16 | 路邦科技授权有限公司 | A multi-connected voice control automatic guided transportation system and its control method |
CN111250409A (en) * | 2020-01-19 | 2020-06-09 | 上海发那科机器人有限公司 | Automatic sorting and feeding system for robot rubber plugs and working method of automatic sorting and feeding system |
CN111266315A (en) * | 2020-02-20 | 2020-06-12 | 南京工程学院 | On-line sorting system and method for ore materials based on visual analysis |
CN111300410A (en) * | 2020-02-25 | 2020-06-19 | 江苏理工学院 | Copper pipe transfer positioning device |
CN111359890A (en) * | 2020-03-13 | 2020-07-03 | 浙江华睿科技有限公司 | Mobile phone screen sorting equipment and mobile phone screen sorting method |
CN111359890B (en) * | 2020-03-13 | 2022-02-18 | 浙江华睿科技股份有限公司 | Mobile phone screen sorting equipment and mobile phone screen sorting method |
CN111482961A (en) * | 2020-03-31 | 2020-08-04 | 广州富港万嘉智能科技有限公司 | Positioning control method for movement of manipulator, computer-readable storage medium, and device with manipulator |
CN113551661A (en) * | 2020-04-23 | 2021-10-26 | 曰轮法寺 | Pose identification and track planning method, device and system, storage medium and equipment |
CN111553955B (en) * | 2020-04-30 | 2024-03-15 | 北京航达青云科技有限公司 | Multi-camera three-dimensional system and calibration method thereof |
CN111553955A (en) * | 2020-04-30 | 2020-08-18 | 苏州龙抬头智能科技有限公司 | Multi-view camera three-dimensional system and calibration method thereof |
CN111780628A (en) * | 2020-05-25 | 2020-10-16 | 中煤科工集团淮北爆破技术研究院有限公司 | Binocular stereo medicine height measuring device |
CN111780628B (en) * | 2020-05-25 | 2022-05-13 | 中煤科工集团淮北爆破技术研究院有限公司 | Binocular stereo medicine height measuring device |
CN113232015A (en) * | 2020-05-27 | 2021-08-10 | 杭州中为光电技术有限公司 | Robot space positioning and grabbing control method based on template matching |
CN111595266A (en) * | 2020-06-02 | 2020-08-28 | 西安航天发动机有限公司 | Spatial complex trend catheter visual identification method |
CN111731837A (en) * | 2020-06-17 | 2020-10-02 | 浙江省机电设计研究院有限公司 | A Grabbing System Based on Industrial Robot Visual Recognition |
CN111731837B (en) * | 2020-06-17 | 2021-10-15 | 浙江省机电设计研究院有限公司 | A Grabbing System Based on Industrial Robot Visual Recognition |
CN111721202A (en) * | 2020-06-18 | 2020-09-29 | 博众精工科技股份有限公司 | Mechanism for positioning carrier position |
CN112098137A (en) * | 2020-08-05 | 2020-12-18 | 湖南华菱涟源钢铁有限公司 | Automatic steel plate sampling method and automatic steel plate sampling system |
CN112198192A (en) * | 2020-09-07 | 2021-01-08 | 江苏理工学院 | Infrared image detection device and detection method for fan welding assembly |
CN112288815B (en) * | 2020-11-06 | 2023-10-17 | 山东产研信息与人工智能融合研究院有限公司 | Target die position measurement method, system, storage medium and device |
CN112288815A (en) * | 2020-11-06 | 2021-01-29 | 山东产研信息与人工智能融合研究院有限公司 | Target mode position measuring method, system, storage medium and equipment |
CN112265000A (en) * | 2020-11-13 | 2021-01-26 | 辽宁科技大学 | Device and method for quickly acquiring grinding track of magnetic particles on inner surface of bent pipe |
CN112265000B (en) * | 2020-11-13 | 2023-12-22 | 辽宁科技大学 | Device and method for rapidly acquiring magnetic particle grinding track on inner surface of bent pipe |
CN112342331A (en) * | 2020-11-13 | 2021-02-09 | 中冶赛迪工程技术股份有限公司 | Intelligent mud gun mud filling system and gun mud filling method in iron casting field |
CN112459734A (en) * | 2020-11-26 | 2021-03-09 | 湖南三一石油科技有限公司 | Manipulator positioning method and device, manipulator and storage medium |
CN112518748A (en) * | 2020-11-30 | 2021-03-19 | 广东工业大学 | Automatic grabbing method and system of vision mechanical arm for moving object |
CN112518748B (en) * | 2020-11-30 | 2024-01-30 | 广东工业大学 | Automatic grabbing method and system for visual mechanical arm for moving object |
CN112605990A (en) * | 2020-12-04 | 2021-04-06 | 广东拓斯达科技股份有限公司 | Robot vision control method and system |
CN112719830A (en) * | 2020-12-18 | 2021-04-30 | 江苏大学 | Mechanical arm flexible assembling equipment and control method |
CN112719830B (en) * | 2020-12-18 | 2022-05-17 | 江苏大学 | A kind of mechanical arm compliant assembly equipment and control method |
CN114693798B (en) * | 2020-12-31 | 2023-12-26 | 北京小米移动软件有限公司 | Method and device for controlling manipulator |
CN114693798A (en) * | 2020-12-31 | 2022-07-01 | 北京小米移动软件有限公司 | Manipulator control method and device |
CN113021037A (en) * | 2021-03-09 | 2021-06-25 | 苏州名匠阀门设备有限公司 | Gate valve blank non-contact self-alignment clamping system and self-alignment clamping method |
CN113020959A (en) * | 2021-03-11 | 2021-06-25 | 中国科学院自动化研究所 | Binocular vision-based automatic joint tightening angle prediction device and system |
CN115072357B (en) * | 2021-03-15 | 2023-07-07 | 中国人民解放军96901部队24分队 | Robot reloading automatic positioning method based on binocular vision |
CN115072357A (en) * | 2021-03-15 | 2022-09-20 | 中国人民解放军96901部队24分队 | Robot reprint automatic positioning method based on binocular vision |
CN113221953A (en) * | 2021-04-14 | 2021-08-06 | 上海交通大学宁波人工智能研究院 | Target attitude identification system and method based on example segmentation and binocular depth estimation |
CN113219926A (en) * | 2021-05-13 | 2021-08-06 | 中国计量大学 | Human-machine co-fusion manufacturing unit safety risk assessment method based on digital twin system |
CN113205561A (en) * | 2021-05-20 | 2021-08-03 | 清华大学 | Rotating shaft pose detection method and device based on vision |
CN113297221A (en) * | 2021-05-25 | 2021-08-24 | 山东万国云大数据科技有限公司 | Data center monitoring system based on three-dimensional model |
CN113547520A (en) * | 2021-07-21 | 2021-10-26 | 广东拓斯达科技股份有限公司 | Manipulator movement alignment method, device and system and storage medium |
CN113602799A (en) * | 2021-08-05 | 2021-11-05 | 西南科技大学 | A kind of airport luggage handling system and control method thereof |
CN113706628A (en) * | 2021-08-17 | 2021-11-26 | 成都信息工程大学 | Intelligent transfer robot cooperation system and method for processing characteristic image by using same |
CN113716323A (en) * | 2021-08-17 | 2021-11-30 | 成都新气象科技实业股份有限公司 | Intelligent container carrying method |
CN113689509A (en) * | 2021-09-15 | 2021-11-23 | 集美大学 | Binocular vision-based disordered grabbing method and system and storage medium |
CN113822945A (en) * | 2021-09-28 | 2021-12-21 | 天津朗硕机器人科技有限公司 | Workpiece identification and positioning method based on binocular vision |
CN113815323A (en) * | 2021-09-30 | 2021-12-21 | 河南埃尔森智能科技有限公司 | Rail marking device and marking method based on three-dimensional vision guidance |
CN113954072B (en) * | 2021-11-05 | 2024-05-28 | 中国矿业大学 | Visual-guided intelligent wood door workpiece recognition and positioning system and method |
CN113954072A (en) * | 2021-11-05 | 2022-01-21 | 中国矿业大学 | A vision-guided intelligent identification and positioning system and method for wooden door workpieces |
CN113977637A (en) * | 2021-12-03 | 2022-01-28 | 深圳市超准视觉科技有限公司 | Robot vision identification grabbing system and method applicable to non-precision work bin |
CN114368153A (en) * | 2021-12-13 | 2022-04-19 | 赫比(上海)家用电器产品有限公司 | Injection molding adhesive assembling equipment and assembling method thereof |
CN115115931A (en) * | 2022-01-06 | 2022-09-27 | 华中科技大学无锡研究院 | Rapid workpiece positioning method for robot machining system |
CN114408532A (en) * | 2022-02-25 | 2022-04-29 | 宝钢湛江钢铁有限公司 | Method, device and system for adjusting code bits of impact sample blank |
CN114838659B (en) * | 2022-04-26 | 2024-04-12 | 深圳市商汤科技有限公司 | Manipulator testing device, testing method, calibration method and storage medium |
CN114838659A (en) * | 2022-04-26 | 2022-08-02 | 深圳市商汤科技有限公司 | Manipulator testing device, testing method, calibration method and storage medium |
CN114852624B (en) * | 2022-05-18 | 2024-04-30 | 珠海格力智能装备有限公司 | Automatic wire feeding device and production line thereof |
CN114852624A (en) * | 2022-05-18 | 2022-08-05 | 珠海格力智能装备有限公司 | Automatic feeding device and production line thereof |
CN115214135A (en) * | 2022-05-30 | 2022-10-21 | 武汉新威奇科技有限公司 | Full-automatic feeding system and method for forging line based on 3D vision |
CN115106312A (en) * | 2022-06-13 | 2022-09-27 | 中实洛阳重型机械有限公司 | Intelligent ore sorting device based on binocular camera laser guide |
CN116216272A (en) * | 2023-03-15 | 2023-06-06 | 沃尔林自动化(苏州)有限公司 | Automatic production line for mechanical parts |
CN116216272B (en) * | 2023-03-15 | 2024-03-19 | 沃尔林自动化(苏州)有限公司 | Automatic production line for mechanical parts |
CN116330285A (en) * | 2023-03-20 | 2023-06-27 | 深圳市功夫机器人有限公司 | Mechanical arm control method and device, mechanical arm and storage medium |
CN116985182A (en) * | 2023-05-31 | 2023-11-03 | 齐鲁工业大学(山东省科学院) | Four-axis robot work piece wobble plate system based on machine vision |
CN118062559A (en) * | 2024-02-28 | 2024-05-24 | 石家庄优创科技股份有限公司 | Card finishing device with manipulator |
CN118062559B (en) * | 2024-02-28 | 2024-12-13 | 石家庄优创科技股份有限公司 | Card sorting device with a mechanical arm |
Also Published As
Publication number | Publication date |
---|---|
CN110509281B (en) | 2024-10-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110509281A (en) | Device and method for pose recognition and grasping based on binocular vision | |
CN104369188B (en) | Based on workpiece gripper device and the method for machine vision and ultrasonic sensor | |
CN104842362B (en) | A kind of method of robot crawl material bag and robotic gripping device | |
CN112132889A (en) | Soft magnet posture recognition and automatic grabbing method based on binocular vision | |
CN109230580B (en) | Unstacking robot system and unstacking robot method based on mixed material information acquisition | |
CN107618030B (en) | Robot dynamic tracking grabbing method and system based on vision | |
CN110014426B (en) | Method for grabbing symmetrically-shaped workpieces at high precision by using low-precision depth camera | |
CN108177143B (en) | Robot positioning and grabbing method and system based on laser vision guidance | |
CN102073303B (en) | Method for controlling feeding and discharging of mobile robot serving two numerically controlled machines | |
CN111791239A (en) | Method for realizing accurate grabbing by combining three-dimensional visual recognition | |
CN210757745U (en) | A device for binocular visual pose acquisition and grasping | |
CN107053173A (en) | The method of robot grasping system and grabbing workpiece | |
CN108080289A (en) | Robot sorting system, robot sorting control method and device | |
WO2017015898A1 (en) | Control system for robotic unstacking equipment and method for controlling robotic unstacking | |
CN105666485B (en) | A kind of automatic identification based on image procossing and positioning Bai Qi robots | |
CN203031254U (en) | Automatic screw-locking device and system | |
CN106965180A (en) | The mechanical arm grabbing device and method of bottle on streamline | |
CN103895042A (en) | Industrial robot workpiece positioning grabbing method and system based on visual guidance | |
CN112010024B (en) | Automatic container grabbing method and system based on laser and vision fusion detection | |
CN106583268A (en) | PE bottle inspecting and sorting system based on machine vision | |
CN113911728A (en) | Electric toothbrush brush head dynamic feeding system and feeding method based on vision | |
CN109623815B (en) | A wave compensation dual robot system and method for unmanned salvage ships | |
CN108480239A (en) | Workpiece quick sorting method based on stereoscopic vision and device | |
CN109625922A (en) | A kind of automatic loading and unloading system and method for intelligence manufacture | |
CN109278021A (en) | It is a kind of for grabbing the robot tool system of thin-wall case class workpiece |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
CB03 | Change of inventor or designer information | ||
CB03 | Change of inventor or designer information |
Inventor after: Wu Suyang Inventor after: Sun Jian Inventor before: Sun Jian Inventor before: Wu Suyang |
|
GR01 | Patent grant | ||
GR01 | Patent grant |