CN105976380A - Vision based calibration method for spraying track of robot - Google Patents

Abstract

The invention belongs to the field of robot spraying, and discloses a vision-based robot spraying trajectory calibration method, including: calibration of the internal parameters of the real camera and hand-eye calibration between the spraying robot and the real camera; using the real camera to obtain the calibration The reference image when the workpiece is in an ideal position; using the real camera to obtain the target image of the workpiece to be sprayed; setting the virtual camera and obtaining the position and attitude information of the real camera under the coordinate system of the virtual camera; the workpiece to be sprayed Calibration of the spray trajectory. The robot spraying trajectory calibration method of the present invention uses a visual method to calibrate the pose of the workpiece to be sprayed, and then calibrates the spraying trajectory, without the need for strict positioning of the workpiece to be sprayed and the design of specific fixtures, which reduces costs and simplifies the calibration process.

Description

A vision-based method for robot spraying trajectory calibration

technical field

The invention belongs to the field related to industrial robot spraying, and more specifically relates to a vision-based method for calibrating robot spraying tracks. The robot spraying trajectory calibration method uses a visual method to calibrate the position and posture of the workpiece to be sprayed, and then calibrates the spraying trajectory, which does not require strict positioning of the workpiece to be sprayed, nor does it need to design specific fixtures, which reduces costs and simplifies the spraying trajectory. Calibration process.

Background technique

With the rapid development of the spraying industry, higher requirements are put forward for the spraying quality. The original manual spraying method is inefficient, the spraying quality cannot be guaranteed, and it will cause obvious harm to the human body, which is no longer suitable for the current development needs. The emergence of spraying robots has replaced the original manual operation, which greatly improves the work efficiency and also improves the spraying quality.

The spraying trajectory required by the spraying robot is generally directly generated by manual teaching or based on the ideal model of the workpiece. The spraying robot performs the spraying operation according to the generated ideal trajectory. In order to ensure the quality of the spraying, it is required that the workpiece to be sprayed must be placed in an ideal position. Spraying workpieces are strictly positioned. The positioning process of the workpiece to be sprayed is cumbersome and requires the design of specific fixtures, which has a long cycle and high cost. Correspondingly, there is an urgent need in this field to further optimize the robot spraying trajectory calibration scheme in order to meet the higher requirements for spraying quality in robot spraying.

Contents of the invention

Aiming at the above defects or improvement needs of the prior art, the present invention provides a vision-based robot spraying trajectory calibration method, which combines the spraying conditions and adopts a visual method to treat the sprayed workpiece when the workpiece to be sprayed does not need strict positioning Calibrate the pose and calibrate the spraying trajectory. The robot spraying trajectory calibration method does not require strict positioning of the workpiece to be sprayed, nor does it need to design specific fixtures, which reduces production costs and improves spraying quality.

To achieve the above object, the invention provides a vision-based robot spraying trajectory calibration method, which includes the following steps:

(a) hand-eye calibration between the calibration of the internal parameters of the real camera and the spraying robot and the real camera;

(b) setting the calibration workpiece provided with the first feature points at two intervals at an ideal position; measuring the relative position of the two first feature points in the world coordinate system; using the real camera to shoot the calibration An image of the workpiece at the ideal position, using this image as a reference image;

(c) placing the workpiece to be sprayed at any position in the field of view of the real camera, the structure of the workpiece to be sprayed is the same as that of the calibration workpiece, and there are two positions respectively corresponding to the two first A second feature point corresponding to the position of the feature point; the real camera shoots the workpiece to be sprayed to obtain a target image;

(d) a virtual camera is set adjacent to the position of the workpiece to be sprayed, the image of the workpiece to be sprayed obtained by the virtual camera is the same as the reference image obtained in step (b); the image matching algorithm is used to calculate the The essential matrix E between the reference image and the target image obtained in step (c); the rotation matrix R between the virtual camera and the real camera is obtained by decomposing the essential matrix E, based on the first feature The position of the point in the reference image, calculating the coordinates of the first feature point in the virtual camera coordinate system, and then obtaining the translation vector t of the real camera relative to the virtual camera;

(e) Calibrating the spraying track of the workpiece to be sprayed.

Further, the internal parameters of the real camera are the same as the internal parameters of the virtual camera.

Further, the image matching algorithm is one of the following methods: SURF algorithm, SIFT algorithm

Further, the essential matrix E is preferably obtained by the following formula:

E＝K ^T FK

Where K is the internal parameter matrix of the real camera, and F is the fundamental matrix between the reference image and the target image.

Further, the decomposition process of the essential matrix E is based on singular value decomposition, and according to the result of singular value decomposition E=UDV ^T , the two values R _a and R _b of the rotation matrix R can be calculated by the following formula:

R _a = UWV ^T , R _b = UW ^T V ^T

Wherein matrix U and matrix V are the decomposition result of described singular value decomposition E=UDV ^T ,

Further, the translation vector t can be calculated by the following formula:

t _a1 ＝K ^-1 (KR _a X ₁ -λ′ _a1 x ₁ ′)

t _a2 ＝K ^-1 (KR _a X ₂ -λ′ _a2 x′ ₂ )

or

t _b1 ＝K ^-1 (KR _b X ₁ -λ′ _b1 x ₁ ′)

t _b2 ＝K ^-1 (KR _b X ₂ -λ′ _b2 x′ ₂ )

Wherein x ₁ ' and x' ₂ are the pixel coordinates of the two second feature points on the workpiece to be sprayed in the target image, K is the internal parameter matrix of the real camera, λ' _a1 , λ' _a2 , λ' _b1 , λ' _b2 , X ₁ and X ₂ are preferably obtained by the relative positions Δ, R _a and R _b of the two first feature points in the world coordinate system.

Further, the following formula is preferably used for calibrating the spray trajectory of the workpiece to be sprayed:

Where P′ is the target spraying trajectory point, R _w and t _w are the results of the hand-eye calibration, and P is the ideal spraying trajectory point.

Further, the ideal spraying trajectory points are generated when the calibration workpiece is located at the ideal position, and the ideal spraying trajectory points are described in the world coordinate system.

Generally speaking, compared with the prior art, the above technical solution conceived by the present invention adopts the vision-based robot spraying trajectory calibration method of the present invention, which adopts the visual method to treat the workpiece under the condition that the workpiece to be sprayed does not need to be strictly positioned. Calibrate the pose of the sprayed workpiece and calibrate the spraying trajectory; neither strict positioning of the workpiece to be sprayed nor design of specific fixtures is required, which reduces production costs and improves spraying quality.

Description of drawings

Fig. 1 is a schematic diagram of a calibration workpiece in an ideal position in a vision-based robot spraying trajectory calibration method provided by a preferred embodiment of the present invention.

FIG. 2 is a schematic diagram of the workpiece to be sprayed in any position in the robot spraying trajectory calibration method in FIG. 1 .

In all the drawings, the same reference numerals are used to represent the same elements or structures, wherein: 1-spraying robot, 2-first feature point, 3-calibrated workpiece, 4-real camera, 5-virtual camera, 6 - the workpiece to be sprayed, 7 - the second feature point.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

Referring to Fig. 1 and Fig. 2, the vision-based robot spraying trajectory calibration method provided by the preferred embodiment of the present invention includes the following steps:

Step 1, calibration of internal parameters of the real camera 4 and hand-eye calibration between the painting robot 1 and the real camera 4 . The so-called real camera 4, that is, the camera used for actual shooting in reality

In this embodiment, the calibration method of the internal parameters of the real camera 4 adopts the Zhang Zhengyou calibration method in the traditional camera calibration method, that is, placing a black and white checkerboard in front of the real camera 4, fixing the real camera 4, changing the Describe the position of the checkerboard to obtain multiple images; and extract the corner points of the multiple images, use the relationship between the corner points and the internal parameters of the real camera 4 to list the matrix equation, and obtain by singular value decomposition Get the internal parameters of the real camera 4. It can be understood that in other implementation manners, other methods may be used to calibrate the internal parameters of the real camera 4 , such as an active vision camera calibration method, a camera self-calibration method, and the like.

Step 2, obtaining a reference image when the calibration workpiece 3 is in an ideal position. Specifically, the calibration work 3 is set at an ideal position, on which two first feature points 2 are arranged at intervals; measure the relative positions of the two first feature points 2 in the world coordinate system; use the The real camera 4 captures a reference image of the calibration workpiece 3 at the ideal position. It can be understood that, in other implementation manners, the number of the first feature points 2 may be changed according to actual needs, for example, the number of the first feature points 2 may be three.

Step 3, acquiring the target image of the workpiece 6 to be sprayed. Specifically, the workpiece 6 to be sprayed is placed at any position within the field of view of the real camera 4; the real camera 4 shoots the workpiece 6 to be sprayed to obtain a target image.

In this embodiment, the workpiece 6 to be sprayed and the calibration workpiece 3 are workpieces of the same type, and the workpiece 6 to be sprayed is provided with two second feature points 7, and the two second feature points 7 The positions on the workpiece 6 to be sprayed correspond to the positions of the two first feature points 2 on the calibration workpiece 3 respectively, that is, the two second feature points 7 correspond to the two first feature points 7 respectively. Point 2 corresponds one to one. It can be understood that, in other implementation manners, the number of the second feature points 7 may also be other numbers, such as 3, 4, and so on.

Step 4, setting the virtual camera 5 and obtaining the position and attitude information of the real camera 4 in the coordinate system of the virtual camera 5 . The so-called virtual camera 5 is a virtual imitation object that does not exist in reality. Specifically, at first, a virtual camera 5 is set at a position adjacent to the workpiece 6 to be sprayed, and the image of the workpiece 6 to be sprayed by the virtual camera 5 is the same as the workpiece 6 to be sprayed by the real camera 4. The image at the ideal position is the same, that is, the same as the reference image. In this embodiment, the internal parameters of the real camera 4 are the same as the internal parameters of the virtual camera 5; secondly, the image matching algorithm is used to calculate and reflect the An essential matrix E of the pose relationship between the reference image and the target image, the essential matrix E is preferably obtained by the following formula;

E＝K ^T FK

Wherein K is the internal parameter matrix of described real camera 4, and F is the basic matrix between described reference image and described target image, and described basic matrix F is calculated and obtained by image feature matching algorithm, and described image matching algorithm can be SURF algorithm, SIFT algorithm, etc.

Afterwards, the position and attitude information of the real camera 4 in the coordinate system of the virtual camera 5 is calculated based on the essential matrix E. Specifically, a rotation matrix reflecting the attitude relationship between the virtual camera 5 and the real camera 4 is obtained by decomposing the essential matrix E, and the decomposition process of the essential matrix E is based on a singular value decomposition method. The rotation matrix R is preferably calculated by the following formula to obtain two values R _a and R _b of the rotation matrix R:

R _a = UWV ^T , R _b = UW ^T V ^T

Wherein matrix U and matrix V represent the decomposition result to described singular value decomposition E=UDV ^T respectively,

Based on the position of the first feature point 2 in the reference image, calculate the coordinates of the first feature point 2 in the virtual camera 5 coordinate system, and then solve the relative relationship between the real camera 4 and the virtual camera 5 The translation vector t. The translation vector t is preferably calculated by the following formula:

t _a1 ＝K ^-1 (KR _a X ₁ -λ′ _a1 x ₁ ′)

t _a2 ＝K ^-1 (KR _a X ₂ -λ′ _a2 x′ ₂ )

or

t _b1 ＝K ^-1 (KR _b X ₁ -λ′ _b1 x ₁ ′)

t _b2 ＝K ^-1 (KR _b X ₂ -λ′ _b2 x′ ₂ )

Wherein x ₁ ' and x' ₂ are the pixel coordinates of the two second feature points 7 in the target image, K is the internal parameter matrix of the real camera 4, λ' _a1 , λ' _a2 , λ' The intermediate values of _b1 , λ′ _b2 , X ₁ and X ₂ are preferably solved through the relative positions Δ, R _a and R _b of the two first feature points 2 in the world coordinate system.

The algorithm for extracting the first feature point 2 and the second feature point 7 from the reference image and the target image may be a color recognition algorithm, a template matching algorithm, and the like.

Two sets of possible rotation matrices R and translation vector t obtained by the above method can be sieved by the following criteria:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&alpha;t</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}\\ {\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&equiv;</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&equiv;</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right.<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span>$

Wherein α is a non-zero constant, and t _n is the last column of the matrix U in the decomposition result of the singular value decomposition E=UDV ^T of the essential matrix.

Step 5, calibrate the spray trajectory of the workpiece 6 to be sprayed. Using the rotation matrix R and the translation vector t between the virtual camera 5 and the real camera 4, combined with the result of hand-eye calibration between the painting robot 1 and the real camera 4, the workpiece 6 to be sprayed 6 The pose is calibrated, and then the spraying trajectory is calibrated. For any ideal spraying trajectory point, it can be calibrated according to the following formula:

Where P′ is the target spraying trajectory point, R _w and t _w are the results of the hand-eye calibration, and P is the ideal spraying trajectory point. The ideal spraying trajectory points are described in the world coordinate system, and the ideal spraying trajectory points are generated when the calibration workpiece 3 is at the ideal position.

The vision-based robot spraying trajectory calibration method of the present invention adopts a visual method to calibrate the pose of the workpiece to be sprayed and calibrate the spraying trajectory under the condition that the workpiece to be sprayed does not need to be strictly positioned. The robot spraying trajectory calibration method does not require strict positioning of the workpiece to be sprayed, nor does it need to design specific fixtures, which reduces production costs and improves spraying quality.

It is easy for those skilled in the art to understand that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, All should be included within the protection scope of the present invention.

Claims (8)

1. a vision-based robot spraying trajectory calibration method, is characterized in that, the method comprises: (a) Calibrate the internal parameters of the real camera (4), and simultaneously perform hand-eye calibration between the spraying robot (1) and the real camera (4); (b) Place the calibration workpiece (3) in an ideal position, the calibration workpiece is selected to have the same structure as the workpiece (6) to be sprayed, and the calibration workpiece (3) is also provided with a plurality of first A feature point (2), then measure the relative position of each of the first feature points (2) in the world coordinate system; then, use the real camera (4) to shoot the calibration workpiece (3) located at the Describe the image under the ideal position, and use this image as a reference image; (c) the workpiece to be sprayed (6) is placed within the field of view of the real camera (4), and the workpiece to be sprayed (6) is provided with second feature points (7) and these second features Points (7) are in one-to-one correspondence with the first feature point (2) respectively; then, continue to adopt the real camera (4) to the workpiece (6) to be sprayed, thereby obtaining and reflecting the workpiece (6) to be sprayed. ) the target image of the real position; (d) a virtual camera (5) is set at a position adjacent to the workpiece (6) to be sprayed, and the image of the workpiece (6) to be sprayed acquired by the virtual camera (5) is consistent with the acquired image of the workpiece (6) in step (b) The above reference image remains the same; then, use the image matching method to calculate the essential matrix E reflecting the pose relationship between the reference image and the target image; in addition, obtain the reflection of the essential matrix E by decomposing the essential matrix E The rotation matrix R of the posture relationship between the virtual camera (5) and the real camera (4), thus obtaining the translation vector t of the real camera (4) relative to the virtual camera (5); (e) Calibrate the spraying trajectory of the workpiece (6) to be sprayed based on the rotation matrix R and translation vector t of the real camera (4) relative to the virtual camera (5) obtained in step (d). 2. The vision-based robot spraying trajectory calibration method according to claim 1, characterized in that: the internal parameters of the real camera (4) are identical to the internal parameters of the virtual camera (5). 3. The vision-based robot spraying trajectory calibration method according to claim 1, characterized in that: the image matching algorithm is one of the following methods: SURF algorithm, SIFT algorithm. 4. vision-based robot spraying trajectory calibration method as claimed in claim 1, is characterized in that: described essential matrix E is obtained by following formula: E＝K ^T FK Where K is the internal parameter matrix of the real camera (4), and F is the fundamental matrix between the reference image and the target image. 5. the robot spraying track calibration method based on vision as claimed in claim 4, is characterized in that: the decomposition process of described essential matrix E adopts singular value mode to decompose, and the result of this singular value decomposition adopts matrix U, D and V, and then calculate the two values R _a and R _b of the rotation matrix R by the following formula: R _a = UWV ^T , R _b = UW ^T V ^T Wherein matrix U and matrix V represent respectively the decomposition result to described singular value decomposition E=UDV ^T , 6. vision-based robot spraying trajectory calibration method as claimed in claim 5, is characterized in that: described translation vector t is calculated by following formula: t _a1 ＝K ^-1 (KR _a X ₁ -λ′ _a1 x ₁ ′) t _a2 ＝K ^-1 (KR _a X ₂ -λ′ _a2 x′ ₂ ) or t _b1 ＝K ^-1 (KR _b X ₁ -λ′ _b1 x ₁ ′) t _b2 ＝K ^-1 (KR _b X ₂ -λ′ _b2 x′ ₂ ) Wherein _x'1 and x'2 are the pixel coordinates of _two second feature points (7) in the target image on the workpiece (6) to be sprayed, and K is the internal parameter of the real camera (4) matrix, λ′ _a1 , λ′ _a2 , λ′ _b1 , λ′ _b2 , X ₁ and X ₂ respectively represent intermediate parameter values, which can be based on the relative position Δ of the first feature point (2) in the world coordinate system , R _a and R _b to solve. 7. the vision-based robot spraying track calibration method as claimed in claim 1, is characterized in that: the spraying track of workpiece (6) to be sprayed is calibrated and can adopt following formula to carry out: Where P′ is the target spraying trajectory point, R _w and t _w are the results of the hand-eye calibration, and P is the ideal spraying trajectory point. 8. The vision-based robot spraying trajectory calibration method as claimed in claim 7, characterized in that: the ideal spraying trajectory point is generated when the calibration workpiece (3) is located at the ideal position, and the The above ideal spraying trajectory points are described in the world coordinate system.