JP2021024056A - Calibration device - Google Patents

Abstract

To calibrate various parameters that define state of a robot and a sensor to appropriate values.SOLUTION: A calibration device comprises: a sensor data acquisition unit that generates sensor data based on output of sensors; a robot control unit that controls a robot; a data storage unit that stores sensor parameters related to the sensors and robot parameters related to the robot; a virtual data generation unit that generates virtual data in which sensors in a virtual space measures the robot based on the sensor parameters and the robot parameters; an environment determination unit that compares the sensor data with the virtual data so as to determine the difference between the two; a relative position/orientation prospect unit that searches for sensor parameters or robot parameters that reduce the difference; and an update unit that updates the sensor parameters or robot parameters stored in the data storage unit to the sensor parameters or the robot parameters searched by the relative position/orientation prospect unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a calibration device that updates various parameters indicating states such as the position and posture of a robot or a sensor to more appropriate values.

As a conventional calibration device, the one described in Patent Document 1 is known. In the abstract of this document, a solution for "for the calibration of a single camera or a stereo camera, it is possible to set a calibration range in advance in the image coordinate system and calibrate in an arbitrary range". As "a target mark is set on the robot, and the robot is moved to detect the target mark at a plurality of places in the field of view of the camera, so that the robot coordinate system of the robot and the image coordinate system of the camera are associated with each other. An image range setting unit that sets the image range in the image coordinate system of the camera, which is a calibration device, and the operation of the robot corresponding to the image range by moving the robot to detect the target mark before executing the calibration. A calibration device including a calibration range measuring unit for measuring a range is disclosed.

That is, in the calibration device of Patent Document 1, the visual sensor (monocular camera, stereo camera) is calibrated by continuously moving the arm tip of the robot and detecting the target mark attached to the arm tip at a plurality of places. Is going.

Japanese Unexamined Patent Publication No. 2018-11116

However, in the articulated robot arm used in Patent Document 1, manufacturing errors and control errors of each joint are accumulated and appear at the target mark at the tip of the arm, so that the actual position of the target mark deviates from the theoretical position without error. Even if the visual sensor was calibrated based on the actual position of the target mark, there was a high possibility that the calibration was not appropriate.

Therefore, the present invention provides a calibration device capable of updating various parameters of a robot or a sensor to more appropriate values without using an unstable reference such as a target mark attached to the tip of the arm of the robot. The purpose.

In order to solve the above problems, the calibration device of the present invention relates to a sensor data acquisition unit that generates sensor data based on the output of the sensor, a robot control unit that controls the robot, sensor parameters related to the sensor, and the robot. A data storage unit that stores robot parameters, a virtual data generation unit that generates virtual data that measures the robot from the sensor in the virtual space based on the sensor parameters and the robot parameters, and the sensor data and the virtual An environment determination unit that compares data and determines the difference between the two, a relative position / orientation estimation unit that searches for the sensor parameter or the robot parameter that reduces the difference, and the sensor parameter or the sensor parameter stored in the data storage unit. It is assumed that the sensor parameter searched by the relative position / orientation estimation unit or the update unit for updating the robot parameter to the robot parameter is provided.

According to the calibration device of the present invention, various parameters of the robot and the sensor can be updated to more appropriate values without using an unstable reference such as a target mark attached to the tip of the arm of the robot.

Functional block diagram of the calibration device of the first embodiment Coordinate space representation diagram of the robot and the sensor of the first embodiment Processing flowchart of the virtual data generation unit of the first embodiment Explanatory drawing of matching process of virtual image data and sensor data Processing flowchart of the imaging environment judgment unit and the relative position / orientation estimation unit An example of a robot's attitude change when attitude information is changed Data representation of parameter updates and differences Example of output screen for updating parameters of Example 2

Hereinafter, examples of the sensor position / orientation calibration device of the present invention will be described with reference to the drawings.

The calibration device 1 according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 7.

FIG. 1 is a diagram showing a functional block of the calibration device 1 of this embodiment together with a schematic arrangement of a robot 2, a sensor 3, and a target mark 4. In the following, after the robot 2, the sensor 3, and the target mark 4 are outlined, the details of the calibration device 1 will be described.

<Robot 2, sensor 3, target mark 4>
The robot 2 is an articulated robot arm that grips an object with a robot hand at the tip of the arm, and in response to a command from the robot control unit 1a included in the calibration device 1, the rotation angle θ of each joint and the open / closed state of the robot hand Etc. are controlled. This embodiment is applicable not only to robot hands but also to various end effectors such as suction hands. In the following, the robot 2 having three joints will be described as an example, but it goes without saying that the application target of the present invention is not limited to the robot 2 having three joints.

The sensor 3 is a camera, a laser displacement meter, or the like that observes the robot 2 in front of the target mark 4 whose installation position is known. In the following, the sensor 3 will be described as a camera.

The target mark 4 is a plurality of marks arranged on a plane. When the relative position and the relative posture of the sensor 3 and the target mark 4 are different, the relative position and the relative posture of the sensor 3 and the target mark 4 are detected by utilizing the difference in the appearance of the target mark 4 observed by the sensor 3. can do. The plurality of marks arranged on the target mark 4 may be individually equivalent or may be unique ones that can be individually identified.

Here, the coordinate system related to this embodiment will be described with reference to FIG. In the figure, W is a world coordinate system, which is a reference system for specifying the position of the robot 2, the sensor 3, the target mark 4, or the object held by the robot 2. R is a robot coordinate system with the fixed point (for example, the base) of the robot 2 as the origin, and is a coordinate system that moves in parallel while maintaining each axial direction of the world coordinate system W. S is a sensor coordinate system with the fixed point of the sensor 3 (for example, the light receiving element of the camera) as the origin, the line-of-sight direction of the sensor 3 is the Sz axis, and the Sx axis and the Sy axis are set so as to be orthogonal to the Sz axis. It is a coordinate system.

Further, C _RS is a coefficient for mutually converting the coordinates on the sensor coordinate system S and the coordinates on the robot coordinate system R. By using this coefficient C _RS , the position and shape of the object on the sensor coordinate system S detected by the sensor 3 can be recognized as the position and shape of the object on the robot coordinate system R, so that the robot control unit 1a can recognize the position and shape of the object. Using the coordinates on the robot coordinate system R, it is possible to instruct the robot 2 to grasp a desired object or the like.

When performing coordinate conversion using such a coefficient C _RS , if the coefficient C _RS is appropriate, the detection result of the sensor 3 can be accurately transmitted to the robot 2, but if the coefficient C _RS is inappropriate. For example, the detection result of the sensor 3 cannot be accurately transmitted to the robot 2, and there is a possibility that the robot 2 may not be able to correctly grasp the desired object.

Accuracy of the coefficient C _RS, in order to rely on sensor parameter P _S and the robot parameters P _R to be described later, if optimization of these parameters, the coefficient C _RS is also naturally optimized. Therefore, the calibration apparatus 1 of this embodiment, the sensor parameters P _S and the robot parameters P _R to be to be able to automatically optimize easily optimize the coefficients C _RS regardless operator proficiency I made it possible.

<Calibration device 1>
Here, returning to FIG. 1, the detailed structure of the calibration device 1 required for optimizing _{the coefficient CRS will be described.} The calibration device 1 is actually a computer such as a personal computer equipped with an arithmetic unit such as a CPU, a main storage device such as a semiconductor memory, an auxiliary storage device such as a hard disk, and hardware such as a communication device. .. Then, each function shown in FIG. 1 is realized by the arithmetic unit executing the program loaded in the main storage device while referring to the database recorded in the auxiliary storage device. Details of each function shown here will be described later, but in the following, well-known techniques in the computer field will be omitted as appropriate.

As shown in FIG. 1, the calibration device 1 of this embodiment includes a robot control unit 1a, a data storage unit 1b, a virtual imaging data generation unit 1c, a sensor data acquisition unit 1d, an imaging environment determination unit 1e, and a relative position / orientation estimation. A unit 1f, a robot parameter updating unit 1g, and a sensor parameter updating unit 1h are provided.

The robot control unit 1a communicates with the robot 2 in both directions to control the posture and behavior of the robot 2. For example, the robot control unit 1a controls the posture of the robot 2 by designating _{the rotation angles θ 1 to θ 3 of each joint of the robot 2.} Hereinafter, referred to as the combination of the rotation angle theta ₁ through? ₃ transmitted from the robot control unit 1a to the robot 2 and the orientation information I _S.

Data storage unit 1b is used to generate a virtual image pickup data D _V in the virtual data generation unit 1c to be described later, the environment information I _E, sensor parameters P _S, and stores the robot parameters P _R. The environmental information _IE is, for example, information such as the installation positions of the robot 2, the sensor 3, and the target mark 4 in the world coordinate system W. Sensor parameters P _S, for example, when the sensor 3 is a camera, the focal length (fx, fy), and internal parameters such as distortion coefficients, the optical axis center (cu, cv), the translation to the world coordinate system of the sensor It is an external parameter indicating movement (Tx, Ty, Tz) and rotation (3x3 matrix of r11 to r33). Robot parameter P _R is, for example, a parameter of the opening and closing state of and the robot hand (the rotation angle θ of each joint) each arm of the posture of the robot 2.

Virtual imaging data generation unit 1c is environmental information _{I E} obtained from the data storage unit 1b, the sensor parameter _{P S,} based on the robot parameter _{P R,} to generate the virtual image pickup data _{D V.} The generation processing of the virtual image pickup data D _V of now be described with reference to the flowchart of FIG.

First, in step S31, the virtual imaging data generation unit 1c acquires the environment information _{I E} from the data storage unit 1b, based on it, to place the robot _{2 V,} sensor _{3 V,} the target mark _{4 V} in the virtual space .. Next, in step S32, the virtual imaging data generation unit 1c on the basis of the robot parameters P _R obtained from the data storage unit 1b, the opening and closing state of the rotation angle θ and the robot hand of each joint in the robot 2 _V in the virtual space To set. Next, in step S33, the virtual imaging data generation unit 1c on the basis of the sensor parameter P _S acquired from the data storage unit 1b, set the internal parameters to the sensor 3 _V in the virtual space _{(camera). As a result, the robot 2 V} , the sensor 3 _V , and the target mark 4 _V , as illustrated in the virtual space data display area 5b of FIG. 8 to be described later, are arranged in the virtual space.

Finally, in step S34, the virtual imaging data generation unit 1c is a robot 2 _V and the target mark 4 _V in the virtual space, the virtual image pickup data D _V, which corresponds to that taken from the perspective of the sensor 3 _V in the virtual space To generate.

Any point on the virtual space to projection (x, y, z) of the virtual image pickup data D _V, for example, utilizing Equation 1. In Expression 1, u, v are coordinates in uv coordinate system virtually set on the virtual image pickup data D _V (virtual coordinate system). fx and fy are focal lengths per pixel of the camera, and are values obtained by dividing the focal length of the camera by the physical spacing between the pixels of the camera in the vertical and horizontal directions. cu, cv is the virtual image pickup data D _V to the optical axis of the camera intersects the coordinates of the optical axis in the virtual coordinate system.

Sensor data acquisition unit 1d has a real robot 2 and the target mark 4, based on the observed from the perspective of the actual sensor 3 output, acquires sensor data D _S. In this embodiment, since the sensor 3 is a camera, sensor data D _S to be obtained here it is image data.

In the imaging environment judging unit 1e, it compares the virtual imaging data D _V output by the virtual imaging data generation unit 1c, and the sensor data D _S to the sensor data acquisition unit 1d outputs, determines imaging environment. The details of the processing here will be described with reference to FIGS. 4 and 5.

Figure 4 is an explanatory diagram of the matching process of the virtual image pickup data D _V and the sensor data D _S, (a), the virtual generated by the virtual imaging data generation unit 1c based on the stored in the data storage unit 1b parameters imaging data _D V, (b), the sensor data _D S, which is generated on the basis of the actual output of the sensor 3 (c) is a process of matching the virtual image pickup data _{D V} and the sensor data _{D S.}

As it is evident from the virtual image pickup data D _V and the sensor data D _S is greatly different, 4, parameters stored in the data storage unit 1b has deviated the actual situation that requires the modification Is equivalent to.

To optimize the parameters may be selected parameters that can generate a virtual image pickup data D _V, which coincides with the sensor data D _S. Specifically, as shown in (c) matching process may be performed the same parameter adjustment when subjected to moving processing and rotation processing necessary virtual imaging data D _V to match the sensor data D _S.

FIG. 5 is a flowchart showing an example of the imaging environment determination process executed by the imaging environment determination unit 1e.

First, in step S51, the imaging environment judging section 1e obtains the sensor data _{D S} and the virtual image pickup data _{D V.} Next, in step S52, the imaging environment determination unit 1e measures the reference marker position of the target mark 4 included in both data, and in step S53, the imaging environment determination unit 1e matches the corresponding points between the two data. ..

By the processing of S53 from step S51 described above, the imaging environment judging unit 1e, since it calculates the movement amount and the rotation amount required virtual imaging data D _V to match the sensor data D _S, at step S54, the imaging environment The determination unit 1e determines whether these rotation amounts or movement amounts are equal to or greater than a predetermined threshold value. Then, when the amount of rotation and the amount of movement required for parameter calibration are less than the threshold value, the imaging environment determination unit 1e determines that the currently stored parameters are valid according to the actual situation, and the data storage unit 1b The processing of this flowchart is terminated without updating the parameters of.

On the other hand, when the rotational amount or the amount of movement required to match the virtual imaging data D _V sensor data D _S is not less than the threshold value, the imaging environment judging unit 1e includes parameters currently stored in the data storage portion 1b It is judged that the correction is necessary because it does not conform to the actual situation, and the process shifts to the relative position / orientation estimation process for parameter correction.

After shifting to the relative position and orientation estimation processing, first, in step S55, the relative position and orientation estimation unit 1f searches the sensor parameters P _S that minimizes the difference between the target mark the fourth virtual imaging data D _V and the sensor data D _S .. For example, a sensor parameters _{P S is,} if four types of _{P S1, P S2, P S3} , P S4, in order from the sensor parameter _{P S1,} will seek the optimum value that minimizes the difference between the target mark 4. Next, in step S56, the relative position and orientation estimation unit 1f searches the robot parameters P _R that minimizes the difference of the robot 2 of the virtual image pickup data D _V and the sensor data D _S. For example, the robot parameters _{P R is,} if four types of _{P R1, P R2, P R3} , P R4, in order from the robot parameters _{P R1,} to seek the optimum value that minimizes the difference of the robot 2.

Then, in step S57, the robot parameter updating unit 1g or the sensor parameter updating unit 1h updates each parameter stored in the data storage unit 1b to the optimum value of each parameter obtained in steps S55 and S56. In FIG. 5, after the optimum value search of the sensor parameter P _S, were subjected to optimum value search of the robot parameters P _R, the difference of the robot 2 side when comparing virtual imaging data D _V and the sensor data D _S If it is greater than the difference in target mark 4 side, after searching the optimum value of the robot parameters P _R, may be searched optimum value of the sensor parameter P _S.

Next, a method of evaluating the posture accuracy of the robot 2 will be described with reference to FIGS. 6 and 7.

When evaluating the posture accuracy of the robot 2, first, the robot control unit 1a transmits arbitrary posture information _IS (θ ₁ , θ ₂ , θ ₃ ) to the robot 2. At this time, since the robot 2 has the posture shown in FIG. _6A , the error amount d corresponding to the posture information IS is measured and recorded.

Next, the robot control unit 1a is sent 'was changed to the posture information _{I S'} only rotation angle theta ₁ of the posture information _{I S} rotational angle _{θ 1 (θ 1 ', θ} 2, θ 3) of the robot 2 To do. In this case, the robot 2, since the posture shown in FIG. 6 (b), measured _'error amount d corresponding to' orientation information I _S is recorded.

If the error amount d and the error amount d'are different and the error amount d'is large, _{the indirect posture accuracy corresponding to the rotation angle θ 1} is poor, or the accuracy of the link on the tip side of the indirect is poor. It can be judged to be bad.

By executing the evaluation work attitude accuracy as described above for a number of orientation information I _S, as illustrated in FIG. 7, it is possible to create a correspondence table of orientation information I _S and the error amount d. Then, the relationship between the orientation information I _S and the error amount d shown here, the elements of multiple regression analysis and attitude information increases or decreases the error amount d by a general analysis method such as logistic regression analysis I _S or attitude information I, it is possible to obtain the combination of _S elements, it is possible to identify it with optimal robot parameters P _R.

Then, 5 and on the basis of the sensor parameters P _S and the robot parameters P _R that has been modified with reference to FIG calculates the coefficients C _RS, coordinates of the sensor coordinate system S · robot coordinate system R using the coefficients C _RS By performing the conversion, the position and shape of an object in the vicinity of the robot 2 can be correctly recognized, so that the robot 2 can appropriately grasp the object.

As explained above, according to the calibration apparatus of the present embodiment, without using unstable criteria, such as target marks attached to the arm tip of the robot, Ya robot parameter P _R that define the state of the robot , possible to calibrate the sensor parameters P _S which defines the state of the sensor to a more appropriate value. Then, by using these parameters, the control of the robot becomes more accurate, and for example, the robot hand can grasp the object more accurately.

Next, the calibration device 1 according to the second embodiment of the present invention will be described with reference to FIG. It should be noted that the common points with the first embodiment will be omitted.

In the first embodiment, the calibration apparatus 1 is automatically updates the sensor parameters P _S and the robot parameters P _R, in the present embodiment, the operator for processing by the relative position and orientation estimation unit 1f has to be involved ..

FIG. 8 is an example of the display screen 5 of the application software of the calibration tool displayed on the display device connected to the calibration device 1.

As shown here, the image pickup data display area 5a, the virtual space data display area 5b, the sensor parameter setting area 5c, the robot parameter setting area 5d, the decision button 5e, and the pointer 5f are displayed on the display screen 5. ..

The imaging data display area 5a, the sensor data D _S which sensor 3 has actually acquired, the current sensor parameter P _S, virtual imaging data D _V generated by the virtual imaging data generation unit 1c using a robot parameters P _R Are displayed side by side. Incidentally, if both differences are inadequate current sensor parameters P _S or robot parameter P _R, If they match, that the current sensor parameters P _S and the robot parameters P _R is appropriate examples It is the same as 1.

_{The arrangement of the robot 2 V} , the sensor 3 _V , and the target mark 4 _{V in} the virtual space is displayed in the virtual space data display area 5b, and various types displayed in the sensor parameter setting area 5c and the robot parameter setting area 5d. by adjusting the parameters, the virtual imaging data D _V to be displayed on the imaging data display area 5a, the state of the robot 2 _V to be displayed on the virtual space data display area 5b changes.

At such a display screen 5, the operator operates the pointer 5f, when each parameter of the sensor parameter setting region 5c and the robot parameters setting area 5d appropriately setting, the sensor data D _S and the virtual image pickup data D _V Are almost the same. At this time, when the operator presses the decision button 5e, the robot parameter update unit 1g and the sensor parameter update unit 1h update each parameter stored in the data storage unit 1b based on the displayed parameter values.

According to the configuration of the second embodiment described above, not only the same effect as that of the first embodiment can be obtained, but also the operator can easily grasp which parameter has a problem by an intuitive operation. be able to.

1 ... Calibration device,
1a ... Robot control unit,
1b ... Data storage unit,
1c ... Virtual imaging data generator,
1d ... Sensor data acquisition unit,
1e ... Imaging environment judgment unit,
1f ... Relative position / orientation estimation unit,
1g ... Robot parameter update unit,
1h ... Sensor parameter update unit,
2, 2 _V ... Robot,
3, 3 _V ... sensor,
4, 4 _V ... Target mark,
5 ... Display screen,
5a ... Imaging data display area 5a,
5b ... Virtual space data display area 5c ... Sensor parameter setting area 5d ... Robot parameter setting area 5e ... Enter button 5f ... Pointer

Claims (6)

A sensor data acquisition unit that generates sensor data based on the output of the sensor,
A robot control unit that controls the robot and
A data storage unit that stores sensor parameters related to the sensor and robot parameters related to the robot, and
A virtual data generation unit that generates virtual data obtained by measuring the robot from the sensor in the virtual space based on the sensor parameter and the robot parameter.
An environment determination unit that compares the sensor data and the virtual data and determines the difference between the two.
A relative position / orientation estimation unit that searches for the sensor parameter or the robot parameter that reduces the difference, and
An update unit that updates the sensor parameter or the robot parameter stored in the data storage unit to the sensor parameter or the robot parameter searched by the relative position / orientation estimation unit, and an update unit.
A calibration device characterized by comprising. In the calibration apparatus according to claim 1,
The environment determination unit obtains the amount of rotation or movement required to match the virtual data with the sensor data.
When the rotation amount or the movement amount is equal to or more than the threshold value, the relative position / orientation estimation unit searches for the sensor parameter or the robot parameter that reduces the difference, and the rotation amount or the movement amount is less than the threshold value. In some cases, a calibration device that does not search for the sensor parameter or the robot parameter that reduces the difference. In the calibration apparatus according to claim 1 or 2.
The data storage unit further stores environmental information regarding the positions of the sensor and the robot.
The virtual data generation unit is a calibration device that generates the virtual data by observing the robot from the sensor in the virtual space in which the sensor and the robot are arranged based on the environmental information. In the calibration apparatus according to claim 1 or 2.
The robot is an articulated robot arm.
The robot parameter is a calibration device characterized in that the rotation angle of each joint of the articulated robot arm or the operating state of an end effector mounted on the robot. In the calibration apparatus according to claim 1 or 2.
The sensor is a camera
A calibration device characterized in that the sensor parameter is the position / orientation, focal length, distortion coefficient, or optical axis center of the camera. In the calibration apparatus according to claim 1,
A calibration device characterized in that the sensor parameter or the robot parameter input by an operator is used for the search by the relative position / orientation estimation unit.

Images