JPH09105608A - Measuring method for position of load - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a measuring method in which the position of a load can be measured in a simple configuration, with good accuracy and surely by measuring the height of a camera from the load and compositing an image which is imaged by illuminating the load from its oblique direction and a mask image. SOLUTION: From a palletizing computer 3, measured position data is transmitted to a robot control device 2, and the device 2 moves a hand at a robot 1 to a measuring position. The computer 3 which receives a movement completion signal from the device 2 turns on a laser marker 6 in order to measure the height position of a camera from a load. Image data which is picked up by a CCD camera 8 by applying slit light to the load in its oblique direction is processed by an image processing device 4, and a height position is computed. Then, the computer 3 turns on an illuminator, it takes in an image which is picked up by applying light to the load in its oblique direction, and it takes in an image obtained by turning off the illuminator. The device 4 computes a position ad the plane position of the load and the angle of rotation of the load.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of measuring the position of a load in handling a load such as a cardboard box loaded on a pallet for picking up and unloading the load.

[0002]

2. Description of the Related Art A handling device such as a robot is used in automation of unloading work for unloading a cargo such as a cardboard box loaded on a pallet from the pallet or reloading to another pallet. In general, the loads loaded on the pallets are of the same type and size and are loaded on the pallets in a prescribed arrangement, and the handling device stores in advance the loading position and loading direction of each load on the pallet. The information is used to control the operation of the handling device. However, the loads loaded on the pallet are higher than the specified position in the horizontal or horizontal direction due to misalignment due to the intermediate transfer process, bending of the pallet, and variations in the loading position when manually loading the pallet. Due to the translational deviation or horizontal rotation deviation, the handling device may fail to grab the luggage, or even if it can be grabbed, it cannot be loaded accurately when it is loaded on another pallet, which causes trouble when loading the next luggage. become.

Therefore, as a conventional method for correcting the grip position by measuring the position of the load immediately before gripping the load, Japanese Patent Application Laid-Open No. Hei 2
In Japanese Patent No. 310214, a handling robot for measuring the three-dimensional position of a package and taking it out from a pallet is shown by combining the measurement in the height direction by a height measurement sensor and the two-dimensional measurement in the horizontal direction by an image measuring device. Has been done.

Further, Japanese Patent Application Laid-Open No. 6-241719 describes an apparatus which images an article irradiated with slit light and measures the position of the article by using dimension information of the article.

[0005]

However, in the measuring method described in Japanese Patent Laid-Open No. 2-310214, the control is complicated because the height measurement and the two-dimensional measurement are performed by separate device configurations, and the image measurement method is used. Since the dimension measurement uses a method of measuring the contour of the package, pattern matching, or the like, there is a risk of erroneous detection due to the pattern on the top surface of the package, and the processing time is long because the calculation is complicated. In addition, the imaging device has a low measurement accuracy because it captures a wide field of view of the entire pallet, and although it is equipped with an illumination device for imaging, it is stable because the illuminance of the luggage changes throughout the day and night due to the effects of sunlight. However, it is difficult to apply it to a plastic container or the like whose surface is not flat because it is necessary to put a mark on the baggage.

Next, in the measuring method described in Japanese Patent Laid-Open No. 6-241719, only the horizontal position of the article is detected, so the position in the height direction of the article is deviated from the specified position due to the crushing of the article, the bending of the pallet, or the like. If there is a deviation, a horizontal measurement error occurs in proportion to the amount of deviation. Further, since the height position of the article cannot be measured, the handling device may collide with the article when taking out the load when the number of items loaded on the pallet is different from the regular number, and the item or the handling device may be damaged. Further, since many measurement points cannot be taken, there is a problem in accuracy when the box is deformed, and it is difficult to apply to a plastic container whose surface is not flat or a cylindrical object having a curved surface.

[0007] Therefore, the present invention provides a method capable of performing accurate and reliable measurement with a simple device configuration and measuring the position of a corrugated board, a plastic container, a cylindrical luggage, and the like.

[0008]

SUMMARY OF THE INVENTION The present invention irradiates a package with slit light from an oblique direction of the package, measures the height of the camera from the package by triangulation, and illuminates the package obliquely. The captured image and the mask image are combined, the disturbance element in the luggage position measurement area is removed, the contour of the luggage upper surface is extracted, and the plane position of the luggage is determined by the height position and the luggage dimension information obtained earlier. The feature is that the rotation angle is obtained.

[0009]

1 is a block diagram of a measuring system according to the present invention. In FIG. 1, the measuring system is composed of a robot 1, a robot controller 2, a palletizing computer 3, and an image processing device 4. Hand one 1
Is provided with a laser marker 6, an illumination 7, and a CCD camera 8, which are connected to a power source 10 or an image processing device 4 by a power cable and a signal cable 9. The luggage 11 is irradiated with the laser light and the illumination obliquely.

FIG. 2 is an overall processing flow of the system according to the present invention. In FIG. 2, the measurement position data is transmitted from the palletizing computer to the robot control device, and the robot control device is based on the received measurement position data. To move to the measurement position.

Upon receiving the movement completion signal from the robot controller, the palletizing computer turns on the laser marker in order to measure the height position of the camera from the luggage. Image data obtained by irradiating a package with slit light obliquely and picking up an image with a CCD camera is processed by an image processing device to calculate a height position.

Next, the palletizing computer turns on the illumination to measure the position / orientation of the box, illuminates the luggage obliquely and captures the imaged image, then turns off the illumination and turns off the illumination. The obtained image is captured, and the image processing apparatus based on the obtained image data,
The plane position of the package and the posture which is the rotation angle of the package in the horizontal plane are calculated.

The palletizing computer calculates the shift amount, which is the relative distance between the robot hand and the luggage, from each data of the height position, the plane position, and the posture, transmits the shift data to the robot controller, and uses the received shift amount as the shift amount. Based on this, the robot controller causes the robot hand to start the picking operation.

Next, a method of calculating the height position, the plane position and the posture will be described.

FIGS. 3 and 4 are a plan view and a front view showing the positional relationship between the luggage and the robot hand for explaining the measurement purpose of the present invention. In the present invention, the robot hand center is the center of the luggage. The shift amounts Δx, y, z, and θ shown in FIGS. 3 and 4 are measured in order to bring the robot, and the robot hand center is shifted by Δx, y, z, and θ to match the center of the luggage. Therefore, in the present invention, the height position, the plane position, and the posture are measured.

(1) Measurement in the height direction FIG. 5 is an image processing flow chart for detecting the height position h, and FIG. 6 is an explanatory view schematically showing the processing of the slit image, from the laser marker attached to the hand. The slit light is radiated onto the package from an oblique direction, the irradiation state of the slit light on the package is captured by a CCD camera, and the slit image is captured by the image processing device. The image processing apparatus sets a threshold value so that only slit light can be extracted, and binarizes the image data to obtain a binary image. Next, the central axis (center of gravity) coordinates P _xl of the slit image in the P _x axis direction are obtained in the detection area of the slit. Since the slit light is radiated to the luggage from an oblique direction, the height position (h) is obtained from the following equation using the principle of triangulation by utilizing the fact that P _xl varies depending on the height of the luggage.

FIG. 7 is an explanatory view for obtaining the height position (h), and the height position is obtained from h = b _h / (p _xl + a _h ). Here, a _h and b _h are the coordinates P _xl1 and P _{xl2 of} the slit light observed at known heights h ₁ and h ₂ in _advance.
Can be obtained from the following equation. Known height h
_{For 1} and h ₂ , h ₁ = b _h / (P _xl1 + a _h) and h ₂ = b _h
/ (P _xl2 + a _h ), and when a _h and h ₂ are obtained from this equation, a _h = (h ₁ · P _xl1 −h ₂ · P _xl2 +) / (h ₁ −h ₂ ) b _h = to become (P _xl1 over P _xl2) · (h ₁ over h ₂₎ / (h ₂ over h _1).

(2) <Calculation of Plane Position and Attitude> FIG. 8 is an image processing flowchart for detecting the plane position and attitude. The halogen lamp attached to the hand of the robot or the like diagonally above the package to be picked. After the image is taken in such a manner that shadows are formed on the two edge portions of the upper surface of the luggage when the light is applied more and the CCD camera looks into the image, the image taken with the light OFF is taken. Next, in order to reduce the influence of ambient light and extract only the brightness component of the illumination light to emphasize the edge portion of the luggage, the illumination is turned off from the image captured with the illumination turned on.
After the difference image obtained by subtracting the image captured in F is obtained, the threshold value is set in advance, and the binarization processing sets the dark portion of the shadow to white and the bright portion of the luggage upper surface to black. Invert to obtain a binary image.

FIG. 9 is an explanatory view for extracting the baggage edge points, and is an image (FIG. 9d) in which a binary image (FIG. 9a), a white mask pattern image (FIG. 9b) and a black mask pattern image (FIG. 8c) are combined. ) Is obtained, a hole filling process (FIG. 8e) is performed to make a dark region surrounded by white pixels white. By this image processing, the contour of the upper surface of the package is extracted.

FIG. 10 is an explanatory view of the edge point extraction method of the package, in which the edge point changing from white to black is obtained by scanning the image of FIG. The edge line segment (L ₁ ) and the edge line segment in the horizontal direction (L ₂ ) are calculated by the least square method.

<Calculation of Edge Segment> From the straight line formula, (L ₁ ): Py = a ₁ Px + b ₁ (L ₂ ): Px = a ₂ Py + b ₂ and the least square method is used to calculate (a ₁ , b ₁ ),
Find (a ₂ , b ₂ ).

A ₁ = {nΣ (x _i · y _i ) −Σx _i · Σ
y _i } / {nΣx _i ² − (Σx _i ) ² } b ₁ = {(Σx _i ² ) · (Σy _i ) − (Σx _i ) · (Σx _i ·
_{y i)} / {nΣx i} 2 chromatography ^{_{(Σx i) 2} a 2}} = {nΣ (x i · y i) over _{Σx i · Σy i} / {} nΣy i
²⁻ (Σy _i ) ² } b ₂ = {(Σy _i ² ) · (Σx _i ) − (Σy _i ) · (Σx _i ·
y _i )} / {nΣy _i ² − (Σy _i ) ² } where n is the number of scans.

<Calculation of Rotation Angle of Package> FIG. 11 is an explanatory view of the rotation angle of the package. The rotation angle θ ₁ formed by the straight line L ₁ and the straight line L ₂ is shown.
And θ ₂ are averaged, and the rotation angle θ is calculated from the following equation.

Θ = (θ ₁ + θ ₂ ) / 2 = {tan ⁻¹ (−a ₁ ) + tan ⁻¹ (a ₂ )} / 2 <Calculation of central coordinates of package> FIG. 12 shows central coordinates and diagonal points. An explanatory diagram for obtaining the coordinates, in which the central coordinates of the package (p
x ₀ , py ₀ ) is obtained from the intersection (px ₁ , py ₁ ) of L ₁ and L ₂ and the diagonal point (px ₂ , py ₂ ).

The intersection coordinates of L ₁ and L ₂ are obtained from Py ₁ = (a ₁ · b ₂ + b ₁ ) / (1-a ₁ · a ₂ ) Px ₁ = a ₂ · Py ₁ + b ₂ and the pair corner points _{coordinates, p x2 = p x1 + L} , cosθ over _{^{W, sinθ p y2 = px1 +}} L, sinθ over ^W, obtained from cos [theta]. In the above equation, L 1 ^, W ^, (pixel)
Is the apparent dimension of the length L and width W (mm) of the luggage at the distance h, and the length L and W (mm) of the luggage is (mm) → (pixels) at the distance h that is obtained in advance. The conversion coefficients Fpx and Fpy can be converted by the following equation.

L ^, = L / Fpx W ^, = L / Fpy When the coordinates of the intersection and the diagonal point are obtained, then the package center coordinates (px ₀ , py ₀ ) are calculated by the following equation.

Px ₀ = (Px ₁ + Px ₂ ) / 2 Py ₀ = (Py ₁ + Py ₂ ) / 2 <Conversion to robot coordinate system> FIG. 13 is for converting the shift amount from the pixel coordinate system to the robot coordinate system. In the explanatory diagram of FIG. 3, the shifts εx and y in the x-axis direction between the camera center and the luggage center
Axial offset εy are translated from advance correction coefficients of sought of the optical axis of the camera is shifted c _x and c _y and the transform coefficients Fpx and Fpy by the following equation.

[0028] _{εx = {(p x0 -px)} + c x} · F px (mm) εy = {(p y -p y0) + c y} · F py (mm) Therefore, the shift amount is calculated from the following equation To be done.

.DELTA.x = Sx-.epsilon.x .DELTA.y = Sy-.epsilon.y In this description, the relationship between the pixel coordinate system and the robot coordinate system is shown in FIG. 13, but the combination of coordinate systems, that is, the orientation of the hand in measurement is The combinations shown in FIGS. 14 (a) to 14 (c) are selected at appropriate times depending on the arrangement and orientation of the.

Further, in the present measuring method, the edge of the package is detected from the outside as shown in FIG. 15 (a), so that it is possible to measure the plastic container without being affected by the contents, Furthermore, FIG.
As shown in (c), if the mask pattern is synthesized, the edge is detected, and then the center point of the baggage is obtained from the circle formula,
It can also be applied to cylindrical luggage.

[0031]

The effects of the present invention are as follows.

(1) Accurate and reliable measurement can be performed with a simple device configuration.

(2) It is possible to measure a wide range of positions such as a cardboard box whose surface is not flat, a plastic container, and a cylindrical luggage.

(3) It is possible to perform measurement that is not easily influenced by the image around the luggage.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a measurement system according to the present invention.

FIG. 2 is an overall processing flowchart of the present invention.

FIG. 3 is a plan view showing the positional relationship between luggage and hands.

FIG. 4 is a front view showing the positional relationship between luggage and hands.

FIG. 5 is an image processing flowchart for detecting a height position.

FIG. 6 is an explanatory diagram for obtaining a height position (h).

FIG. 7 is an image processing flowchart for detecting a plane position and orientation.

FIG. 8 is an image processing flowchart for extracting the outline of a package.

FIG. 9 is an explanatory view of contour extraction of a package.

FIG. 10 is an explanatory view of an edge point extraction method for luggage.

FIG. 11 is an explanatory diagram of a rotation angle of luggage.

FIG. 12 is an explanatory diagram for obtaining center coordinates and diagonal point coordinates.

FIG. 13 is an explanatory diagram for converting a shift amount from a pixel coordinate system to a robot coordinate system.

FIG. 14 is a combination diagram of a pixel coordinate system and a robot coordinate system.

FIG. 15 is an explanatory diagram of position measurement of a plastic container and a cylindrical load.

[Explanation of symbols]

1 robot, 2 robot control device, 3 palletizing computer, 4 image processing device, 5 hands,
6 laser marker, 7 illumination, 8 CCD camera, 9 power cable, signal cable, 10 power source, 11 luggage

Claims (1)

[Claims] 1. A package is irradiated with slit light from an oblique direction of the package, the height of the camera is measured from the package by triangulation, and illumination is applied from the oblique direction of the package to synthesize a captured image and a mask image. The feature is that the disturbance element in the luggage position measurement area is removed, the contour of the luggage upper surface is extracted, and the position and rotation angle in the plane direction of the luggage are obtained from the height position and luggage dimension information previously obtained. How to measure the position of luggage.