JPH1053316A - Sliding quantity measuring method of conveyor and tracking action correcting method of robot - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method to simply detect the sliding quantity of a conveyor by means of a visual sensor and correct the tracking movement of a robot. SOLUTION: The mark M provided in the edge section area 3 of a circulating conveyor 1 is detected by a camera, which repeats intermittent photographing in a short cycle. Mark images are positioned at random. M' and M" represent the image positions where images was taken at the previous chance and will be taken at the next chance respectively. After calculating in terms of the counted value of a pulse-coder of the conveyor compensating the passing distance di measured in the direction downstream of the conveyor belt 1 from the datum line 33 in the visual field, the pulse-coder's-discrete value per the circulation of the conveyor belt 1 is measured. The sliding quantity of the conveyor per one circulation is obtained by converting the difference between the maximum and minimum discrete values into distance. The tracking action of a robot can also be corrected by detecting the local sliding quantity using irregular texture.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for detecting a slip amount of a conveyor using a visual sensor and a method for correcting a tracking operation of a robot according to the detected slip amount of the conveyor. The method of the present invention is advantageously applied to a visual tracking system.

[0002]

2. Description of the Related Art In a production line in a factory, a system combining an automatic machine such as a robot and a conveyor for transporting an object (hereinafter, referred to as a "work") such as an assembly part is generally used. Has become. In particular, a system that employs a visual sensor to allow a robot to perform a tracking operation on a workpiece being transported,
It is called a visual tracking system and can be used without stopping the conveyor.

In order to improve work accuracy in such a system, it is necessary to accurately grasp the movement of a work, that is, the movement of a conveyor. The movement of the conveyor is
Typically, this is grasped using the output of a pulse coder coupled to the drive shaft of the conveyor. Since the accuracy of pulse coders is generally very high, it is not very difficult to accurately detect the movement of the conveyor drive shaft.

[0004] However, in general, a conveyor has a property that slippage easily occurs in a traveling member such as a belt due to its structure. Therefore, when constructing a visual tracking system or the like, it is necessary to grasp how much slippage occurs on the conveyor. Conventionally, an operator measures the amount of slippage of a conveyor by comparing the output of a pulse coder with the travel distance of a traveling member.

[0005] In general, it is unknown which part of the running member slides when approaching the drive. For this reason, when the traveling member is long, the burden on the operator becomes extremely large, and when slippage occurs intermittently, the slip detection itself is not easy. Further, since it is actually difficult to perform the measurement under the same conditions as in the actual work in which the conveyor continuously travels, the measured value of the slip amount may not be reliable.

[0006]

SUMMARY OF THE INVENTION Accordingly, it is a basic object of the present invention to solve the above-mentioned problems of the prior art. That is, the present invention intends to provide a method for easily detecting the amount of slippage of a conveyor using a visual sensor. The present invention further contemplates providing a method for detecting the amount of slippage of a conveyor in a form capable of correcting a tracking operation of a robot in a visual tracking system in real time and outputting the detected amount to a robot control means. It is.

[0007]

SUMMARY OF THE INVENTION The present invention is basically embodied in two forms. In the first embodiment, the slip amount of the traveling body of the conveyor per circumference of the conveyor is obtained.
That is, the mark on the traveling body of the conveyor having the position detecting means (for example, a pulse coder) is repeatedly detected by the camera means of the visual sensor, and the position detecting means when the mark is detected at a substantially fixed position within the visual field of the camera means. The output value is analyzed for a number of rounding cycles to determine the slip amount of the traveling body per round of the conveyor.

The output value of the position detecting means when the mark is detected at a substantially fixed position in the field of view of the camera means is preferably analyzed after correcting the mark detection position from the fixed position. . The slip amount per conveyer can be estimated based on the difference between the maximum value and the minimum value of the output value of the position detecting means per conveyer.

In the second embodiment of the present invention, a local slip amount of the traveling body of the conveyor is obtained. That is, a pattern on the traveling body of the conveyor having the position detecting means is repeatedly detected in a short cycle by the camera means of the visual sensor, and the movement calculated based on the output value (change amount) of the position detecting means during the short cycle. By comparing the distance with the movement distance of the pattern detected by the visual sensor, a local slip amount of the traveling body is obtained.

As a pattern recognized by the visual sensor, an irregular texture can be used. If the data of the local slip amount is continuously accumulated, the tracking operation of the robot cooperating with the conveyor can be corrected based on the accumulated data.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 schematically shows an example of an overall arrangement for implementing the method of the present invention in a visual tracking system. In the figure, reference numeral 1 denotes a linear transport conveyor that transports works W and W ′ supplied one by one from a work supply source 100 such as a final press step execution unit. Work source 1
Although the work supply interval from 00 is irregular, it is assumed that the robot RB performing the tracking operation at the downstream position of the conveyor 1 has such a size that the robot RB can work on all the supply works with some margin. . Conveyor 1
Are driven by a conveyor drive motor built in the drive unit 2. A pulse coder 3 is connected to the conveyor drive motor as position detection means, and outputs a position signal of the motor (drive shaft) in the form of a pulse train.

The conveyor 1 is provided with an endless belt (or an alternative running body; the same applies hereinafter) that circulates between an upstream end 1a and a downstream end 1b.
In the present embodiment, this conveyor belt is a target for detecting the slip amount. Further, in the following description, reference numeral 1 is also used as a reference for designating the conveyor belt itself.

Reference numeral VS denotes the image processing device 20 and the camera 3
0 (for example, a CCD camera). The camera 30 is installed on the upstream side of the robot RB so as to have a visual field 31 covering almost the entire width of the conveyor belt 1 from above the conveyor traveling path. Reference numeral 32 denotes an edge area used for detecting a slip amount in a manner described later.
This is performed so that W ′ is not placed.

As shown by the broken line in the figure, the image processing device 20 has a form built in the robot controller RC. The robot controller RC has a robot control unit 10 that controls a robot RB that performs a tracking operation. The robot controller 10 obtains data representing the position and orientation of the workpiece W from the built-in image processing device 20 and counts the output value N of the pulse coder 3.
Is used to control the tracking operation of the robot RB. Further, when the slip amount is detected by the visual sensor VS, data representing the slip amount of the conveyor belt 1 is received, the result is displayed on a display of a teaching operation panel (described later), or the tracking operation of the robot is corrected. Use.

An outline of the internal configuration of the robot controller RC is shown in FIG. In the figure, an image processing device 20 built in a robot controller RC has a CPU (central processing unit) 21 composed of a microprocessor. The CPU 21 includes a camera interface 22, a frame memory (image memory) 23, a program memory 24, an image processor 25, a data memory 26, and a monitor interface 27.
Are connected via a bus BS ″.

The camera interface 22 has a camera 3
0 is connected, and when a shooting command is sent via the camera interface 22, the electronic shutter function set to the camera 30 (the shutter speed is set to, for example, 1000
The image is converted into a gray scale signal via the camera interface 22 and stored in the frame memory 23.

The monitor interface 27 has a monitor C
The RT 40 is connected, and an image being captured by the camera 30, a past image stored in the frame memory 23, an image processed by the image processor 25, and the like are displayed as necessary.

Work W stored in frame memory 23
Are analyzed using the image processor 25 in accordance with the analysis program stored in the program memory 24. The analysis programs prepared in the present embodiment include the following three types. The processing content corresponding to each program will be described later.

(1) An analysis program for extracting the characteristic points of the work and calculating the position and orientation of the work. Here, as shown in FIG. 1, the work W has four feature points a, b,
c and d, and when all four points are detected, the position / posture / posture of the work is calculated based on the detected points.

(2) An analysis program for calculating the amount of slip per circumference of the conveyor belt 1 for checking the performance of the conveyor belt 1 or diagnosing the condition.

(3) An analysis program for calculating a local slip amount of the conveyor belt 1 to correct the tracking operation of the robot RB.

The data memory 26 has an area for storing various set values related to the visual sensor system,
Includes an area used for temporary storage of data necessary for various processes executed by the server. Note that registers (to be described later) used for storing data in the processing related to the slip amount detection are set in the data memory 26.

The CPU 21 of the image processing device 20
Is connected to the CPU 11 of the robot controller 10 via a bus BS inside the robot controller RC. As a result, the entire visual sensor 20 substantially constitutes a device integrated with the robot controller 10 (a robot controller RC with a built-in visual sensor).

The robot control unit 10 has a CPU connected to the CPU 21 of the image processing apparatus 20 via a bus BS.
11 is provided. The CPU 11 has a ROM 12 storing a program for controlling the entire system,
RAM1 used for temporary storage of data for PU processing
3. Non-volatile memory 14 for storing operation programs, coordinate system setting data, other various setting parameters, etc., axis controller 15 connected to the mechanical unit of robot RB including hand via servo circuit 16, digital signal processor A (DSP) data memory 17, a digital signal processor (hereinafter, referred to as "DSP") 18, and an interface (for a teaching operation panel or general-purpose) 19 are connected via a bus BS '.

The DSP 18 is a processor for processing the count output signal of the pulse coder 3 connected thereto.
The SP data memory 17 is a DSP-specific memory that stores various processing data and setting parameters by the DSP 18. The DSP 18 detects the count output of the pulse coder 3 at an arbitrary time according to the command of the CPU 11,
It has a function of writing to a predetermined area of the P data memory 17. The CPU 21 of the image processing apparatus 20 can access the DPS data memory 17 via the CPU 11.

The interface 19 is connected to a teaching operation panel TP which also functions as an operation panel of the entire system and is equipped with an LCD display. This display can also display the calculation result of the amount of slip of the conveyor belt.

Based on the above system configuration and function, the method of the present invention will be described in two embodiments 1 (detection of slip per rotation) and Embodiment 2 (detection of local slip and tracking based thereon). The operation will be described separately.
The outline of the tracking operation of the robot RB in the present embodiment is as follows. (1) When the work W reaches the tracking start line 60, a tracking operation with the initial position P0 is started, (2) the work W is encountered at a point Q0 near the teaching position, and (3) the work W is By the time the robot reaches the tracking end line 70 (position Q1), the work on the work W (for example, application of a sealing agent, pouring of a resin material, etc.) is completed, and (4) the initial position after being separated from the conveyor 1 to the side. Return to P0.

[Preparation work (common to the first and second embodiments)] (1) Determination of scale factor The travel distance l of the conveyor 1 is determined by the following procedure similar to the conventional one.
And the count output of pulse coder 3 (incremental amount Δ
N), and a scale factor α = 1 / ΔN representing the relationship with N) is obtained. However, it should be noted that the scale factor α obtained here includes an error caused by slippage of the conveyor 1 that may occur irregularly.

1. The work is set at a position within the operation range of the robot RB, and the count output N1 of the pulse coder 3 at that time is set.
Is stored in the robot controller RC. 2. The robot RB is manually operated to touch an appropriate fixed point on the workpiece W, and the robot position (X1, Y1) at that time is touched.
, Z1) are stored in the robot controller RC.

3. The conveyor 1 is run for an appropriate distance, the work is stopped at an appropriate position within the operation range of the robot RB, and the count output N2 of the pulse coder 3 at that time is stored in the robot controller RC. 4. By manually operating the robot RB, the user touches an appropriate fixed point on the workpiece W, and the robot position (X2, Y2,
Z2) is stored in the robot controller RC. 5. In the robot controller RC, α = (X2-X1)
/ (N2 -N1) to calculate the scale factor α
From the nonvolatile memory 1 of the robot controller RC.
4 and the data memory 26 of the image processing apparatus 20.

(2) Setting of Sensor Coordinate System Σs The sensor coordinate system Σs is added to the visual sensor VS by appropriate means.
Set. For example, a known calibration jig is arranged at a position on the base coordinate system Σb where the coordinate value is known, and the calibration program stored in the program memory 24 of the image processing apparatus 20 is started to activate the camera 30. Is stored in the data memory 26 of the image processing apparatus 20 for converting the pixel value data of the image processing device into the data of the sensor coordinate system Σs. Using the result, data representing the relationship between the sensor coordinate system Σs and the base coordinate system Σb is stored in the non-volatile memory 14 of the robot controller 10.

(3) The work W is set on the conveyor 1 in the field of view 31 of the camera 30, and the pulse coder count output value Nvs at that time is stored in the DSP data memory 17 of the robot control unit 10 and the data memory of the image processing device 20. 26
To be stored. Further, the camera 30 performs photographing, captures an image of the work W as a reference image, and stores the image in the frame memory 23.

(4) The conveyor 1 is run and the robot R
When the workpiece W reaches a position (line 60) suitable for starting the tracking operation at B, the conveyor 1 is stopped. Then, the pulse coder count output value Ntr at that time
Difference between 1 and Nvs previously stored (incremental amount) Δ
Ntr1 -vs = Ntr1 -Nvs is calculated and stored in the DSP data memory 17.

(5) The conveyor 1 is further moved, and the robot RB brings the work W to a position suitable for starting work on the work W. Then, from the pulse coder count output value Ntc at that time and the previously stored Nvs, ΔNtc−vs
= Ntc-Nvs and stores it in the DSP data memory 17.

(6) The robot RB is instructed of an operation necessary for executing an intended operation (for example, injection of a resin material, application of a sealing agent, etc.). Here, the details are omitted. (7) A position (line 7) suitable for moving the conveyor 1 again to end the tracking operation of the robot RB.
When the workpiece W reaches 0), the conveyor 1 is stopped. Then, the pulse coder count output value Ntr2 at that time is obtained.
Then, ΔNtr2-vs = Ntr2-Nvs is calculated from the previously stored Nvs, and the result is stored in the DSP data memory 17.

(8) Setting of Tracking Coordinate System In order to share the coordinate system of the robot RB and the conveyor 1, the tracking coordinate system Δtr is set under the following conditions. Here, it is assumed that the base coordinate system Σb in which the X axis matches the traveling direction of the conveyor has been set. That is, the tracking coordinate system Σtr is set in the same orientation as the base coordinate system Σb, and the origin position (X0, Y0, Z0) of the tracking coordinate system Σtr until the tracking coordinate system Σtr starts moving.
) Are X0 = -L0; Y0 = 0; Z0 = 0 (1). The coordinate values (X, Y, Z) on the base coordinate system Σb and the coordinate values on the tracking coordinate system target Σtr ( (x, y, z) is expressed as follows: x = X + L0; y = Y; z = Z (2) Here, L0 is the distance between the position of the teaching point Q0 and the tracking start line 60.
In order to determine the value of L0, L0 = α (ΔNtc-vs-ΔNtr1 -vs) (3) is calculated using the scale factors α, ΔNtr1 -vs, and ΔNtc-vs already obtained. Good. The data of L0 is stored in the non-volatile memory 14 of the robot controller 10 as the parameter described above for the tracking coordinate system Σtr.

By setting the tracking coordinate system Σtr in this way, when the robot is operated on the tracking coordinate system あ る tr at the initial position, the robot controller 10 sets the position of the teaching point Q0 on the tracking start line 60. Point Q
Recognize as 0 '.

The tracking coordinate system Σtr receives the tracking operation start command and moves at the same speed as the conveyor 1 in the x-axis direction (=
Start moving in the X-axis direction). Tracking coordinate system Σtr
Is determined in real time based on the amount of change in the pulse coder count output value N and the scale factor α, as will be described later.

(9) In addition, setting of registers to be accessed in processing to be described later, setting of parameters for adjusting the length of the field of view 31, shooting intervals, reference values, flag values, etc. are performed (initial values of w, q, F). Values, f30, ε1, ε2, etc.). Thus, the basic preparation for executing the tracking operation is completed.

[Embodiment 1] In this embodiment, the slip amount per revolution of the conveyor belt 1 is detected for the purpose of checking the performance of the conveyor, diagnosing the operation state, and the like. It can be executed before the operation is started or when the operation is stopped. However, the work supply and the robot R
By executing the operation in parallel with the operation B, a load during actual work may be applied to the belt. Here, for convenience of explanation, the work supply and the operation of the robot RB will not be described.

[1] Preparation Preparation A mark recognizable by the visual sensor VS is provided at an appropriate position of a portion of the conveyor belt 1 corresponding to the edge region 32 (see FIG. 1). This mark provides a visual target for detecting the amount of slip per lap, and its shape, how to provide it (sticking a sticker, drawing directly, using an existing mark, etc.), and the like, do not hinder recognition by the visual sensor VS. It is generally arbitrary unless otherwise specified. Here, as shown in FIG. 3, a mark having a contrast between the color of the background of the conveyor belt 1 and the brightness is adopted.

In FIG. 3, the mark M is the field of view 3 of the camera.
The reference line 33 is drawn in the edge area 32 in the position 1 beyond the distance di. When photographing by the camera is executed intermittently in a short cycle while the conveyor belt 1 is running, the position of the acquired mark image also takes a discrete position in the field of view. Symbols M ′ and M ″ exemplify mark positions or image positions obtained at the previous and next shooting opportunities with respect to the shooting opportunity at which the image of the mark M was obtained, respectively.

That is, the mark M represents the mark position or mark image position at the first photographing opportunity after the mark position reaches the in-field reference line 33 in each belt rotation. The overtaking distance measured downstream from the reference line 33 in the visual field is represented by di. The subscript i represents the distance measured in the i-th round. For the distance di, di ≧
Needless to say, 0 (i = 1, 2, 3,...) Holds. The position of the reference line 33 in the visual field is set by designating one of the pixel columns of the camera in advance. here,
It is assumed that the position of the center pixel column is set to the reference line 33 in the visual field. 2. Execution (1) First, the traveling of the conveyor belt 1 provided with the mark M is started. In general, the traveling speed may be arbitrary. However, in consideration of the possibility that the amount of slip changes depending on the traveling speed, it is also preferable to change variously during the measurement together with the load condition (low-speed low-load, high-speed high-load, etc.). As described below, in the present embodiment, the slip amount is calculated by regarding the time when the minimum slip amount is observed as the slip amount 0.

(2) Processing flow (processing 1) shown in FIG.
, The slip amount per belt rotation is estimated and calculated.
The main points of the processing in steps G1 to G11 are as follows. In the process, after the start of traveling of the conveyor belt 1, the initial value of the belt rotation index i is set to 1 and the saturation value is set to i0.
The process is started by setting all initial values of the i0 registers to 0. Generally, for accurate measurement, the saturation value i0
The larger the value of is, the better. However, an appropriate value is set in consideration that setting a large value increases the time required for measurement.

[G1] An image in the field of view 31 is obtained by photographing with the camera 30. [G2] The i-th register value Ni is updated to the pulse coder output count value at the time of photographing. [G3] Attempt to detect the mark M, and if the detection is successful, proceed to step G4. If the detection is unsuccessful, the process returns to step G1 to perform photographing again. [G4] The position of the detected mark M (● center position) is
It is determined whether or not it is on the near side (upstream) of the visual field reference line 33. If yes, the process returns to step G1, and if no, the process proceeds to step G5.

[G5] Overtaking distance di of visual field reference line 33
Is calculated and converted into a pulse coder count value. For the conversion, a preset photographing magnification (distance per pixel) of the camera 30 and a distance / pulse coder count conversion coefficient α are used. [G6] The i-th register value Ni is updated to Ni-di. As a result, an error caused by the variation in the mark detection position given to the slip amount measurement is substantially corrected. [G7] The value of the belt rotation index i is increased by one. [G8] It is checked whether or not the value of the circulation index i exceeds the saturation value i0. If it does, the process proceeds to step G9 (when the conveyor circulates in the i0 cycle), and if not, the process proceeds to step G1.
(When the conveyor circulates less than i0 lap). [G9] i0 register values Nj (j = 1, 2, 3,...)
i0), the difference Nj + 1-N between successive register values
The minimum value of j, MIN [Nj + 1−Nj] (j = 1, 2, 3,
··· Find i0 -1).

[G10] i0 register values Nj (j =
1,2,3... I0), the maximum value of the difference Nj + 1-Nj between successive register values, MAX [Nj + 1-Nj]
(J = 1, 2, 3,... I0 -1).

[G11] Dround = α (MAX [Nj + 1
−Nj] −MIN [Nj + 1−Nj]). Data including the calculation result Dround is displayed on the display of the teaching operation panel TP. Such a calculation method is called MIN
This is equivalent to estimating the maximum amount of slip per rotation by assuming that the slip amount (actual distance) when the belt circulates, where [Nj + 1-Nj] is obtained, is 0.

As an example for explanation, the results obtained when the following register values N1 to N15 are obtained by performing measurement for 15 rounds are as follows. The conversion coefficient is α =
The value was 0.01 mm / count value.

N1 = 24,511 N2 = 824,628 N2-N1 = 800,117 N3 = 1,624,736 N3-N2 = 800,108 N4 = 2,424,962 N4-N3 = 800,226 N5 = 3,224,747 N5 -N4 = 799,785 N6 = 4,024,752 N6 -N5 = 800,005 N7 = 4,825,003 N7 -N6 = 800,251 N8 = 5,625,111 N8 -N7 = 800,108 N9 = 6,425,102 N9 -N8 = 799,991 N10 = 7,225,396 N10-N9 = 800,294 N11 = 8,025,685 N11-N10 = 800,289 N12 = 8, 825,806 N12-N11 = 800,121 N13 = 9,625,807 N13-N12 = 800,001 N14 = 10,425,797 N14-N13 = 7 9,990 N15 = 11,226,272 N15-N14 = 800,275 MIN [Nj + 1-Nj] = 799,785 MAX [Nj + 1-Nj] = 800,294 MAX [Nj + 1-Nj]- MIN [Nj + 1−Nj] = 509 Dround = 5.09 mm The slip amount per one turn estimated and calculated in this manner is:
It is used for checking the performance of the conveyor belt 1 or diagnosing the condition. For example, in the case of the belt conveyor 1 in which the slip amount per rotation exceeding an allowable standard (for example, 1 cm) is measured in advance check or maintenance inspection, parts replacement, cleaning,
Measures such as model change are taken.

Further, when the tracking operation of the robot RB is hindered during the operation of the system, if the slip amount is measured according to the present embodiment, the cause of the system abnormality is determined and the action to be taken is determined. Useful data is provided above. In addition, during the actual work (work supply and tracking operation of the robot, etc.) described in the second embodiment, if the above-described processing is periodically performed in parallel, the present embodiment can reduce the belt slip during the operation of the system. It can also be used as monitoring means.

[Embodiment 2] In this embodiment, a technical means is provided for preventing the tracking operation of the robot RB from deteriorating even if a certain amount of slippage occurs on the conveyor belt. That is, unlike the first embodiment, the detection of the slip amount is repeatedly executed in a cycle sufficiently shorter than the belt rotation cycle. Detection result is robot RB
Are accumulated in a form suitable for correcting the tracking operation in real time. In this specification, the slip (amount) detected in such a short belt running period is referred to as “local slip (amount)”.

In general, a visual target for detecting local slip at all times must be present over the entire length of the conveyor belt 1. 5 and 6 show examples of a visual target for detecting local slip. In the following description, the imaging performed for detecting the position and orientation of the workpiece is referred to as imaging 1, and the imaging performed for detecting local slip is referred to as imaging 2. As will be described later, the former is repeated almost every time the conveyor travels with a visual field length of f30,
The latter has a sufficiently shorter period (for example, several tenths).
Cycle).

1. As shown in FIG. 5, when a regular scale SC exists at an appropriate density over the entire length of the edge of the conveyor belt 1; in this case, the pulse coder count output value N
As shown in the chart based on the time point 0, the actual movement amounts Δd1, Δd2,... Are obtained from the images of the scale SC obtained in the imaging 2 at every sufficiently short pulse coder counting period ΔN. Then, these are compared with the movement amount (movement amount when there is no slip) represented by the increment ΔN of the pulse coder count output value during the corresponding period, and ΔDk = αΔN−Δdk (k = 1, 2,...) By calculating (4), the local slip amount can be obtained. Generally, in order to calculate the slip amount during a certain period specified by the pulse coder count values N1 to N2, the integrated value 求 め (N1-N2) ΔDk of ΔDk during that period may be obtained.

2. As illustrated in part in FIG. 6, when the irregular texture TX such as dirt, irregular pattern, or the like does not extend over the entire edge of the conveyor belt 1; in this case, the visual field 31 ( The reference region 321 is set as a constant region having an appropriate width β in the edge region 32 in the inner region 32 (see FIG. 1), and photographing is performed in a short cycle (every pulse coder counting cycle ΔN).
repeat. Then, the images acquired at the time of the execution of the imaging 2 before and after are compared, and the moving distance of the texture image TX included in the reference area 321 is obtained. Then, by comparing this with the moving distance calculated from the pulse coder output, the local slip amount can be detected in a short period (imaging 2 execution interval).

Now, the pulse coder count output values at the time of executing the first, second, and third photographing 2 are represented by N 0, N 0 + ΔN, N 0 +, respectively.
When 2ΔN is set, the texture image TX moves as shown in the chart with the movement of the conveyor. That is, the texture image element that was in the reference area 321 at the time of the count value N0 moves into the area of the width β indicated by the reference numeral 322 when the count value N0 + ΔN. Similarly, the texture image element which was in the reference area 321 at the time of the count value N0 + .DELTA.N moves to the area of the width .beta. Here, it should be noted that the region 322 and the region 323 do not match if there is irregular local slip.

If the distance obtained by converting the texture image element movement amount for each photographing 2 into the actual movement amount is δ 1, δ 2..., The local slip amount Δ is obtained by the following expression (5) instead of the above expression (4).
Dk can be calculated. .DELTA.Dk = .alpha..DELTA.N-.delta.k (k = 1, 2,...) (5) In order to calculate the cumulative slip amount in a certain period specified by the pulse coder count values N1 to N2, the integrated value of .DELTA.Dk during that period.
(N1-N2) ΔDk may be obtained. As in this example, a gray scale normalized correlation method can be used to determine the moving distance of an irregular image pattern. The moving distance δk (k = 1, 2) is applied by applying the gray scale normalized correlation method.
..) Will be described later.

Next, an outline of the processing when the actual operation is performed while correcting the tracking operation of the robot RB using the detection result of the local slip will be described with reference to FIGS. The entire process is roughly divided into a process (process 2 / process 3) on the image processing apparatus 20 side and a process (process 4) on the robot control unit 10 side. The former is further a process for work position / posture detection (process 2). ) And a process for detecting a local slip amount (process 3). These processes are handled as a series of tasks in the robot controller RC and are executed in parallel. The following description is made under the following assumptions (1) to (3).

(1) w indicates that the storage of the data representing the detection result of the work position / posture has been completed and the robot controller 1
This is an index value (register value) representing the number of works for which data reading by 0 has not been completed, and its initial value is w
= 0. (2) F is a flag for separating the photographing 1 for detecting the work position / posture and the photographing for detecting the local slip in the processing. As will be understood from the content of a processing 2 described later, F = 1 is almost attained during the operation of the system, and F = 0 is obtained only for a very short period before and after photographing 1. The initial value of the flag F is F = 0.

(3) There is some variation in the posture of the supplied workpiece, and the maximum length of the workpiece in the transport direction is s0 as shown in FIG. (4) Process 2 to Process 4 are started simultaneously by an appropriate external signal output that indicates that the work supply source 100 has started supplying the work onto the conveyor 1. (5) v is an index indicating the number of executions of shooting 1. The data of the local slip amount is accumulated for each period specified by the index v (details will be described later). The initial value of the index v is v = 0.

The main points of each step of the process 2 (process for detecting the work position / posture) described in the flowchart of FIG. 8 are as follows. [S1] The photographing 1 is executed, and the obtained image is stored in the frame memory. [S2] The pulse coder count output value Nsd at the time of executing the photographing 1 is stored in the DSP memory 17 and the data memory 26. [S3] The flag F is inverted to F = 1. [S4] The index v is incremented by one. [S5] The qv value (described later) of the v-th local slip amount accumulation register is set to 1.

[S6] The image acquired in the photographing 1 is processed and analyzed, and an attempt is made to detect a workpiece from the most downstream side. [S7] If the work has been successfully detected (all the points a to d have been detected), the process proceeds to step S11. If not, the process proceeds to step S8.

[S8] While repeatedly checking the pulse coder count output value N in a short cycle, the value α (N-Nsd) obtained by converting the pulse coder count value (N-Nsd) from the execution of the latest photographing 1 into a movement amount is obtained. f30-ε1 (From the field length f30, ε1
Wait just over a short distance). If it exceeds, go to step S9. Here, ε1 is used to prevent a region where photographing is omitted from occurring between two images acquired in the preceding and succeeding photographing 1, and that all points a to d are detected at any photographing opportunity for all works. .Epsilon.1> 0. It is preferable that .epsilon.1 be set to a substantially minimum value, provided that it does not fall below the maximum work length s0.

For example, the visual field length is 90.0 cm, s0 = 1.
When set to 0.0 cm, ε1 = 10.5 cm to 11.0 c
m. If ε1 is set unnecessarily large, the double photographing region (the region included in the image acquired in the preceding and succeeding photographing 1) is undesirably large. In particular, ε1
If> 2s0, there is a possibility that two detections are performed for one work.

If local slippage of the conveyor 1 is considered, there is a possibility that α (N−Nsd) does not accurately represent the traveling amount of the conveyor 1 since the latest photographing 1 was executed. However, in this case, it is assumed that the local slip is small even if it is present, and has only an effect of slightly shifting the imaging cycle (an effect equivalent to slightly changing ε1), and there is no problem in system operation.

[S9] The flag F is inverted to F = 0. [S10] After confirming that the processing end signal has not been output, the process returns to step S1 to execute the next photographing 1. If the processing end signal has been output, the processing ends. [S11] The result of the detection of the position and orientation of the work is stored in association with the pulse coder count output value Nsd at the time of executing photographing 1. The storage format will be described later. [S12] The value of the register value w is incremented by one. [S13] Attempt to detect another work (undetected work) from the downstream side of the image acquired in the photographing 1. [S14] If the work is successfully detected (all the points a, b, c, and d have been detected), the process returns to step S11. If unsuccessful, the process proceeds to step S8. The processing cycle described above (step S1
To S14) until the processing is completed, so that the conveyor 1
Data representing the positions a to d is sequentially acquired for all the workpieces supplied above. FIG. 9 is a diagram for explaining the storage format of these data. In each column, the detection positions of the line numbers, a, b, c, and d are represented by deviations (deviations) from the values of the work reference image in order from the left. The data represents the pulse coder count output value Nsd at the time of acquiring the image whose position has been detected.

In this example, the data of three works has been written (w = 3), and the positional deviation data (Δxa1, Δya) of points a, b, c, d for the three works.
1), (Δxb1, Δyb1) (Δxd3, Δyd3) are written together with the value of Nsd.

The pulse coder count output value Nsd at the time of image acquisition
Are merely exemplary, but Nsd on the first line
And Nsd in the second row coincide with each other, which means that the positions of the points a, b, c, and d have been detected with respect to two workpiece images in the images obtained at the same photographing 1 execution opportunity. doing. The writing of data is performed on the uppermost blank line, and the data access from the robot side is performed from the uppermost writing line (data of the most downstream work). Then, the data is erased after the data access is completed, and a process for creating another data storage row for another row is executed.

Next, the process for detecting local slip (process 3) will be described with reference to FIGS. 10 and 11 in addition to the reference diagrams. The main points of each step of the process 3 (the process for detecting the local slip amount) shown in the flowchart of FIG. 11 are as follows. [H1] It is determined whether the flag F is F = 1. When the system is started, F = 0, so the first photographing 1
Immediately after that, wait until F = 1. Thereafter, F = 0 only during a very short period before and after the photographing 1, and F =
It is one. [H2] Capture 2 is performed to capture an image. [H3] Pulse coder count output value Nref when photographing 2 is executed
Is stored. [H4] It is checked whether there is an image obtained in the previous shooting 2. Unless it is the first time, since the judgment output is yes, the process proceeds to step H5. In the case of the first time, the process returns to Step H1 (you may return to Step H2). [H5] From the image acquired in the current shooting 2, the reference area 3
An image of an area in which 21 has been moved in the conveyor traveling direction by the following distance hm is extracted. The number of extracted images is 2m0 + 1 in total. hm = α (current Nref−previous Nref) + mε2 (6) where ε2 is a minute distance value (for example, 0.1 mm) that is appropriately set in advance.

[H6] Each of 2m0 + 1 images in total,
The grayscale normalized correlation method is applied to the image of the reference region 321 extracted from the image obtained at the previous imaging 2 to obtain m0 + 1 correlation values CRm.

Generally, a correlation value between a model image and an input image is given by the following equation. CRm = {(sign A) × (A × A) / B × C} × 100 (7) where A = SΣ (I × M) − (ΣI) × (ΣM) B = SΣ (I × I) − (ΣI) × (ΣI) C = SΣ (M × M) − (ΣM) × (ΣM) Here, S: area of the model image M: density of the model image I: density of the input image A: cross-correlation between the input image and the model image B: auto-correlation of the input image C: auto-correlation of the model image The image of the reference area 321 extracted from the image obtained in the previous shooting 2 is regarded as a model image, and the image obtained in the current shooting 2 is used as the reference area 321.
Is regarded as each input image, and a correlation value CRm can be obtained by assuming each image of an area in which is moved by a distance corresponding to hm in the conveyor traveling direction. A gray scale value for each pixel may be used for the image densities I and M, and the sum Σ may be obtained for all pixels of the model image and the input image.

Note that CRm always takes a value between -1 and +1 and takes a maximum value of +1 if the images overlap each other. Therefore, under the condition that the shift amount hm substantially coincides with the belt travel amount, the correlation value CRm should be less than 1 and close to 1. The processing of the next step H7 is based on this.

[H7] The value mcr of m giving the maximum correlation value CRm is obtained.

[H8] Add mcrε2 to the accumulated value Σ (Nsd-N) of the v-th local slip amount accumulation register. However, the local slip amount accumulation register has a configuration as shown in FIG. 10, and the “v-th” is a cyclic interpretation of the row in FIG. For example, if a local slip amount of 0.5 mm is detected in the cycle starting from the fourth photographing 1, 0.5 is added to 1.3 in the fourth row to obtain 1.8. Further, for example, if a local slip amount of 1.0 mm is detected in the cycle starting from the 26th photographing 1, 1.0 is added to 0.5 in the fifth row to obtain 1.5.

The qv column of the row to be added is always 1 (see step S5 of processing 2). [H9] After confirming that the processing end signal has not been output, the process returns to step H1.

F = 1 until immediately before the next photographing 1 is executed
Therefore, the process of detecting the local slip amount and accumulating the data starting from the photographing 2 is repeated many times (for example, several tens of times) during the succeeding photographing 1. Each time the F value returns to 1 via 0, the data writing position (step H8) to the local slip amount accumulation register advances by one line. In this way, the detection data of the local slip amount is accumulated in a form of a break at each opportunity of photographing 1. The reason why all of the lines other than the fourth and fifth lines in FIG. 10 are 0 will be described in the description of the process 4.

Finally, an outline of the processing 4 performed on the robot control unit 10 side will be described. The main points of each step of the processing 4 shown in the flowchart of FIG. 12 are as follows. [R1] The operation program data including the position data of the teaching point Q0 (see FIG. 1) is read. [R2] A state is entered in which the register value w is repeatedly checked in a short cycle. When the position / posture of the work W is detected by the processing after the photographing 1 in the processing 2 on the image processing apparatus 20 side,
The result is stored in the format shown in FIG. 9 and the register value w becomes a value other than 0. If it is determined that w ≠ 0, the process proceeds to step R3. [R3] Data for one work is read. In the example of FIG. 9, the data in the first row is read. [R4] When the reading is completed, the work data is updated. In the example shown in FIG. 9, the data in the first row is erased, and the rows in which the data in the second row and subsequent rows are written are moved up by the first row in the following procedure. 1. The data in the first row is deleted. 2. The data in the second row is copied to the first row. 3. The data in the second row is deleted. 4. The data in the third row is copied to the second row. 5. The data in the third row is deleted.

[R5] The pulse coder count output value N is N
= ΔNtr1 -vs + Nsd.
If it has arrived, proceed to step R6. In the above example, Nsd = 85
4,065. ΔNtr1 -vs has already been measured in the above-mentioned preparation work. The pulse coder count output value of N = ΔNtr1 -vs + Nsd is an indication that the work has reached the position of the tracking start line 60.

If the position / posture of the work at the time of photographing 1 matches the position / posture of the work when the reference image was obtained in the preparatory work, and if the conveyor 1 does not slide, this value is set to that value. This should accurately indicate that the work has reached the position of the tracking start line 60. However, if the work position / posture varies or the conveyor 1 slides, the accuracy decreases. However, this degree of inaccuracy is absorbed by the tracking operation correction of the robot RB described below.

[R6] The movement of the robot RB is started on the tracking coordinate system Σtr, and the movement of the tracking coordinate system Σtr is started. The movement of the tracking coordinate system Σtr is performed by changing the origin position [X0, Y0, Z
0] is represented by the following equation. X0 = −L0 + α ｛N− (ΔNtr1−vs + Nsd)｝ − Σ (Nsd−N) Dk (8) Y0 = 0; Z0 = 0 (9) The tracking coordinate system Σtr is moving. The conversion formula of the coordinate value is as follows: X = x−L0 + α ｛N− (ΔNtr1−vs + Nsd)｝ − Σ (Nsd−N) Dk (10) Y = y; Z = z (11) Becomes

In Expressions (8) and (11), 式 (Nsd-N) Dk represents the time (Nsd counting time) from the time when the imaging 1 for detecting the position and orientation of the target work is executed (Nsd counting time). ) Is the cumulative value of the local slip amount up to). The value of Σ (N-Nsd) Dk is
In the register configuration shown in FIG. 10, the row of qv = 1 (FIG.
In the example of (4), the accumulated value of the fourth and fifth lines) can be obtained as a value obtained by adding the accumulated values.

In the example shown in FIG. 10, the distance is 1.8 mm. Such calculation is necessary because, when the imaging 1 in the subsequent cycle is executed, qv in the next line is rewritten from 0 to 1 as described above, and the added value of the detected value of the local slip amount is also added thereto. Because it moves. When the tracking of one or more workpieces detected in one photographing 1 is completed, the old data is no longer necessary, and is cleared (see steps R11 and R12).

When the tracking operation of the robot RB is started, the robot RB starts moving toward the teaching point on the tracking coordinate system Δtr (curved trajectory 9 in FIG. 1).
0). At that time, the position of the interpolation point created by the interpolation operation is
The correction is made based on the data (the amount of deviation from the reference position) read in step R3.

The teaching point position at the time of starting the movement is substantially on the tracking start line 60 as indicated by reference numeral Q0 'in FIG. 3, considering the method of setting the initial position of the tracking coordinate system Δtr. After the start of the movement of the tracking coordinate system Δtr, the robot RB moves the point Q0 ′ to the point where the point Q0 ′ is moved synchronously with the traveling of the conveyor 1 (more precisely, the position corrected by the data of the visual sensor). Moves.

[R7] When the robot RB reaches the teaching point on the tracking coordinate system $ tr, the process proceeds to step R8. [R8] The taught work is executed on the tracking coordinate system $ tr. The robot position during this time is also corrected based on the data (the amount of deviation from the reference position) read in step R3. [R9] The pulse coder count output value N is N = ΔNtr2 -vs
When it reaches + Nsd, the process proceeds to step R10.

[R10] The operation of the robot RB on the tracking coordinate system $ tr is terminated, and the robot RB is moved to the initial position P.
Return to 0. [R11] In order to confirm whether or not tracking of the work detected in one shooting 1 has been completed, it is checked whether there is work data having the same Nsd. For example, 1
If two works are detected in the first shooting,
Even if the tracking of the work on the downstream side is completed and the work data of the work (FIG. 8) is deleted, another work data having the same Nsd remains. In such a case, the processing for one cycle is terminated.

In the case where one work is detected in one photographing 1, when tracking of the work is completed, no work data having the same Nsd remains.
The determination output is no, and the process proceeds to step R12.

Further, even when two or more works are detected in one photographing 1, the step of arriving at the time when the tracking of the most upstream work (the last work detected in the photographing 1) is completed. In R11, the judgment output is also NO, and the process proceeds to step R12.

[R12] q of the local slip amount accumulation register
The data in the oldest row where v = 1 is no longer needed, so clear it.

When the operation cycle for one work is completed in this way, the process returns to step R1, and the processing for the next detected work is started again. As described above, the example in which the local slip amount calculated using the irregular texture image of the conveyor belt 1 is used for the tracking operation correction has been described. As described with reference to FIG. 5, the local slip amount Dk is used. Can be repeatedly obtained in a short cycle from a regular scale, and this can be stored in the same manner as described above and used for tracking operation correction.

[0091]

According to the present invention, the amount of slippage of the conveyor can be easily detected using a visual sensor. Also,
It is possible to continuously detect the local slip amount of the conveyor and correct the tracking operation of the robot in the visual tracking system in real time.

[Brief description of the drawings]

FIG. 1 schematically shows an example of an overall arrangement for implementing the method of the present invention in a visual tracking system.

FIG. 2 is a main block diagram schematically illustrating an internal configuration of a robot controller RC.

FIG. 3 is a diagram illustrating a mark M provided on a conveyor belt.

FIG. 4 is a diagram illustrating a variation in the posture of a work and a maximum length s0 of the work in the transport direction.

FIG. 5 is a diagram illustrating a case where a regular scale SC exists over the entire length of the edge portion of the conveyor belt.

FIG. 6 is a diagram illustrating a case where an irregular texture TX exists over the entire length of the edge portion of the conveyor belt.

FIG. 7 is a flowchart describing the main points of processing 1;

FIG. 8 is a flowchart illustrating a main point of a process 2;

FIG. 9 is a diagram illustrating a storage format of work position / posture data.

FIG. 10 is a diagram illustrating a configuration of a local slip amount accumulation register.

FIG. 11 is a flowchart illustrating a main point of a process 3;

FIG. 12 is a flowchart illustrating a main point of a process 4;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Conveyor 1a, 1b End of conveyor 2 Conveyor drive part 3 Pulse coder 10 Robot control part 11 CPU (robot controller) 12 ROM 13 RAM 14 Nonvolatile memory 15 Axis controller 16 Servo circuit 17 Data memory for digital signal processor (DSP) Reference Signs List 18 digital signal processor (DSP) 19 interface 20 image processing device 21 CPU (image processing device) 22 camera interface 23 frame memory 24 program memory 25 image processing processor 26 data memory 27 monitor interface 30 camera 31 visual field 32 edge area 33 visual field reference Line 40 Monitor CRT 60 Tracking start line 70 Tracking end line 90 Curve trajectory 100 Work source 321 Reference image area BS, BS ', BS "Bus M mark P0 Robot initial position RB Robot RC Robot controller TP Teaching operation panel TX Irregular texture VS Visual sensor W, W', W" Work or work image a, b, c, d Work Features of

Continued on the front page (72) Inventor Taro Arimatsu 3580 Kobaba, Oshino-mura, Oshino-mura, Minamitsuru-gun, Yamanashi Prefecture Inside FANUC Co., Ltd. In company

Claims (7)

[Claims] 1. A mark on a traveling body of a conveyor having position detecting means is repeatedly detected by a camera means of a visual sensor, and the position detection is performed when the mark is detected at a substantially fixed position within a field of view of the camera means. A method for measuring the amount of slip of a conveyor, wherein the output value of the means is analyzed for a number of rounding cycles to determine the amount of slip of the traveling body per one round of the conveyor. 2. The method according to claim 1, wherein an output value of said position detecting means when said mark is detected at a substantially fixed position within a field of view of said camera means is corrected according to a deviation of said mark detecting position from said fixed position. 2. The method for measuring the amount of slip of a conveyor according to claim 1, wherein the method is used to analyze from the following. 3. An analysis of an output value of the position detecting means, wherein the slip amount of the traveling body per one revolution of the conveyor is determined based on a difference between a maximum value and a minimum value of the output value of the position detecting means per one revolution of the conveyor. 3. The method for measuring the amount of slippage of a conveyor according to claim 1, wherein the following formula is obtained. 4. A movement calculated based on an output value of the position detecting means during a short period, wherein a pattern on a traveling body of a conveyor having a position detecting means is repeatedly detected by a camera means of a visual sensor in a short cycle. By comparing the distance and the movement distance of the pattern detected by the visual sensor, to determine the local slip amount of the traveling body,
Conveyor slip amount measurement method. 5. The method according to claim 4, wherein the pattern is an irregular texture. 6. A pattern on a traveling body of a conveyor having position detecting means is repeatedly detected in a short cycle by a camera means of a visual sensor, and is calculated based on an output value of the position detecting means during the short cycle. Comparing the travel distance of the traveling body with the travel distance of the pattern detected by the visual sensor to determine a local slip amount of the traveling body, and continuously accumulating data of the local slip amount, A tracking operation correction method for a robot, wherein the tracking operation of a robot cooperating with the conveyor is corrected based on the accumulated local slip amount data. 7. The method according to claim 6, wherein the pattern is an irregular texture.