JP3215507B2 - Distance measuring device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distance measuring apparatus for automatically measuring a distance to a subject by applying the principle of stereoscopic vision to two subject images picked up at different positions.

[0002]

2. Description of the Related Art Conventionally, a distance measuring apparatus to which such a principle of stereoscopic vision is applied is provided with a pair of left and right video cameras or the like corresponding to two eyes, and two video cameras obtained by imaging each video camera from different positions. A pair of points corresponding to each other on the subject image is obtained, and based on these points (hereinafter referred to as corresponding points), 3
By applying the principle of angle measurement, coordinates in a subject image are determined, and the distance to the subject is measured.

Further, the principle of distance measurement will be described with reference to FIG. 4. First, two video cameras are installed at the same height, and their optical axes Za and Zb are set so as to be parallel to the same direction. You. With this configuration, the lens centers Oa and Ob of the two video cameras, a point Ps in the scene, and the imaging points Pa and Pb in the image planes A and B of each video camera are the same. Exists on the epipolar plane EP, and further forms an image P
“a” and “Pb” always exist on the intersection lines (epipolar lines) La and Lb of the epipolar plane EP and the image planes A and B.
Therefore, corresponding points Pa and Pb are searched between the pair of left and right epipolar lines La and Lb, the distance between the lens centers Oa and Ob is 2L, the focal length of both video cameras is f, and X in the figure is X.
Assuming that the distance between the corresponding points Pa and Pb in the coordinates a and Xb is d, the distance z in the optical axis Za and Zb directions to the point Ps
Is calculated by the following equation (1).

[0004]

(Equation 1)

[0005] Such a distance measuring device exhibits visual functions such as measuring the distance to various objects in a three-dimensional space and recognizing the convex and concave portions of the object. Driving robots, autonomous driving cars, industrial robots that transport, inspect, and assemble mechanical parts in factories, measuring devices that measure the volume of objects, electron microscopes that measure three-dimensionally, and measure the shape of the human body It is applicable to a wide range of fields such as medical equipment.

[0006]

As is apparent from the above equation (1), in such a distance measuring apparatus, the accuracy of searching for a corresponding point directly affects the distance measuring accuracy. Further improvement in search accuracy is desired.

An example of the corresponding point searching means in the conventional distance measuring apparatus will be described. For example, in the case of measuring the distance of an object having a curved curved surface whose central portion is convex, as shown in FIGS. The brightness level of each pixel of a pair of frame images obtained by imaging with a video camera is curved (for convenience of description, the brightness level of each pixel of one horizontal scanning line is shown).
And the luminance level of each pixel is set to a predetermined threshold value I
And performs a binarization process in which a pixel having a higher luminance than the threshold value I is logic “1” and a pixel having a lower luminance is logic “0”. When binarization is performed based on the predetermined threshold value I, image planes A and B
6 (a) and 6 (b), the equal luminance lines Ka and K with the threshold I luminance as the outline of the image.
b, and the intersections Xa1 and Xb1 between these equal luminance lines Ka and Kb and the epipolar lines La and Lb can be set as a set of corresponding points, and the intersections Xa2 and Xb2 can be set as another set of corresponding points.
The distance is obtained by performing the calculation of the above equation (1) for these corresponding points.

However, it is an ideal case that clear equi-luminance lines Ka and Kb as shown in FIGS. 6A and 6B are obtained, as shown in FIGS. 5A and 5B. If the S / N of the luminance level of each pixel is poor, quantization noise is generated at the boundary corresponding to the threshold value I as shown in FIGS. 7A and 7B, and the intersection Xa1 with the epipolar lines La and Lb is generated. , Xb1, Xa
2, Xb2 is not uniquely specified. As a result, the search accuracy of the corresponding point is reduced, and the distance measurement accuracy is reduced.

In order to solve this problem, a plurality of frame images obtained by each video camera taking an image in each frame scanning period are integrated for each pixel and then averaged by the number of integrations. In some cases, a method of creating evaluation images smoothed by the above method and binarizing these evaluation images with a predetermined threshold value I for obtaining equiluminance lines may be taken. That is, in this smoothing method, each pixel array of each frame image is i, the order of a plurality of frame images is j, the luminance level of each pixel is di _{, j} , and the total number of frame images to be integrated is N. For example, this means that the smoothed luminance level D _{i, j} of each pixel is obtained by the operation of the following equation (2), and is smoothed and quantized as compared with the case shown in FIGS. Since an evaluation image with reduced noise is obtained, the accuracy of searching for an equal luminance line with respect to the threshold value I is improved.

[0010]

(Equation 2)

However, there is a limit even if the luminance level of each pixel is smoothed and the boundary with respect to the predetermined threshold value I is sharpened as described above.
In order to completely remove the quantization noise at the boundary portion as shown in (1), it is necessary to repeat the integrated averaging operation for an extremely large number of frame images. That is, the convergence until the quantization noise was removed was not good. In addition, there is a problem that a large-capacity memory is required because the so-called raw pixel data obtained from the video camera needs to be integrated many times, which leads to an increase in the size of the apparatus. If it is attempted to realize a distance measuring apparatus under some restrictions as described above, it has been difficult to achieve sufficiently satisfactory corresponding point search accuracy and distance measurement accuracy.

The present invention has been made in view of such a conventional problem, and a distance measuring apparatus capable of improving the accuracy of searching for a corresponding point and, consequently, improving the distance measuring accuracy. The purpose is to provide.

[0013]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides a pair of image pickup devices having an image pickup optical system having the same focal length and the optical axis directed in the same direction.
A first method of forming binarized pixel data by binarizing each pixel data of an image obtained by imaging of each imaging device at a predetermined cycle based on a threshold for setting an equal luminance line A binarizing means, an integrating operation means for forming the first evaluation data by integrating the binarized pixel data for each pixel of the image, and a new first calculating means for each of the predetermined periods; A second binarizing unit for forming the second evaluation data by binarizing the first evaluation data based on a predetermined threshold every time the evaluation data is formed; The matching between the obtained second evaluation data and the currently obtained second evaluation data is determined based on a predetermined condition. When the condition is satisfied, the currently obtained second evaluation data is compared with the corresponding point extraction pixel data. Comparing means and corresponding point extraction Calculating means for obtaining an isoluminance line from the pixel data, obtaining coordinates of a corresponding point where the isoluminance line and the epipolar line intersect, and calculating a distance to the subject from the coordinates based on the principle of stereoscopic vision. And

[0014] Further, the above configuration further includes control means for changing the threshold value of the first binarizing means when the distance is obtained by the arithmetic means, and restarting the distance measuring operation based on the threshold value. The configuration was adopted.

[0015]

According to the distance measuring apparatus having such a configuration, binarized pixel data is formed from each pixel data of an image.
The first evaluation data is formed by integrating the valued pixel data, and the first evaluation data is further binarized based on a predetermined threshold to form second evaluation data from which unnecessary quantized data is selectively removed. Then, by judging the coincidence of the second evaluation data before and after the predetermined cycle to determine whether or not the quantization noise has been removed, the corresponding point extraction pixel data from which the quantization noise has been removed is obtained. Then, a distance is obtained from the corresponding point extraction pixel data by a calculation based on the principle of stereoscopic vision. Therefore, it is technically different from binarizing the pixel data of a plurality of frame images after integrating and averaging the pixel data as in the related art. Since the pixel data determined to have a logic “1” or “0” is integrated, the integration operation is performed. The convergence up to the removal of quantization noise is improved, the smoothing of the outline of the image is completed, and the image for corresponding point extraction is finally formed based on a predetermined threshold, so that a clear equal luminance A line and a corresponding point specified are determined with high accuracy, and highly accurate ranging can be performed.

[0016]

An embodiment of the present invention will be described below with reference to the drawings. First, the overall configuration of the apparatus will be described with reference to FIG. A pair of video cameras 1a and 1b for capturing an image of a subject are provided at a predetermined distance from each other, the focal lengths f of the imaging optical systems are equal to each other, and further, the optical axis directions are parallel and directed toward the subject. It has become. Each of the video cameras 1a and 1b has a field scan period (1/60 seconds) and a frame scan period (1/30 seconds) of the NTSC system, for example.
Second) and other standard television systems.
Since the respective signal processing systems connected in cascade to the respective video cameras 1a and 1b have the same configuration, the parts corresponding to each other will be described together.

A / D converters 2a and 2b convert analog luminance signals Sa and Sb for each pixel, which are output by the video cameras 1a and 1b in synchronization with the so-called point-sequential scanning timing, into digital pixel data. , First binarizing circuit 3
a, 3b are compared with a first threshold value IV set by the reference value generation circuit 16 and each pixel data. When the luminance level of the pixel data is higher than the threshold value IV, the logic is “1”, and the luminance level is equal to the threshold value IV. If it is smaller, a binarization process for generating binarized pixel data a _{i, j} and b _{i, j} for each pixel which becomes logic “0” is performed. That is, as shown in FIG. 2A, by comparing the raw pixel data with the threshold value IV, binarized binary pixel data as shown in FIG. 2B is formed. , Quantization noise is included in the contour portion of the image corresponding to the threshold value IV. The binarized pixel data a
The attached symbol i of _{i, j} and bi _{, j} indicates the order of arrangement of each pixel, and the symbol j
Indicates the order of the frame images to be processed.

The integrating circuits 4a and 4b provide feedback evaluation data A transferred from the image memories 6a and 6b to the binarized pixel data a _{i, j} and b _{i, j} via the gate circuits 5a and 5b.
_{i, j-1} and B _{i, j-1} are integrated and the first result is obtained.
The evaluation data A _{i, j} and B _{i, j} are stored in the image memories 6a and 6b. That is, each of the image memories 6a and 6b is composed of a random access memory (RAM) or the like having a storage capacity for storing the first evaluation data of one frame fraction, and the binarization circuits 4a and 4b perform the binarization operation processing. Pixel data a
The first evaluation data A _{i, j-1} and B _{i, j-1} one frame scanning cycle before stored in the storage area corresponding to the same arrangement as the arrangement in the frame image of _{i, j} and b _{i, j} are The data is output as feedback evaluation data, and the current first evaluation data A _{i, j} and B _{i, j} accumulated by the integration circuits 4a and 4b are stored again in the same storage area. Therefore, for example, when the processing is performed over the M frame scanning cycle, the first evaluation data A _{i, j} and B _{i, j} obtained by the integration operation of the following equation (3) are stored in the image memories 6a and 6b. It has become.

[0019]

(Equation 3)

The gate circuits 5a and 5b are composed of, for example, analog switches, etc.
The feedback evaluation data A _{i, j-1} and B become conductive or non-conductive in accordance with the processing continuation or stop command from b.
Transfer or stop transfer of _{i, j-1} .

The second binarizing circuits 7a and 7b are provided with first evaluation data A _{i, j} and B output from the integrating circuits 4a and 4b _, respectively.
_{i, j} is determined by the order j of the frame images (j / 2) −
The first evaluation data A _{i, j} and B are compared with the threshold values THa1 and THb1 obtained by the calculation of 0.5.
_{If i and j} are large, logic "1" and threshold values THa1 and THb1
If the value is smaller than the value, the second evaluation data α becomes logic “0”.
Output _{i, j} and β _{i, j} . That is, as shown in FIG. 2C _, the maximum value of the first evaluation data A _{i, j} and B _{i, j} obtained by the integration circuits 4a and 4b is equal to the number j of integration operations, and
Since the minimum value is 0 and the quantization noise is an intermediate value between them, the binarization circuits 7a and 7b set the threshold values THa1 and THb1 as described above with respect to the number of integration operations j. The second evaluation data α _{i, j} and β _{i, j} of logic “1” or “0” as shown in FIG. Thus, the first evaluation data A _{i, j} and B _{i, j} are determined based on the threshold values THa1 and THb1.
Is binarized, quantization noise that does not reach the threshold values THa1 and THb1 is selectively removed.

The order j of the frame images is determined by the counter 8
a and 8b count respectively, and based on the count value j, the binarization circuits 7a and 7b obtain thresholds THa1 and THb1, and perform binarization processing.

The luminance image memories 9a and 9b store the second evaluation data α _{i, j} output from the second binarization circuits 7a and 7b.
And β _{i, j} are stored in the storage area of the array corresponding to each pixel array of the frame image.

The comparison circuits 10a and 10b are stored in storage areas corresponding to the pixel arrays of the second evaluation data α _{i, j} and β _{i, j} output from the second binarization circuits 7a and 7b. The second evaluation data α _{i, j-1} and β _{i, j-1} one frame period before
Is read out from the luminance image memories 9a and 9b.
Of the second evaluation data α _{i, j-1} and β _{i, j-1} before the frame period and the latest second evaluation data α _{i, j} and β _{i, j} output from the binarization circuits 7a and 7b. By performing a division operation, sums SA and SB of division results for all pixels of one frame image
Ask for. If the sum SA, SB exceeds the predetermined threshold THa2, THb2, the counter 8
a, 8b and the gate circuits 5a, 5b to instruct the continuation of the processing, and the sums SA, SB are respectively set to predetermined threshold values THa2, Tha.
When it becomes less than Hb2, a command to stop the processing is issued.

That is, the comparison circuits 10a and 10b repeat the operation of the following equation (4) every one frame scanning cycle, and
When A and SB are less than the respective predetermined threshold values THa2 and THb2, the corresponding point extraction pixels from which uncertain and unnecessary quantization noise as shown in the boundary portions in FIGS. It is determined that the data is held in the luminance image memories 9a and 9b, and the processing is stopped. Note that the threshold values THa2 and THb2 are set according to the desired measurement accuracy, and the accuracy can be increased by reducing these threshold values.

[0026]

(Equation 4)

In the signal processing system relating to the video camera 1a and the signal processing system relating to the video camera 1b, the respective comparison circuits 10a and 10b perform the above-described comparison / judgment processing independently of each other. Therefore, even if one of the signal processing systems obtains corresponding point extraction pixel data quickly, the other signal processing system continues processing until corresponding point extraction pixel data from which quantization noise has been removed is obtained. It is supposed to.

When the corresponding point extracting pixel data is stored in the luminance image memories 9a and 9b of both signal processing systems, the corresponding point detecting section 11 connects the equal values of the corresponding point extracting pixel data by connecting them. The isoluminance line is obtained, and the intersection of the epipolar lines La and Lb of each of the video cameras 1a and 1b described with reference to FIG. 4 and these isoluminance lines is set as a corresponding point, and the coordinates of the corresponding point are determined. For example, corresponding point extraction pixel data α _{i, j} representing an isoluminance line as indicated by Ka1 in FIG. 3A is stored in the luminance image memory 9a, and Kb1 in FIG. 3B is stored in the luminance image memory 9b. Assuming that corresponding point extraction pixel data β _{i, j} representing isoluminance lines as shown in FIG. 3 are stored, the intersections Xa1 and Xa1 of the respective epipolar lines La, La
b1 is a corresponding point, and intersections Xa2 and Xb2 are other corresponding points.

The distance calculation unit 12 specifies the subject corresponding to the corresponding point by performing a calculation based on the triangulation principle shown in FIG. 4 for the corresponding points Xa1, Xb1, Xa2, and Xb2 obtained in this way. The distance to the portion is calculated and stored in the distance image memory 13.

The distance storage memory 13 stores distance data in a storage area corresponding to the coordinates of the specific part of the subject.
The video drive circuit 14 causes the monitor 15 to display the distance data stored in the distance storage memory 13 as an image.

When the distance measurement based on a certain threshold value IV is completed, the reference value generation circuit 16 changes the threshold value to another threshold value IV and starts the distance measuring operation again. That is, the threshold IV
Are for setting the luminance level of the isoluminance line, and by changing the value of the threshold value IV, the isoluminance lines Ka2 and Kb2 of other luminance levels as shown in FIGS. Since Xa3, Xb3, Xa4, and Xb4 can be obtained, the above process is repeated while changing the threshold value IV to sequentially obtain the distance to another portion of the subject, thereby performing a process such as three-dimensionally measuring the subject. I do.

As described above, according to this embodiment, the pixel data obtained from the image pickup device is binarized based on the threshold value IV for obtaining the equal luminance line, and the binary value is converted to the logic "1" or "0".
Since equalization lines are obtained by repeating integration and binarization on the binarized pixel data that has been converted, smoothing is performed, quantization noise is rapidly removed, and luminance contours corresponding to the equalization lines are obtained. Is quickly converged to a specific shape. As a result, the time required for the distance measurement processing is greatly reduced, and at the same time, the accuracy of searching for the corresponding point is improved, and the accuracy of the distance measurement is improved.

That is, the conventional method of searching for a corresponding point uses raw pixel data obtained from an image pickup apparatus (pixel data of an intermediate value level instead of a value fixed to logic "1" or "0" as in this embodiment). ) Is to remove the quantum noise by performing the integral averaging operation, and thus the convergence until the quantum noise is removed is poor. However, the present embodiment can significantly improve this point.

Furthermore, since the arithmetic processing is first performed on the binarized pixel data after the binarization, the storage capacities of the image memories 6a and 6b and the luminance image memories 9a and 9b can be greatly reduced. Can be simplified, so that the size of the device can be reduced. Furthermore, since the number of bits of each data provided for the arithmetic processing is reduced in this manner, a simple logic circuit can be applied without performing the arithmetic processing by program control using a microprocessor or the like. High-speed processing becomes possible.

Further, since the comparators 10a and 10b perform determination control to repeat the integration operation until the quantization noise is eliminated, the accuracy of searching for the corresponding point is further improved.

Furthermore, since the reference value generation circuit 16 changes the threshold value IV to obtain equal luminance lines having different luminance levels, it is possible to perform processing such as three-dimensional measurement of unevenness and the like of a subject.

In general, when an image of a subject is taken, not only the light from the subject itself but also specular reflection light (light in which illumination light and the like are totally reflected by the subject) is taken. The specular reflection light may be removed by providing a polarizing filter in the optical system 1b. By taking such measures, the search for the corresponding point and the distance measurement can be realized with higher accuracy.

Further, in this embodiment, the case where a video camera for capturing a moving image is applied has been described, but an electronic still camera for capturing a still image can also be applied. That is, any imaging device that can electronically capture images can be applied.

[0039]

As described above, according to the present invention, binarized pixel data is formed from each pixel data of an image obtained by imaging at predetermined intervals, and the binarized pixel data is integrated to form the first pixel data. Forming evaluation data, further binarizing the first evaluation data based on a predetermined threshold to form second evaluation data from which unnecessary quantized data is selectively removed, and forming second evaluation data before and after a predetermined period By determining whether or not quantization noise has been removed by determining data consistency,
Since pixel data for corresponding point extraction from which quantization noise has been removed is obtained, it is technically different from binarization after integrating and averaging pixel data of a plurality of frame images as in the related art. The convergence up to the removal of the noise is improved, and the smoothing of the outline portion of the image is completed. Furthermore, since a corresponding point extraction image is finally formed based on a predetermined threshold value, a clear isoluminance line and a specified corresponding point are determined with high accuracy, and highly accurate ranging can be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is an explanatory diagram for explaining an operation of one embodiment.

FIG. 3 is an explanatory diagram for further explaining the operation of one embodiment.

FIG. 4 is an explanatory diagram for explaining the principle of distance measurement.

FIG. 5 is an explanatory diagram for explaining a conventional technique.

FIG. 6 is an explanatory diagram for further explaining a conventional technique.

FIG. 7 is an explanatory diagram for explaining a problem of the related art.

[Explanation of symbols]

1a, 1b video camera, 2a, 2b A / D converter, 3a, 3b first binary circuit, 4a, 4b integration circuit, 5a, 5b gate circuit, 6a, 6b image memory, 7a, 7a: second binarizing circuit, 7a, 7b: counter, 9a, 9b: luminance image, 10a, 10b: comparing unit, 11: corresponding point detecting unit, 12: distance calculating unit, 13: distance image memory , 14 ... video drive circuit, 15 ... monitor, 1
6 reference value generation circuit.

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G01C 3/00 G02B 7/00

Claims (2)

(57) [Claims] 1. A pair of image pickup devices having an image pickup optical system having an equal focal length and an optical axis directed in the same direction, and each pixel data of an image obtained by image pickup of each image pickup device at a predetermined cycle. A first binarizing means for forming binarized pixel data by binarizing based on a threshold for setting an equal luminance line, and integrating the binarized pixel data for each pixel of an image An integration operation means for forming first evaluation data by processing; and each time the integration operation means forms new first evaluation data for each of the predetermined periods, the first evaluation data is based on a predetermined threshold value. A second binarizing unit that forms second evaluation data by binarizing; and a second binarization unit that determines whether the second evaluation data formed by the imaging before the predetermined cycle and the second evaluation data obtained at the present time are coincident with each other. Judge based on predetermined conditions, Is satisfied, the second evaluation data obtained at the present time is used as corresponding point extraction pixel data, and a comparison means is used to obtain an equiluminance line from the corresponding point extraction pixel data, and the correspondence between the equiluminance line and the epipolar line is determined. Find the coordinates of the point,
Calculating means for calculating the distance to the subject from the coordinates based on the principle of stereoscopic vision. 2. A pair of image pickup apparatuses having an image pickup optical system having the same focal length and the optical axis directed in the same direction, and each pixel data of an image obtained by image pickup of each image pickup apparatus at a predetermined cycle. A first binarizing means for forming binarized pixel data by binarizing based on a threshold for setting an equal luminance line, and integrating the binarized pixel data for each pixel of an image An integration operation means for forming first evaluation data by processing; and each time the integration operation means forms new first evaluation data for each of the predetermined periods, the first evaluation data is based on a predetermined threshold value. A second binarizing unit that forms second evaluation data by binarizing; and a second binarization unit that determines whether the second evaluation data formed by the imaging before the predetermined cycle and the second evaluation data obtained at the present time are coincident with each other. Judge based on predetermined conditions, Is satisfied, the second evaluation data obtained at the present time is used as corresponding point extraction pixel data, and a comparison means is used to obtain an equiluminance line from the corresponding point extraction pixel data, and the correspondence between the equiluminance line and the epipolar line is determined. Find the coordinates of the point,
Calculating means for calculating the distance to the subject from the coordinates based on the principle of stereoscopic vision;
Control means for changing the threshold value of the value conversion means and restarting the distance measurement operation based on the threshold value.