JPH07294251A - Obstacle detector - Google Patents

Abstract

PURPOSE:To detect an obstacle efficiently with less computation complexity by recognizing a solid object, detected for the first time in a plurality of dangerous regions around a vehicle, as an obstacle and interrupting the detection of solid object and the processing of the whole image. CONSTITUTION:A solid object extracting means 116 discriminates an obstacle from information on a road and extracts only a solid object. An obstacle detecting means 117 comprises a detection region moving means 20 and a detection stop means 201. The means 200 sections the view field into a plurality of dangerous regions and moves through the detection regions of solid object starting from a maximum likelihood region. Upon detection of a first obstacle by the means 200, means 201 stops operation of the means 200 and presents the results on an alarm or a distance indicator 119. A solid object existing at a most dangerous position in the advancing direction is detected as an obstacle thus finding a danger on the traveling path of vehicle efficiently.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an obstacle warning device for preventing a collision with an obstacle existing around a vehicle, and more particularly, the present invention relates to an image for detecting a danger on a traveling course of a vehicle. The present invention relates to a technique for reducing the amount of recognition calculation.

[0002]

2. Description of the Related Art Some of the obstacle detecting devices described above use a device for measuring the distance between a vehicle and an obstacle. A stereoscopic method is used as a device for measuring this distance. This stereoscopic method is a method of calculating a distance using an image.
In the stereoscopic method, the luminance information from two different viewpoints is used to obtain the positional deviation (parallax) between the objects whose distances are measured for the two luminance information groups, and the distance between the two viewpoints measured in advance, The distance is obtained from the angle of view, the number of brightness information in the angle of view, and the angle of the viewpoint. The details will be described below.

FIG. 8 is a diagram showing a configuration of a conventional distance measuring device based on a general stereoscopic method. The object 11 shown in this figure (a) is an object to be measured. The distance measuring device is provided with two lenses 12 and 13 facing the object 11 to form two viewpoints. These two lenses 1
After 2 and 13, image pickup devices 14 and 15 are provided whose optical axes 16 and 17 match, respectively. Image sensor 1
4 and 15 are, for example, CCD (Charge Coupled Devic)
e). Here, "d" in the figure is the lens 1
2 and 13 is the distance between the object 11 and “f” is the focal length of the lenses 12 and 13, and “a” and “b” are the same points of the object 11 on the image sensors 14 and 15, respectively. It is the distance from the optical axis of the projected position, and "m" is the distance (baseline length) between the optical axes. In this figure (b), the image pickup device on the right side of this figure (a) is moved by the base line length “m”, and is superimposed on the image pickup device on the left side. As shown in this figure (b), the triangle ABC and the triangle ADE are similar to each other. Therefore, the ratio of (a + b) and m becomes the same as the ratio of f and d. In the expression, (a + b): m = f: d, which is transformed into the following expression.

D = fm · (a + b) (1) If m and f are measured in advance and the distance of (a + b) can be measured, the distance d can be derived from the equation (1). This is called stereoscopy. The method of obtaining (a + b) in FIG. 8 is to compare the brightness values of the objects in the left and right images while shifting them little by little,
Find the best matching shift amount. This shift amount is (a +
This is equivalent to b), and this shift amount (a + b) is called parallax p.

Correlation calculation is used as a method for checking the degree of coincidence as a method for obtaining the above-mentioned most coincident shift amount. Regarding the correlation calculation method, the standard method introduced in books such as image recognition will be described below. Another method is an extension of this standard method. To simplify the description, a one-dimensional brightness value sequence will be described first. Figure 9
FIG. 9 is a diagram showing an example of an image formed on the image pickup devices 14 and 15 of FIG. 8. This figure (a) shows the left image by the image sensor 14 (referred to as a "left eye image") and the right image by the image sensor 15 (referred to as a "right eye image").

FIG. 10 is a graph showing an example in which the luminance values of one row indicated by reference numerals 22 and 23 in FIG. 9 are taken out. Reference numerals 24 and 25 shown in the figure are graphs of luminance values of the left-eye image and the right-eye image, respectively. The parallax is the shift amount by which the graph of the left-eye image is shifted to the graph of the right-eye image. FIG. 11 shows the object 21 of FIGS. 10 to 9.
It is a graph which shows the example which took out the part of. The arrangement of the brightness values of the left-eye image indicated by reference numeral 41 in this figure is treated as a sequence b
and the brightness value of the right-eye image with reference numeral 42 is also an
Then, the correlation function of the following formula is calculated.

[0007] Here, n is the pixel number of the image, i is the shift amount of the pixel number, and the shift amount is an integer from 0 to S, and the shift amount is increased by 1 to calculate V (i). S depends on the parallax regarding the shortest distance set in advance as a detection range in the measuring device. w is a width for performing the correlation calculation, and a value is set depending on the purpose. For example, as shown in FIG. 11, since the object 21 corresponds to the pixel numbers 1 to 12 in the reference numeral 42, when w is 10, the best parallax is obtained. However, when this method is actually applied, the position and size of the object on the image are often unknown. Therefore, in general, the value of w that is empirically considered to be the best is adopted.

FIG. 12 is a diagram showing an example of the result of V (i). As shown in the figure, the reference numeral 51 has the smallest correlation value, and i at this time is the parallax. Further, when higher accuracy is required, a point calculated by interpolation calculation like reference numeral 52 is often used as the parallax (there are many interpolation methods). The above description is for the case of using the one-dimensional imaging device or the case of performing the correlation calculation for each row, but in the case of using the two-dimensional imaging device, it is generally not performed for each row. , There is a method of performing the correlation calculation in the area where a plurality of lines are collected.

FIG. 13 is a diagram for explaining the correlation calculation method of the two-dimensional image pickup device. As shown in this figure (a), the first of the methods is a method (projection method) in which the sum is calculated for each vertical column (projection), and the information corresponding to one row is obtained and the above calculation is performed.
Secondly, as shown in this figure (b), there is a method (area method) in which the above equation (2) is directly performed in two dimensions. For the projection method, the one-dimensional calculation described above may be used as it is, but for the area method, V (i) is calculated for each row in the horizontal direction, and the sum at the same i is obtained and evaluated. It Further, since the projection method compresses information in the horizontal direction, it is resistant to vertical displacement of the left and right screens, and the calculation time is short.

[0010]

By the way, in the above obstacle detecting device, when the obstacle is detected using the images acquired by the pair of two-dimensional image pickup devices, the entire image has been processed so far. It was However, since this requires a large amount of calculation, there is a problem that in an emergency, the processing may not be completed in time and may cause trouble.

Therefore, in view of the above problems, it is an object of the present invention to provide an obstacle detecting device capable of efficiently detecting an obstacle.

[0012]

In order to solve the above problems, the present invention provides an obstacle detecting device having the following construction. An obstacle detection device that obtains a forward field of view of the host vehicle by an image sensor and detects an obstacle by extracting a three-dimensional object from an image of the front field of view includes a center of the position of the host vehicle and a traveling direction of the host vehicle. Toward the front, the detection field moving means that divides the forward field of view into a plurality of danger areas, moves the plurality of danger areas outward with the position of the own vehicle as a center, and detects a three-dimensional object in the movement area is provided. To be When the three-dimensional object is first detected in the plurality of dangerous areas, the detection stopping means recognizes the three-dimensional object as an obstacle and stops the detection area moving means from detecting the three-dimensional object.

[0013]

According to the obstacle detecting device of the present invention, the forward field of view is divided into a plurality of dangerous areas centering on the position of the host vehicle and in the traveling direction of the host vehicle, and the position of the host vehicle is determined. The plurality of dangerous areas move outward as a center, and a three-dimensional object is detected in this moving area. When a three-dimensional object is first detected in the plurality of dangerous areas, the three-dimensional object is recognized as an obstacle, and detection of the three-dimensional object by the detection area moving means is stopped, so that the entire image is not processed. The 3D object most desired to be recognized can be detected, the calculation amount can be reduced, and the obstacle can be efficiently detected.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of an obstacle detection device according to an embodiment of the present invention. As shown in the figure, the obstacle detection device is provided with imaging devices 111 and 112. Each of the image pickup devices 111 and 112 has the lens and the image pickup element described in FIG. 8 and outputs an image signal. Each imaging device 111, 11
A / D converters 113 and 114 respectively connected to
(Analog to Digital Converter) converts the image signal obtained by input into a luminance value of 256 gradations, for example. A
The obstacle detection computer 115 connected to the D / D converters 113 and 114 inputs the converted luminance value and outputs the three-dimensional object extraction means 116 for extracting a three-dimensional object using the input luminance value, and three-dimensional object extraction data. Obstacle detecting means 1 for detecting an obstacle based on
17 and parallax calculation means 118 for calculating the distance to the obstacle from the parallax. To the distance calculator 115, a warning for the obtained obstacle, a warning for displaying a distance, or a distance display 119 is connected.

Next, the obstacle detection computer 115 that enables efficient detection of obstacles will be described in detail below. Among obstacles existing in the forward field of view of a vehicle, those which are close to each other and which are present on the course of one's own are the most dangerous obstacles and the objects to be recognized most. Therefore, the obstacle detection computer 115 is configured as follows in order to preferentially detect the object to be most recognized.

First, the obstacle detecting computer 115 is provided with a three-dimensional object extracting means 116 which distinguishes only three-dimensional objects which are obstacles from the information on the road. Here, the three-dimensional object is
It is defined as standing perpendicular to the plane of the road and is described in more detail below. FIG. 2 is a diagram showing an image of a road which can be assumed to be a plane or a two-dimensional plane almost close to a plane, which is captured by placing the image pickup means facing the traveling direction at a certain height from the ground. As shown in the figure, there is an object 1 which is another vehicle in front of the lane of the road on which the vehicle is traveling, and an object 2 which is another vehicle exists in the adjacent lane. Has an object 3 which is a road sign, and an object 4 which is a telephone pole. These objects 1, 2, 3 and 4 are three-dimensional objects that are perpendicular to the plane of the road. The extraction of this three-dimensional object will be described below.

FIG. 3 is a diagram for explaining an example in which a three-dimensional object is extracted by forming a difference image of images obtained by two image pickup devices. As shown in this figure (a), when the images in front of the vehicle are captured by the left and right imaging devices 111, 112, they are superimposed by the three-dimensional object extracting means 116. The solid line is the right image and the broken line is the left image. Here, when the plane of the road is regarded as flat, the parallax of the road surface changes in the vertical direction of the screen, but the parallax of the three-dimensional object does not change and is constant. Therefore, if one image is moved in parallel so as to cancel the road surface difference for each vertical position of the image and then a difference image of both images is created, the information of the plane of the road including the road surface signs is deleted. . On the other hand, as shown in this figure (b), the stereoscopic parallax of the three-dimensional object remains like a diagonal line unlike the road surface parallax, and therefore the area where the three-dimensional object exists remains. The distance to be moved in parallel may be set in advance based on the installation conditions of the device. In this way, the three-dimensional object that is a candidate for an obstacle is easily extracted by removing the information about the plane of the road.

Next, the obstacle detecting computer 115 comprises an obstacle detecting means 117 for detecting an obstacle from the three-dimensional object extracted by the three-dimensional object extracting means 116. The obstacle detecting means 117 divides the field of view into a plurality of dangerous areas and moves the detection area moving means 200 that moves the detection area of the three-dimensional object from the area to be most recognized, and the three-dimensional object first found in the detection area. Detection stopping means 2 for recognizing an obstacle and stopping the detection
01 and. First, the detection area moving means 200 will be described.

FIG. 4 is a diagram for explaining an example of detecting an obstacle from a plurality of three-dimensional objects. As shown in the figure, the distance from the vehicle increases from the lower part of the center of the field of vision toward the periphery of the field of view. That is, the lower part of the visual center is a three-dimensional object close to the own vehicle, and the upper central part of the visual field is a three-dimensional object far from the own vehicle. Since three-dimensional objects existing on the road surface do not float in the air, if three-dimensional objects are detected from the lower part of the field of view toward the upper part, they are detected in order from closer three-dimensional objects, that is, dangerous three-dimensional objects. It will be. In addition, the direction of my movement is from the bottom to the top on the center line, and even if there are three-dimensional objects at the same distance, three-dimensional objects on my path, that is, three-dimensional objects near the center line are more dangerous. is there.

FIG. 5 is a diagram showing an example in which the field of view is divided into a plurality of areas and a three-dimensional object existing in the most dangerous area is detected as an obstacle. FIG. 6 shows a modification of the obstacle detection of FIG. FIG. As shown in FIGS. 5 and 6, in the detection area moving means 200, the distance from the position of the own vehicle and the center line of the own vehicle are used as parameters in order to indicate the degree of danger with respect to the field of view in front of the own vehicle. , For example, the field of view is a plurality of dangerous areas R
It is divided into 1, R2, R3, R4 and R5. The presence of a three-dimensional object is detected by moving from inside to outside of the plurality of dangerous areas, that is, in the order of the dangerous areas R1, R2, R3, R4, and R5. Therefore, three-dimensional objects with high risk,
That is, the three-dimensional object to be recognized is sequentially detected.

Next, the detection area moving means 200 detects the three-dimensional object in a fixed order.
First, the detection area moving means 200 first judges whether or not the obstacle is detected, and when it is judged that the obstacle is detected, the detection processing of the detection area moving means 200 is stopped and the result is displayed on the alarm or the distance indicator 119. Display it. Since the object of the present invention is to efficiently detect the danger on the traveling route of the vehicle, the object can be achieved if the three-dimensional object existing at the most dangerous position in the traveling direction is detected as an obstacle. . That is, the amount of calculation can be reduced and efficiency can be improved.

FIG. 7 is a diagram for explaining a series of operations of the embodiment of the present invention. In step S1, the three-dimensional object extracting unit 116 captures images from the image pickup devices 111 and 112. In step S2, the imaging devices 111 and 112
The right image and the left image from are moved in parallel to form a difference image between the two images, the plane image of the road is erased, and only the image of the three-dimensional object is extracted.

In step S3, the detection area moving means 2
At 00, the dangerous area R is set to 1 and detection of a three-dimensional object is started. In step S4, the detection stopping means 2
In 01, it is judged whether or not a three-dimensional object, that is, a three-dimensional object exists in the dangerous area R = 1. In step S5, if the above determination is "YES", and if there is a three-dimensional object, it is determined as an obstacle and the detection of the detection area moving means 200 is stopped.

In step S6, the warning "distance is present" is issued by the alarm or the distance display 119. In step S7, the distance to the obstacle is calculated by the above equation (1). In step S8, the above distance is displayed by the alarm or the distance display 119, and the process ends.

In step S9, if the determination in step S4 is "NO", that is, if there is no three-dimensional object, it is determined whether or not the dangerous area R = Rmax is satisfied. This judgment is "YES"
If so, there is no obstacle in any of the dangerous areas in the field of view in front of the own vehicle, so the processing ends. If the determination is “NO” in step S10, R is counted up, and the detection area moving means 200 continues the search for the presence or absence of an obstacle in the next dangerous area. Therefore, the process returns to step S4. The above process is repeated.

[0026]

As described above, according to the present invention, the forward field of view is divided into a plurality of dangerous areas centering on the position of the host vehicle and in the traveling direction of the host vehicle, and the position of the host vehicle is centered. As a result, a plurality of dangerous areas move outward, and a three-dimensional object is detected in this moving area.If a three-dimensional object is first detected in a plurality of dangerous areas, the three-dimensional object is recognized as an obstacle and a three-dimensional object is detected. Since detection is stopped, you can detect the three-dimensional object you want to recognize most without performing processing on the entire image,
The calculation amount can be reduced, and the obstacle can be detected efficiently.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an obstacle detection device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an image of a road that can be assumed to be a plane or a two-dimensional plane that is almost plane and is captured by placing an image pickup unit facing the traveling direction at a certain height from the ground.

FIG. 3 is a diagram illustrating an example of extracting a three-dimensional object by forming a difference image of images obtained by two imaging devices.

FIG. 4 is a diagram for explaining an example of detecting an obstacle from a plurality of three-dimensional objects.

FIG. 5 is a diagram showing an example of dividing a field of view into a plurality of areas and detecting a three-dimensional object existing in the most dangerous area as an obstacle.

FIG. 6 is a diagram showing a modification of the detection of an obstacle in FIG.

FIG. 7 is a diagram illustrating a series of operations according to the embodiment of the present invention.

FIG. 8 is a diagram showing a configuration of a conventional general stereoscopic distance measuring device.

9 is a diagram showing an example of an image formed on the image pickup devices 14 and 15 of FIG.

10 is a graph showing an example in which the luminance values of one row of reference numerals 22 and 23 of FIG. 9 are extracted.

FIG. 11 is a diagram showing an example in which a portion of the object 21 of FIGS. 10 to 9 is taken out.

FIG. 12 is a diagram showing an example of a result of V (i).

FIG. 13 is a diagram illustrating a correlation calculation method of the two-dimensional imaging device.

[Explanation of symbols]

 111, 112 ... Imaging device 115 ... Obstacle detection computer 116 ... Three-dimensional object extraction means 117 ... Obstacle detection means 118 ... Parallax calculation means 119 ... Warning or distance indicator 200 ... Detection area moving means 201 ... Detection stopping means

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G08B 21/00 H G08G 1/16 C

Claims (3)

[Claims] 1. An obstacle detection device for acquiring a front field of view of an own vehicle by an image sensor, extracting a three-dimensional object from an image of the front field of view, and detecting an obstacle, the center of the position of the own vehicle and the own vehicle. In the traveling direction, the front view is divided into a plurality of dangerous areas, and the plurality of dangerous areas are moved outward with the position of the own vehicle as a center to detect a three-dimensional object in the moving area. A moving means and a detection stopping means for recognizing the three-dimensional object as an obstacle when the three-dimensional object is first detected in the plurality of dangerous areas and stopping the detection of the three-dimensional object by the detection area moving means. Characteristic obstacle detection device. 2. The obstacle detection device according to claim 1, wherein the obstacle detection device warns a result of the obstacle detection. 3. The obstacle detection device according to claim 1, wherein the detected distance to the obstacle is calculated based on parallax.