JPH1011585A - Object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect an object, especially the solid object of the other vehicle from an image around the vehicle, which is obtained by the image pickup means of a CCD camera or the like, in short time. SOLUTION: The image obtained by on-vehicle CCD cameras 10a and 10b is supplied to an image processing computer 12. The image processing computer 12 detects a white line from the inputted image and sets the travel lane of its own vehicle, which is surrounded by the white line, as a object processing area. The optical flow of the moving object is calculated in the set area and it is judged to be solid or not by stereotype range finding. A detected result is outputted to ECU16 for automatic driving. When the solid object (other vehicle) exists, respective actuators 22-26 are driven and inter-vehicle control is executed and cruise control is executed when a non-solid object such as a road sign exists.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object detecting apparatus, and more particularly to an apparatus for detecting a three-dimensional object on a track.

[0002]

2. Description of the Related Art Conventionally, techniques for detecting objects around a vehicle, particularly three-dimensional objects, with a camera have been developed. For example,
Japanese Patent Application Laid-Open No. 5-26547 discloses a technique in which the front of a vehicle is imaged by a stereo optical system, and a three-dimensional object and a road shape are separated and detected from the obtained three-dimensional position information of the subject.

[0003]

However, in the above prior art, in order to recognize a three-dimensional object particularly at a distant place, it is necessary to obtain an image with a considerably fine resolution in terms of distance accuracy, and the cost of imaging means and the like is low. There is a problem that leads to an increase. In addition, since the three-dimensional position information is calculated for the entire acquired screen, there is a problem that it takes time for the processing. In particular, when considering the use for vehicles, real-time processing is important. Therefore, such an increase in the processing time is not desirable, and an apparatus capable of detecting a three-dimensional object in a short time without increasing the cost is desired. ing.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems of the related art, and has as its object to provide an apparatus capable of detecting an object (particularly, a three-dimensional object) in a short time and reliably. is there.

[0005]

In order to achieve the above object, a first aspect of the present invention is an on-vehicle imaging means for photographing a surrounding image, and an optical flow showing a moving object relative to a vehicle in the obtained image. Area setting means for setting an area from which a vector is to be extracted; vector extraction means for extracting an optical flow vector from image data of the set area in a time-series image obtained by the imaging means; and an optical flow vector group Distance calculating means for calculating the distance to each of the two upper and lower constituent areas of the image area from which is extracted, and identifying means for identifying the moving object by comparing the obtained distance between the two upper and lower constituent areas. It is characterized by. As described above, instead of processing all data in a captured image, a specific area is set in advance, and processing is performed only in the specific area, so that processing time can be reduced. In addition, optical flow is
Means the relative velocity vector of the pixel in the video,
The specific area is desirably set on an area where the possibility of the presence of the object is high, for example, on a lane in which the host vehicle is traveling.

According to a second aspect of the present invention, there is provided an on-vehicle image pickup means for photographing a surrounding image, and an area setting means for setting an area from which an optical flow vector indicating a moving object relative to a vehicle is to be extracted in the obtained image. And a vector extracting means for extracting an optical flow vector from image data of a set area in the time-series image obtained by the imaging means, and an image area from which an optical flow vector group is extracted based on the vector amount. Further, by comparing the dividing means for dividing into the small areas, the distance calculating means for calculating the distance to each of the two upper and lower constituent areas in each divided small area, and the obtained distance between the two upper and lower constituent areas, Identification means for identifying a moving object in each small area. A stationary object such as a sign on the road also generates an optical flow when the own vehicle is moving.However, since the amount of optical flow differs from that of a moving object such as another vehicle, the stationary object is Moving objects can be reliably distinguished. Also in the present invention, the processing time is shortened because only the specific area where the optical flow is considered to be generated is processed, not the processing of the entire image.

[0007]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows an overall configuration diagram of the present embodiment. The vehicle includes a pair of CCD cameras 10a and 1
0b is provided at a predetermined distance, and photographs ahead of the vehicle. The arrangement of the CCD cameras 10a and 10b may be vertical or horizontal. The images obtained by the CCD cameras 10a and 10b are supplied to an image processing computer 12. The image processing computer 12 has an A / D converter, a CPU, and a memory. The image processing computer 12 performs a predetermined process on the input image, and performs white line information, information on the presence or absence of an obstacle, and the presence of a preceding vehicle when there is a preceding vehicle. The inter-vehicle distance to the vehicle is calculated and supplied to the automatic driving ECU 16. The processing in the image processing computer 12 will be described later. ECU 16 for automatic driving
Is constituted by a microcomputer, the inputted information and the cruise setting data from the cruise switch 14, the turn signal data from the turn signal 18 and the wheel speed sensor 2.
Automatic driving is executed based on vehicle speed data from zero. Specifically, when there is a preceding vehicle, the control shifts to inter-vehicle control to follow while maintaining the following distance to the preceding vehicle, and when there is no preceding vehicle or other three-dimensional obstacle, the predetermined vehicle speed is maintained. To move to cruise control with constant speed
Controls various actuators. The actuators to be controlled are, for example, a steering actuator (ACT) 22, a throttle actuator (ACT) 24, and a brake actuator (ACT) 26.

FIG. 2 shows a flowchart of the entire processing executed by the image processing computer 12 and the automatic driving ECU 16. The automatic driving ECU 16 controls the steering actuator 2 based on the white line information supplied from the image processing computer 12 so as to travel along the white line.
2 to perform steering control (S101). Then, during the execution of the steering control, it is determined whether or not the preceding vehicle exists (S102). When the image processing computer 12 detects the preceding vehicle, the determination is YES and the process proceeds to the headway control (S103). When the preceding vehicle is not detected and the determination is NO, the process proceeds to the cruise control (S104). ).
Before moving to S102, it is determined whether there is a three-dimensional obstacle other than the vehicle.
After giving a warning to the driver without shifting to the process of step 2, the brake actuator 26 may be driven to perform stop control.

FIG. 3 shows a detailed flowchart of the steering control processing in S101 in FIG. In this steering control, first, the white line position is detected by the image processing computer 12 (S201). The position of the white line is the CCD camera 10
The images obtained in steps (a) and (10b) can be detected by performing binarization processing to detect luminance edges, or by matching with a predetermined white line template. Since the detected white line information is supplied to the automatic driving ECU 16, the automatic driving ECU 16 calculates the displacement of the vehicle from the white line, and determines the steering amount based on this displacement so as to travel substantially in the center of the lane. To perform steering control (S
202). Specifically, when the displacement amount is δ, the corrected steering amount may be determined by the sum of the amount proportional to the displacement amount and the amount proportional to the derivative of the displacement amount.

FIG. 4 shows a detailed flowchart of the headway control process in S103 in FIG. First, when the image processing computer 12 detects a preceding vehicle,
The inter-vehicle distance to this vehicle is detected (S301). In order to detect the inter-vehicle distance, a pair of CCD cameras 10a and 10b can be used as a stereo optical system to detect the distance based on the principle of triangulation. Specifically, the CCD camera 1
When the focal length of 0a and 10b is f, the attachment interval is r, and the parallax is ε,

## EQU1 ## The distance may be calculated by the following formula: L = rf / ε (1) (hereinafter referred to as stereo ranging). Data on the distance to the vehicle ahead is EC for automatic driving
U16, the automatic driving ECU 16 controls the throttle and brake actuators 24 and 26 such that the inter-vehicle distance matches a predetermined inter-vehicle distance (S302).

FIG. 5 is a detailed flowchart of the vehicle determination process of S102 in FIG. 2, which is a characteristic process of the present embodiment. First, the image processing computer 12 recognizes a white line from the front images obtained by the CCD cameras 10a and 10b by the above-described method (S401). After recognizing the white line, the image area surrounded by the white line is determined as the traveling lane of the own vehicle (S402). The image area determined in this way becomes a processing target of the optical flow calculation, and no processing is thereafter performed on the image area other than the own vehicle lane. Thus, the reason that only the traveling lane of the own vehicle is targeted is that even if the object exists in a region other than the traveling lane of the own vehicle, there is almost no obstacle to the traveling of the own vehicle. In many cases, the vehicle is low, and the existing object is another vehicle. This is because the vehicle exists on the traveling lane. After setting the area to be processed, the image processing computer 12
Calculates an optical flow in a set area from two temporally consecutive images (S403). The optical flow can be extracted by associating two consecutive images using a template of an appropriate size and calculating the displacement (matching method). It goes without saying that the template uses the image data in the setting area. After calculating the optical flow, a moving object region is extracted using the optical flow (S4).
04), and further performs stereo ranging of the moving object area (S)
405).

FIG. 6 schematically shows the moving object extraction processing and the stereo distance measurement processing in S404. FIG. 6A is an image obtained by a CCD camera, and it is assumed that a vehicle ahead is present. Processing target is white line 1
The area surrounded by 00 is indicated by reference numeral 200 in FIG.
This area is indicated by. When the optical flow is calculated within the area, the optical flow of the preceding vehicle is calculated based on the relative speed of the preceding vehicle to the own vehicle. Parts other than the preceding vehicle (for example, the road surface) will actually have a relative speed to the own vehicle (since the road surface is stationary), but since the image has no contrast,
The optical flow itself is undefined. In FIG. 6B, the optical flow in the undefined state is represented by a black dot. When the preceding vehicle is running at the same speed as the own vehicle, the relative flow is 0 and the optical flow is also 0. This is because the two-dimensional component is (0, 0).
And is distinguished from the undefined state. FIG. 6B shows a case where the preceding vehicle is accelerating with respect to the own vehicle, and an optical flow having a size corresponding to the relative speed is calculated. When an optical flow is calculated in the set area, optical flows having the same vector amount are grouped into one group, which is set as a moving object area. In FIG. 6C, the moving object region extracted in this manner is indicated by reference numeral 300. That is, the moving object region means a set of optical flow vectors having the same or similar vector amount. After the moving object region is extracted, the moving object region is divided into two upper and lower constituent regions on the screen, and stereo ranging for each of the upper and lower constituent regions is performed. In FIG. 6D,
The processing at the time of stereo ranging is shown, and the moving object area 3
00 is divided into two upper and lower constituent regions 300a and 300b, and each constituent region is further divided into four rectangular regions, and the distance to each rectangular region is calculated based on the formula (1). Therefore, in FIG. 6D, a total of eight pieces of distance data (four upper configuration areas and four lower configuration areas) are obtained.

Referring again to FIG. 5, after performing the stereo ranging as described above to calculate the distance data, it is determined whether or not the extracted moving object is a three-dimensional object (S406). This determination is made based on whether or not the distance data of the upper and lower constituent regions match, and for each rectangular region, the distance of the upper constituent region =
If the distance is that of the lower component area, it is determined that the object is a three-dimensional object (vehicle). If the two do not match, that is, if the upper component area is farther than the lower component area, it is determined that the object is not three-dimensional.
It should be noted that whether they match is determined by a predetermined allowable value (positive minute amount)
And it may be considered that they match when the difference between them is equal to or smaller than the allowable value.

In the above, the case where only the preceding vehicle is present in the image obtained by the CCD camera has been described. However, depending on the driving situation, the image obtained by the CCD camera may include not only the preceding vehicle but also a sign (for example, speed) on the road surface. Display etc.) may also be taken. The sign part has a difference in brightness from the other parts of the road surface, so that contrast occurs. Further, since the sign part moves relatively to the own vehicle, an optical flow also occurs in the sign part. In this case, it is possible to cope with the processing shown in FIG. 5. In the moving body region extraction processing in S404, it is only necessary to separate each moving object based on the calculated optical flow vector amount.

FIG. 7 shows the state of processing when a preceding vehicle and a road sign are present. As in FIG. 6, the white line 100 is recognized, and the own vehicle lane is set as the processing target area 200. Then, an optical flow is calculated within the set area. At this time, not only the optical flow of the vehicle ahead but also the optical flow of the road sign is calculated. However, since the preceding vehicle is traveling at a certain vehicle speed, while the road sign is stationary, the optical flows of the two are clearly distinguishable due to their different directions and sizes. Therefore, when the calculated optical flow is divided into small regions based on the vector amount, the optical flow can be divided into two small regions A and B as shown in FIG. A in the figure is the optical flow of the vehicle ahead, and B in the figure is the optical flow of the road sign. After dividing the optical flow based on its vector quantity,
As shown in FIG. 7D, the optical flow vector group is divided into two upper and lower constituent regions in each small region, and stereo distance measurement is performed on each constituent region. In the small area A, the distance of the upper constituent area and the distance of the lower constituent area match, so it is determined to be a solid (vehicle). In the small area of B, the distance of the upper constituent area does not match the distance of the lower constituent area. (The upper group is farther)
It will be determined as non-stereo.

In the above example, 2 is set according to the vector quantity.
Although it is divided into one small area, it can be divided into three or more parts depending on the case, and the constituent area can also be divided into three or more parts.

As described above, in the present embodiment, since the optical flow is calculated in advance by limiting the processing area and stereo ranging is performed, a moving object can be detected in a short time. Moreover, in the present embodiment, the small area based on the optical flow is divided into two upper and lower constituent areas, and it is determined whether or not the solid is based on the difference in the distance to each constituent area. Can be determined.

[0019]

As described above, according to the object detection apparatus of the present invention, a moving object, particularly a three-dimensional object such as a vehicle, can be detected in a short time and reliably from an image obtained by an imaging means such as a CCD camera. can do.

[Brief description of the drawings]

FIG. 1 is a configuration block diagram of an embodiment of the present invention.

FIG. 2 is an overall processing flowchart of the embodiment.

FIG. 3 is a flowchart of a steering control process according to the embodiment.

FIG. 4 is a flowchart of an inter-vehicle control process according to the embodiment.

FIG. 5 is a flowchart of a vehicle determination process according to the embodiment.

FIG. 6 is an explanatory diagram illustrating a vehicle determination process according to the embodiment.

FIG. 7 is another explanatory diagram showing the vehicle determination processing of the embodiment.

[Explanation of symbols]

10a, 10b CCD camera, 12 image processing computer, 14 cruise switch, 16 E for automatic driving
CU, 18 turn signal, 20 wheel speed sensor, 22
Steering actuator (ACT), 24 Throttle actuator (ACT), 26 Brake actuator (ACT), 100 white line, 200 setting area, 300
Moving object area.

Claims (2)

[Claims] 1. An on-vehicle imaging unit that captures a surrounding image, an area setting unit that sets an area in an obtained image from which an optical flow vector indicating a moving object relative to a vehicle is to be extracted, and the imaging unit A vector extracting means for extracting an optical flow vector from the image data of the set area in the time-series image obtained in the step, and a distance to each of two upper and lower constituent areas of the image area from which the optical flow vector group is extracted. An object detection device, comprising: a distance calculation means for calculating; and an identification means for identifying a moving object by comparing the obtained distance between the upper and lower constituent regions. 2. An in-vehicle imaging unit for capturing a surrounding image, an area setting unit for setting an area in an obtained image from which an optical flow vector indicating a moving object relative to a vehicle is to be extracted, and the imaging unit Vector extraction means for extracting an optical flow vector from the image data of the set area in the time-series image obtained in the step, and further dividing the image area from which the optical flow vector group is extracted into smaller areas based on the vector amount A distance calculating means for calculating a distance to each of the two upper and lower constituent areas in each of the divided small areas; and a movement of each small area by comparing the obtained distance between the two upper and lower constituent areas. An object detection device comprising: identification means for identifying a body;