JPH1114346A - Vehicular obstacle detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize behavior by detecting a preceding car without using a color image. SOLUTION: Stereo cameras 1 and 2 installed in a vehicle store two images picking up a forward road of the vehicle in image memories 3 and 4, and a matching position detecting part 7 detects a position of a window high in a resemblance degree from the other image as a template by dividing one image by using a window set by a window setting part 6. A distance operation means 8 performs an operation on a distance up to objects in respective windows by stereo image processing. A preceding car detecting part 9 detects a group of windows, on which the same distance value is calculated and to which windows are adjacent, as a preceding car. A white line detecting part 5, a travel lane judging part 10 and a winker flashing judging part 11 judge behavior by judging a preceding car traveling lane and flashing of a winker by monocular image processing. Therefore, a traveling obstacle can be detected by simple processing without requiring a color or the like as an image and without being influenced by a color or the like.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an obstacle detecting device for a vehicle, which detects a preceding vehicle causing a running obstacle by image processing.

[0002]

2. Description of the Related Art Conventionally, as an obstacle detecting device for a vehicle, for example, there is one described in Japanese Patent Application Laid-Open No. 2-30741. In this method, a muffler is detected in a thermal image using a thermal image and a color image to detect an obstacle, and a vehicle is detected by detecting an orange or red blinker from the color image diagonally right and left from the muffler position. It is.

[0003]

However, in the above-mentioned conventional camera, a special infrared camera is used to obtain a thermal image, and two types of cameras are used together with a camera for taking a color image. Must be used, which results in an increase in the size of the device and an increase in cost. In addition, when the muffler is not warm, or when the temperature is high and the car body is heated by the sun, it is difficult to detect the muffler, and the detection position of the orange or red turn signal is not fixed and the preceding vehicle cannot be detected. There is a disadvantage.

Further, since the positional relationship between the muffler and the turn signal differs depending on the distance to the vehicle and the type of the vehicle such as a motorcycle and a truck, the muffler cannot be detected even if the muffler is detected. The vehicle detection is determined only by detecting the orange or red portion, and the detection result is uncertain. Further, since the detection of the turn signal is detected by color, it is difficult to detect the turn signal in a vehicle having an orange body. For the same reason, there is a problem that an orange reflector such as a delineator has a color close to the turn signal, and therefore, there is a high possibility that the reflector is erroneously detected as a turn signal.

[0005] In the above-mentioned conventional example, only the detection of the vehicle is performed, and no judgment is made as to the traveling lane of the vehicle. There is also a problem that is detected as an obstacle. Also, even if an attempt is made to determine this, a white line or the like is not reflected on the thermal image. Also in the color image, the contrast is adjusted so that the orange or red color of the blinker is easily detected, and the image is difficult to detect the white line, so that it is difficult to determine the traveling lane. The present embodiment has been made in view of the above-described conventional problems, and has as its object to provide a vehicle obstacle detection device that can detect any vehicle type without requiring heat or a color image and that detects only a preceding vehicle as an obstacle. And

[0006]

For this purpose, two cameras arranged parallel to each other with a predetermined distance between eyes with their optical axes parallel to each other, and road surface images taken by the two cameras are first and second images. An image memory for storing as two images;
A white line detecting unit for detecting a white line in the first image; a window setting unit for setting a window so as to divide a predetermined area in the first image into a plurality of areas; A matching position detecting means for detecting a position of a high window from the second image, and a position on an image where a window having a high degree of similarity exists between the two images and a positional relationship between the two cameras. A distance calculating means for calculating a distance to an object imaged in each window; a preceding vehicle detecting means for calculating a same distance value and detecting a cluster of windows adjacent to each other as a detection area of the preceding vehicle; A driving lane judging means for judging the driving lane of the preceding vehicle based on the positional relationship between the window cluster and the white line is provided.

A blinker flicker determining means for detecting, as a blinker image pickup area, a region in which luminance changes from a window on the left and right sides of the block of windows at certain fixed time intervals, and determining that the blinker is in an operating state. Can be. Further, it is possible to provide a vehicle behavior determining means for determining the behavior of the preceding vehicle based on the positional relationship between the lump of the window and the white line using the determination result of the blinker blink determining means.

Further, a preceding vehicle type judging means for judging the type and size of the preceding vehicle based on the aspect ratio and the size of the window cluster may be provided. Further, when a cluster of vertically long windows is determined by the preceding vehicle type determination means, windows at both left and right ends of the cluster of windows are further subdivided vertically to provide subdivision means for forming a plurality of small windows. A small window may be used as a detection area for a luminance change.

[0009]

The two cameras form a stereo camera, and first and second images having parallax are captured and stored in an image memory. A white line is detected from the first image by the white line detecting means. Window setting means performs window setting so as to divide the area into predetermined areas in the first image. The matching position detection means detects the position of the window having the highest similarity to each window from the second image by the matching processing.

The distance calculating means calculates the distance to the object imaged in the window by triangulation based on the position of the window having the highest similarity between the first and second images and the positional relationship between the two cameras. Is calculated. When the same distance value is calculated in the adjacent windows, it can be determined that the same object is imaged in those windows. Therefore, the preceding vehicle detecting means calculates the same distance value and sets the window to the adjacent window. The lump is detected as a detection area of the preceding vehicle. The traveling lane determining means determines the lane in which the preceding vehicle travels based on the positional relationship between the lump of windows and the white line.

As described above, since the detection of the preceding vehicle is performed on a stereo image, a monochrome image is sufficient, and heat and vehicle color do not become influential factors, so that an effect can be obtained regardless of the use environment. Further, since the white lane is detected and the traveling lane of the preceding vehicle is determined, it is possible to determine whether or not the traveling lane of the own vehicle is obstructed. If it is determined that the vehicle is in trouble, the distance value has already been obtained when detecting the preceding vehicle, and the distance value can be used to manage the inter-vehicle distance. Can be built.

Then, a blinker blink determination means is added,
If the blinker blink information of the preceding vehicle is also incorporated, the behavior of the preceding vehicle can be predicted. Since the blinker is detected in the windows on both sides of the block of windows and the blinker operation is detected by a change in luminance, no operation for specifying the blinker position is required, and no complicated processing is required. In addition, when the vehicle behavior determining means is provided in addition to the blinking determining means, not only the preceding vehicle on the same running lane as the own vehicle, but also, for example, the distance between the vehicle and the vehicle on the adjacent running lane and the vehicle are in the running lane of the own vehicle. It is possible to make a judgment when interrupting and to have a predictive function.

Since the size and the aspect ratio of the window lump directly reflect the size and the aspect ratio of the vehicle, the preceding vehicle type determining means determines the type and size of the preceding vehicle from the relationship between the number of windows and the arrangement of the window lump. Can be determined. When the preceding vehicle type determination means determines a cluster of vertically long windows, it is possible to determine that the preceding vehicle is a two-wheeled vehicle, so the subdivision means further subdivides the windows at the left and right ends of the cluster of windows further vertically. Form a plurality of small windows. Since each small window is provided as a luminance change detection area, even if the preceding vehicle is a two-wheeled vehicle, the luminance change can be accurately detected without a decrease in the ratio of the turn signals in the detection area.

[0014]

Embodiments of the present invention will be described below with reference to embodiments. Here, before specifically describing the apparatus according to the embodiment, the stereo image obtained by the stereo camera is subjected to image processing to detect the position of the preceding vehicle, which is an obstacle in front, and detect the operation state of the blinker to recognize the behavior. The principle of operation will be described.

FIG. 1 is a diagram for explaining the principle of obtaining a distance Z from a stereo camera to a preceding vehicle C by the principle of triangulation using stereo images. Here, two cameras 1 and 2 constituting a stereo camera are mounted on a vehicle. The cameras 1 and 2 are CCD cameras each having lenses L1 and L2 having the same focal length f and CCDs 1a and 2a arranged such that the distance from each lens to the imaging surface is the focal length f.

In the cameras 1 and 2, the imaging surfaces of the CCDs 1a and 2a are located in the same vertical plane, and the Y axis which is the vertical reference axis of each imaging surface, that is, the axis of the imaging surface of the CCD 1a and the CCD 2a
The axes (YA, YB) of the imaging surfaces of the lenses L1, L2
The two optical axes 1b, 2b are arranged side by side so that they are parallel to each other and the interocular distance (distance between the optical axes 1b, 2b) D is a predetermined value.

The focal length f and the interocular distance D are known,
In a stereo image composed of two images obtained by imaging the front of the vehicle with two cameras 1 and 2 whose optical axes 1b and 2b are parallel to each other, a matching position between the two images (the position of the most similar image, In FIG. 1, if the Y coordinates ya and yb of the rear side upper edge of the preceding vehicle C can be obtained,
The distance Z from the cameras 1 and 2 to the preceding vehicle C can be obtained from the following equation (1). Z = f × D / (yb−ya) (1) Here, the units of f, ya, and yb are pixels of the CCDs 1a and 2a, and the units of D and Z are mm. In general, the focal length f is often expressed in units of mm, but the focal length f in Expression (1) is calculated in units of pixels.

Here, how to determine the focal length f in units of pixels will be described with reference to FIG. FIG. 2A shows an object C ′ having a width W (mm) of known size and a distance Z (m).
m) shows a state where one of the two cameras 1 and 2 (here, the camera 1) captures an image at a distance of m), and FIG. 7B shows an image A obtained at that time. In the following description, the first image captured by the camera 1 is A, and the second image captured by the camera 2 is B.

The focal length f in pixels is the distance Z (mm) of an object C 'having a width W (mm) whose size is known.
Is taken at a distance only, and the width xw (pixels) of the object C ′ on the image A (or image B) obtained at this time
Is detected by image processing such as edge detection, and can be obtained by the following equation (2). f = xw × Z / W (2) Here, the unit of xw is a pixel, and the unit of Z and W is mm.
It is.

FIG. 3 shows a state in which a predetermined area in the image A is cut for each window, and an image pickup position of the same edge in the image B is obtained by using a characteristic edge of the object picked up in each window W. It is a figure showing a result. If the edge matching position is known for each window, the distances of all edges within the area defined by the window can be determined by using the equation (1). Since the distance calculated for each window is a distance to an object having a characteristic portion such as an edge imaged inside the window, if the same distance is obtained in adjacent windows, the distance is calculated for those windows. Can be determined that the same object is imaged.

In the image taken in front of the road, the preceding vehicle is imaged as an object having a strong horizontal edge that is strong relative to the background. Therefore, when the matching position for each window is detected using the horizontal edge component. When the preceding vehicle is included in the window, the same distance value is detected from the adjacent window. That is, when the same distance is continuously obtained in several adjacent windows,
That part (the bold window in the figure) can be regarded as the preceding vehicle.

Next, determination of which traveling lane the preceding vehicle is traveling on will be described based on the positional relationship between the position of the block of windows determined to include the preceding vehicle on the image and the detected white line. . FIG. 4 shows an image of the rear lower edge of the preceding vehicle C at a distance Z (here, camera 1).
FIG. As can be seen from this figure, the position in the Y-axis direction on the image in which the lower edge of the preceding vehicle C detected at the distance Z is captured can be obtained by Expression (3). yc = [(h−H) · f] / Z (3) where H is the height from the road surface to the lower edge of the preceding vehicle, h is the height from the road surface to the center of the camera lens, and f is the focal length. ,
Z is an inter-vehicle distance. This inter-vehicle distance Z and the position y on the image
As shown in FIG. 5 using the relationship of c, if the position of the white line in the X-axis direction is detected at the same position as the distance to the preceding vehicle and the positional relationship with the preceding vehicle is determined, the preceding vehicle moves on the own vehicle traveling lane. Or the vehicle is on an adjacent driving lane.

Next, based on the detected position of the preceding vehicle,
A method of detecting the position of the turn signal of the vehicle will be described.
Winkers are usually on the left and right sides of the vehicle. Here, since the window including the preceding vehicle is obtained from the image A and the image B, as shown in FIG. 6, both sides of the preceding vehicle are located at the left and right ends of the cluster of windows determined to include the preceding vehicle. It means that the image has been taken. Therefore, if there is a portion in which the luminance changes at certain time intervals in the windows E at the left and right ends of the block of windows, it is possible to determine that the position of the blinker and that the blinker E is blinking.

As described above, the distance to the preceding vehicle and the position of the preceding vehicle on the image can be detected by the stereo image processing using the stereo images. Further, since the lane in which the preceding vehicle travels and the blinking of the blinker of the preceding vehicle can be determined by the monocular image processing performed using one of the stereo images, the behavior of the preceding vehicle can be determined.

FIG. 7 is a block diagram showing the configuration of the first embodiment of the present invention. The two cameras 1 and 2 constitute a stereo camera, each having lenses L1 and L2 having the same focal length f, and a CCD 1a arranged such that the distance from each lens to the imaging surface is the focal length f. 2a. The cameras 1 and 2 are attached to predetermined portions on a vehicle suitable for photographing the front. Cameras 1 and 2 are, as shown in FIG.
Each imaging plane of D1a, 2a is located in the same vertical plane, the vertical reference axes (YA axis, YB axis) of each imaging plane coincide, and the optical axes 1b, 2b of the lenses L1, L2 become horizontal and parallel to each other. , With a predetermined interocular distance D.

The captured video signals of the cameras 1 and 2 are output to the image memories 3 and 4 as digital images and stored. The white line detector 5 detects the image A from the image memory 3
To detect two white lines surrounding the own vehicle. The window setting unit 6 fetches the image A from the image memory 3 and sets a plurality of windows of the same size so as to divide the screen into a screen portion excluding an area such as the sky where a vehicle cannot exist.

The matching position detecting section 7 superimposes the window on the differential image of the image A, detects the position of the window having the highest similarity with each window from the image B, and stores the matching position of the window between the two images. I do. The distance calculation unit 8 calculates the distance to the imaging object in the window based on the matching position of each window, the focal length f of the lenses of the cameras 1 and 2, and the distance D between the eyes. The preceding vehicle detection unit 9 calculates the same distance and detects a cluster of adjacent windows as the preceding vehicle.

The traveling lane determining unit 10 determines the traveling lane of the preceding vehicle based on the positional relationship in the image between the position of the window cluster and the white line detected by the white line detecting unit 5. The blinker blink determination unit 11 detects the blinkers by using the windows on both the left and right sides of the cluster of windows as the blinker detection area, and determines the blinking state. The vehicle behavior determination unit 12 determines the behavior of the preceding vehicle from the blinking state of the turn signal and the result of the traveling lane determination.

FIG. 8 is a flowchart showing the flow of the processing of this embodiment. First, in step 101, stereo images, that is, images A and B output from the cameras 1 and 2, respectively, are input and stored in the image memories 3 and 4, respectively. In step 102, the white line detection unit 5 determines that the images A and B
(In this embodiment, the image A stored in the image memory 3) is subjected to image processing to detect a white line. In this example, we will consider detecting only the driving lane.
The two white lines on the left and right of the vehicle are detected. 9 and 10 are explanatory diagrams of the principle of detecting a white line. FIG. 9A is a front view showing a state where the monocular camera (camera 1) is capturing a white line at the center of the road, and FIG. 9B is a side view thereof. When the vehicle is traveling between two white lines, the two white lines appear on both left and right sides of the image. Assuming that the front white line is a straight line and the vehicle is traveling in the center of the two white lines, FIG.
As shown in the above, the coordinates on the image where the white line ahead of the distance Z is picked up can be obtained by equation (4) by geometric calculation. xr = (Wr · f) / (Z · 2), yr = (h · f) / Z (4) where Wr is the width of the road surface, h is the height from the road surface to the center of the camera lens, and f is Focal length, Z is the distance between vehicles, xr,
yr is a position on the image.

The positions xr and yr on the image are obtained for a plurality of distance values Z based on equation (4), and the window setting positions for detecting the white line are determined.
A window Wd centering on xr and yr is set as shown in FIG. The size of the window Wd is determined so that the white line does not deviate in anticipation of the case where the vehicle is not at the center of the white line. Then, a differential operation is performed in each window Wd to detect a position where the left and right edges appear as a pair, and the center position of the positive and negative edges on the X axis is calculated and stored as a white line detection point. A white line is detected by connecting detection points from each window so as to further fit them.
For the fitting method, refer to Japanese Patent Application No. Hei 3-3145.
Or a method introduced in Japanese Patent Application No. 4-171240.

Next, in step 103, the window setting section 6 detects an area such as the sky in one of the images A and B (the image A in the present embodiment), and adjusts the size to a predetermined size in accordance with the detected area. Set up the window In step 104, the matching position detecting unit 7 determines whether the image A and the image B shown in FIG.
Is input and subjected to a differentiation process to create a differentiated image (b) in which horizontal edges are emphasized. (For the sake of simplicity, the differential images are also referred to as the original image by the names of the original images, image A and image B.) At the window position set in step 103, the image A is horizontally width xw and vertically width as shown in FIG. The window is divided for each window 40 of yw. Using each of the divided windows 40 as a template, a position having the highest similarity to the image of the template is obtained from the image B by a normalized correlation method using Expression (5), and a matching position of the window is calculated.

(Equation 1) In Expression (5), the luminance value of each pixel of the template image is Aij, and the luminance value of each pixel of the image B is Bij. The reason why the differential image is used is that many of the preceding vehicles, which are detection targets, have a horizontal edge, and the preceding vehicles are emphasized more than the background and are easily detected.

FIG. 13 shows images A and B obtained by the normalized correlation method.
FIG. 4 is an explanatory diagram of a method of detecting a matching position between the two. Here, the two cameras 1 and 2 are arranged such that their Y axes are on the same line.
At the same X-axis position as the template (window 40), a plurality of position detection target images whose search positions are sequentially shifted in the Y-axis direction are obtained. Then, by comparing the template extracted in (b) of FIG. 13 with the position detection target images (d1) to (d3) extracted from the image B shown in (c), the position detection with the highest similarity is performed. The target image will be obtained. In the figure, (d2) shows the detection target image at the position where the similarity with the template is maximum, (d1) shows the detection target image at a position above it, and (d3) shows the detection target image at a position below it. Thus, a window corresponding to each window is created in image B for image A. Ya and yb indicating the Y-axis positions of both windows are the matching positions of the windows.

In step 105, the distance calculation unit 8 calculates the distance to the object imaged in each of the windows by using equation (1). Since the object occupies the same position in the window of the image A and the image B as shown in FIG. 14, the matching position difference of the object is the window matching position difference (yb-ya). Therefore, the distance to the object in the window is calculated by substituting into the equation (1) using the matching position of the window.

When the above-described processing is performed on all the windows and the distance to the object imaged inside each window is obtained, the cluster of the windows in which the same distance value is continuously calculated in step 106 is obtained. Is detected, and if there are multiple blocks of the window,
The closest one is selected and detected as the detection area of the preceding vehicle. In step 107, it is checked whether a block of windows has been detected. If not, the process returns to step 101 assuming that there is no preceding vehicle, the next image is input, and the above processing is repeated. If a block of windows can be detected, the distance value calculated in the window is stored as a distance detection value to the preceding vehicle, assuming that there is a preceding vehicle.
Proceed to 08.

In step 108, the traveling lane judging section 10 judges whether or not the preceding vehicle is in the same traveling lane as the own vehicle. In this case, as shown in FIG. 5, two points intersecting the two white lines detected in step 102 are detected using the same position yc as the distance of the block of the window as a line. Then, it is determined whether or not the lump of the window is located between the two intersections, and it is determined whether or not the preceding vehicle is in the same traveling lane as the own vehicle. If the traveling lane is different from that of the own vehicle, the process returns to step 101 because the preceding vehicle does not interfere with the traveling of the own vehicle, and the above processing is performed again. If the detected preceding vehicle is in the same running lane as the own vehicle, the process proceeds to step 109 assuming that the running obstacle of the own vehicle is caused.

In step 109, the vehicle behavior determination unit 1 selects one of the blocks of the window determined to be the preceding vehicle.
2 detects the blinking of the turn signal as a detection area of the turn signal by using the windows at the left and right ends to determine the behavior of the preceding vehicle. FIG. 15 is a diagram illustrating a method of detecting a blinker and a method of determining blinking of the blinker. First of all, as shown in FIG.
Are taken out one by one to detect a vertical edge in the window. If the detected vertical edge does not satisfy the two check conditions of not less than a certain threshold and not less than a predetermined length, the entire window is used as a blinker detection area to further detect a horizontal edge and determine the length and intensity. And then
The detection area is defined by the approximate lines u, v, w of the horizontal edge. In each of the divided small windows, a portion having a large change in luminance as compared with other portions is detected.

If a vertical edge satisfying the check condition can be detected, the window is divided by an approximate line t of the vertical edge as shown in FIG. In the window on the right side of the block of windows, only the image on the left side of the approximation line t is set as the blinker detection area in the left window. A horizontal edge is further detected from within the detection area, and the detection area is partitioned by approximate lines u, v, w of the horizontal edge. In each of the divided small windows, a portion having a larger change in luminance than other portions is detected. FIG.
FIG. 5B is a diagram showing a section state when a vertical edge in the right window is detected.

In the above-described detection, when it is confirmed that there is a luminance change and the interval of the change is substantially the same as the blinking interval of the blinker and has continued for a certain period of time or more, it is determined that the blinker is blinking, and the preceding vehicle is judged to be blinking. It is determined that the vehicle is moving to the adjacent driving lane. If there is no change in luminance or the confirmation conditions are not met even if there is, it is determined that there is no blinking of the blinker,
It is predicted that the preceding vehicle will continue to travel in the same traveling lane. If it is confirmed that the brightness in both the left and right windows has suddenly increased since a certain point, it is determined that the preceding vehicle has stopped, and step 1 is performed.
Return to 01.

The present embodiment is configured as described above. A window for dividing an image is set in one of the stereo images, the position of the window having the highest similarity to each window is detected from the other image, and the principle of triangulation is used. The distance calculation based on the distance is performed, and a cluster of windows in which the same distance calculation value has been calculated from an adjacent window is detected as a preceding vehicle. Then, among the lump of windows recognized as the preceding vehicle, the blinking state of the turn signal is detected from the windows on both sides by a change in luminance, and the behavior of the preceding vehicle is determined through the determination of whether the vehicle is in the same running lane as the own vehicle, Compared with the past, only one type of camera is needed, and even monochrome is enough,
The processing can be simplified, and the effect of irrespective of the vehicle type and the detection environment can be obtained.

Next, a second embodiment of the present invention will be described. In this embodiment, the blinker is detected and the behavior is determined for vehicles on an adjacent traveling lane in addition to the same traveling lane. FIG. 16 is a block diagram showing the configuration of the present embodiment. The function of judging a vehicle that is interrupting from an adjacent lane in place of the vehicle behavior judging unit 12 in the block diagram of the first embodiment shown in FIG. The vehicle behavior determination unit 13 provided with. Driving lane judgment unit 1
At 00, the traveling area of the vehicle including the adjacent traveling lane is determined based on the white line detected by the white line detecting unit 5. Others are the same as the first embodiment.

Hereinafter, the flow of processing will be described with reference to the flowchart of FIG. Step 101 to step 10
Up to 9, the stereo images from the cameras 1 and 2 are input and stored in the image memories 3 and 4 as in the first embodiment. Similar to the first embodiment, the white line detection unit 5 detects two white lines on the left and right of the vehicle from one of the stereo images and converts the white lines into data by fitting processing.

The matching position detecting unit 7 detects the position of the window having the highest similarity from the other image by using the normalized correlation method for one image whose window has been set by the window setting unit 6. The preceding vehicle detection unit 9 calculates the same distance value among the windows whose distances have been calculated by the distance calculation unit 8 and detects a cluster of adjacent windows as a detection region of the preceding vehicle. Then, in step 108, from the image A, the traveling lane determining unit 100 indicates the position of the white line on the same Y-axis position as the distance of the lump of the window.
1 and xr are detected. If it is determined that the lump of windows is within the white line positions xl and xr, it is determined that the preceding vehicle is on the same traveling lane as the own vehicle and becomes a driving obstacle, and in step 109 the windows at both ends of the lump of windows are turned to the winkers. A small window is set as a detection area based on the horizontal and vertical edges in the window. In each of the small windows, the operating state of the turn signal is detected based on the change in luminance, and the behavior of the preceding vehicle is determined. On the other hand, if it is determined in step 108 that the vehicle is not in the same traveling lane as the own vehicle, the first embodiment assumes that the preceding vehicle does not become a traveling obstacle for the own vehicle, and returns to step 101. In the present embodiment, the process proceeds to step 210. Further, the behavior is determined, and it is determined whether or not the preceding vehicle on the adjacent lane interrupts the own vehicle traveling lane.

That is, as shown in FIG. 18, when the vehicle is traveling on a two-lane road, there are three white lines, and if all of them are within the imaging range of the cameras 1 and 2, the vehicle travels in the left lane. In this case, the white line ahead of the distance Z is shown in FIG.
As shown in FIG. 9, images are taken at the positions of xl, xr, and xr2 on the Y axis. The traveling lane can be determined by judging the position of the preceding vehicle. Points xl and xr on own vehicle driving lane
Has already been obtained in step 108, and the distance Wr between the white lines is usually about 3.5 m.
As can be seen from FIG. 8, the position xr2 on the image of the white line outside the right adjacent lane at a distance Z can be obtained by Expression (6). xr2 = xr + (f × Wr) / Z (6) Z is the distance detection value to the vehicle, f is the focal length of the camera, W
r is the white line width.

In step 210, the driving lane determining unit 100 virtually calculates the position of the white line xr2 based on the equation (6). It is determined whether a block of windows is between xr and xr2. Note that when the inter-vehicle distance is short, xr2 may go outside the right side of the image. In such a position, the determination is made in consideration of the fact that xr2 is outside xr. Then, as a result of the determination, if the lump of the window is between xr and xr2, the preceding vehicle is on the right adjacent lane, and step 211
Proceed to. If the lump of the window is not between xr and xr2, it is determined that the preceding vehicle is on the left lane and step 21
Proceed to 2.

In step 211, the blinking state of the left blinker is determined by partially blinking the window on the left side of the block of windows as in the first embodiment. When there is blinking, it is determined that the vehicle in the adjacent lane is interrupting. In step 212, the blinking state of the right blinker is determined by the blinking determination of a part of the window on the right side in the same manner. When there is blinking, it is determined that the vehicle in the adjacent lane is interrupting.

The present embodiment is configured as described above, and the behavior is determined not only for the preceding vehicle traveling on the same lane as the own vehicle but also for the vehicle traveling on the adjacent lane, so that the detection range is expanded. Is done. FIG. 20 is a diagram illustrating a state in which the preceding vehicle N on the right running lane is detected at a shorter distance than the preceding vehicle M on the own vehicle lane. In the first embodiment, it is determined that the preceding vehicle N does not become an obstacle. On the other hand, in the present embodiment, the blinking state of the blinker is determined, and it is determined whether or not the vehicle N will be interrupted in the traveling lane. The effect of predictively determining an obstacle is obtained.

Next, a third embodiment will be described. In this embodiment, the type and size of a vehicle are detected based on a cluster of windows. In the case where the detected vehicle is a two-wheeled vehicle, since the turn signal is small, a detection method that does not reduce the detection accuracy will be described. In the above embodiment, since a cluster of windows surrounding a horizontal edge having a predetermined length is detected as a vehicle, it is possible to determine the size and type of the detected vehicle from the aspect ratio and size of the cluster of windows. is there. That is, as shown in FIG. 21, the window of the passenger car in FIG. 12 is three steps in contrast to the two steps in the same distance as shown in FIG. It can be determined that there is. Further, if a cluster of two rows of vertically long windows is detected as shown in FIG. 22 for a passenger car having four rows of windows in FIG. 12, it can be determined that the preceding vehicle is a two-wheeled vehicle. Thus, by detecting the number of windows or the arrangement relationship of windows, the size and aspect ratio of the preceding vehicle can be determined, and the type of vehicle can also be determined.

When the preceding vehicle is a two-wheeled vehicle, the blinker size is small relative to the window size, and it is difficult to detect a vertical edge when the window is divided into small windows. Therefore, only when it is determined that the vehicle is a motorcycle, the window blinking determination unit 11 vertically subdivides the windows at the left and right ends of the cluster of windows into a predetermined size without detecting the vertical edge as the subdividing unit. . In all of the subdivided small windows, as in the first and second embodiments, a horizontal edge is detected as shown in FIG. 23, and the small window is divided into a plurality of regions by an approximate line u. A change in luminance is detected at fixed intervals in each region.

As described above, since the lump of windows reflects the size and the aspect ratio of the preceding vehicle, the type and size of the vehicle can be determined based on the number and arrangement of the windows without additional processing. When the vehicle is a two-wheeled vehicle, the blinker detection area is further subdivided, so that the blinking of the blinker can be detected without leaking even by the same method as a large vehicle such as a passenger car.

[0050]

As described above, according to the present invention,
A window for dividing the first image captured by the stereo camera is set, and the distance to the captured object is calculated based on the position of the window of the second image having the highest similarity to each window. Then, the same distance value is calculated and a cluster of adjacent windows is detected as the preceding vehicle. Therefore, even a monochrome image is sufficient, and heat and vehicle color do not become influential factors, so that the effect of irrespective of the use environment can be obtained. can get. Further, since the white lane is detected and the traveling lane of the preceding vehicle is determined, it is possible to determine whether or not the traveling lane of the own vehicle is obstructed.
If it is determined that the vehicle is traveling, the distance value up to that point has already been obtained when detecting the preceding vehicle, and that distance value can be used for managing the inter-vehicle distance. Functions can be easily constructed.

The behavior of the preceding vehicle can also be predicted by adding the blinker blinking state determination means and incorporating the blinker blinking information of the preceding vehicle. Since the blinker is detected in the windows on both sides of the block of windows and the blinker operation is detected by a change in luminance, no operation for specifying the blinker position is required, and no complicated processing is required. Also, in addition to the blinking state determining means, if a vehicle behavior determining means is provided,
Not only the preceding vehicle on the same driving lane as the own vehicle, but also the distance between the vehicles on the adjacent driving lane and the distance between the vehicles and the judgment when the vehicle interrupts the own driving lane can have predictive functions. It is.

Since the size and aspect ratio of the lump of windows directly reflect the size and aspect ratio of the vehicle, the preceding vehicle type determining means determines the type and size of the preceding vehicle based on the relationship between the number of windows and the arrangement of the lump of windows. can do. When the preceding vehicle type determination means determines a cluster of vertically long windows, it is possible to determine that the preceding vehicle is a two-wheeled vehicle, so the subdivision means further subdivides the windows at the left and right ends of the cluster of windows further vertically. Form a plurality of small windows. Since each small window is provided as a luminance change detection area, even if the preceding vehicle is a two-wheeled vehicle, the luminance change can be accurately detected without a decrease in the ratio of the turn signals in the detection area.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram for explaining a distance calculation method by stereo image processing.

FIG. 2 is an explanatory diagram showing a relationship between a pixel and a focal length.

FIG. 3 is an explanatory diagram showing a principle of calculating a distance based on a window position difference.

FIG. 4 is an explanatory diagram of a positional relationship between a distance and an image.

FIG. 5 is an explanatory diagram of a traveling lane determination of a preceding vehicle.

FIG. 6 is a diagram showing a detection area of a turn signal from a block of windows.

FIG. 7 is a block diagram showing the configuration of the first embodiment.

FIG. 8 is a flowchart illustrating a flow of a process according to the first embodiment.

FIG. 9 is an explanatory diagram for detecting a white line.

FIG. 10 is an explanatory diagram showing window settings.

FIG. 11 is a diagram illustrating an effect of a differentiation process.

FIG. 12 is a diagram illustrating window setting for detecting a preceding vehicle.

FIG. 13 is an explanatory diagram showing a method of detecting a matching position between stereo images by a normalized correlation method.

FIG. 14 is a diagram showing a positional relationship between an image in a window and the window.

FIG. 15 is a diagram illustrating a method of detecting blinking of a turn signal.

FIG. 16 is a block diagram showing the configuration of the second embodiment.

FIG. 17 is a flowchart illustrating a flow of a process according to the second embodiment.

FIG. 18 is a diagram illustrating a principle of detecting an adjacent lane.

FIG. 19 is a diagram illustrating an imaging position of an adjacent lane.

FIG. 20 is a diagram showing a state in which a preceding vehicle interrupts the own vehicle traveling lane.

FIG. 21 is a diagram illustrating a block of windows detected as tracks.

FIG. 22 is a diagram showing a lump of windows detected as a motorcycle.

FIG. 23 is an explanatory diagram for subdividing a window.

[Explanation of symbols]

 1, 2 Camera 1a, 2a CCD 1b, 2b Optical axis 3, 4 Image memory 5 White line detection unit 6 Window setting unit 7 Matching position detection unit 8 Distance calculation unit 9 Leading vehicle detection unit 10, 100 Driving lane judgment unit 11 Turn blinker Judgment units 12 and 13 Vehicle behavior judgment unit C Leading vehicle E Winker f Focal length D Interocular distance L1, L2 Lens t, u, v, w Approximation line W Window wd Window Wr Road width ya, yb Matching position yc Imaging position xr Window setting position on X axis xw Horizontal width of window yw Vertical width of window yr Window setting position on Y axis Z Imaging distance

Claims (5)

[Claims] 1. Two cameras whose optical axes are parallel to each other and are arranged at a predetermined interocular distance, and images storing road surface images captured by the two cameras as first and second images. A memory, a white line detecting means for detecting a white line in the first image, a window setting means for setting a window so as to divide a predetermined area in the first image into a plurality of areas, A matching position detecting means for detecting a position of a window having a high degree of similarity from the second image; and a positional relationship between the two cameras and a position on an image having a window having a high degree of similarity between the two images. A distance calculating means for calculating a distance to an object imaged in each window, and a block of windows in which the same distance value is calculated and the windows are adjacent to each other are detected. Preceding vehicle detection means for detecting as
An obstacle detecting device for a vehicle, comprising: a traveling lane determining unit that determines a traveling lane of a preceding vehicle based on a positional relationship between the cluster of windows and the white line. 2. A blinker blink determination means for detecting, as a blinker image pickup area, a region in which luminance changes from a window on the left and right sides of the block of windows at a certain time interval, and determining that the blinker is in an operating state. The vehicle obstacle detection device according to claim 1, wherein 3. A vehicle behavior judging means for judging the behavior of a preceding vehicle based on the positional relationship between the lump of the window and the white line based on the judgment result of the blinker blink judging means. The vehicle obstacle detection device according to claim 2. 4. A preceding vehicle type judging means for judging the type and size of the preceding vehicle based on the aspect ratio and the size of the cluster of windows is provided.
An obstacle detection device for a vehicle according to the above. 5. When the preceding vehicle type determining means determines a cluster of vertically long windows, the subdivision means for further subdividing the windows at both left and right ends of the cluster of windows vertically to form a plurality of small windows. 5. The vehicle obstacle detecting device according to claim 4, wherein the small window is provided as a detection area of a luminance change.