JPH10188198A - White line detector for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable memory capacity reduction and high-speed arithmetic by setting a window inside a picked-up image, calculating the average value and maximum value of its luminance value and setting the threshold value of binarization base on these values. SOLUTION: An analog video signal from a CCD camera is inputted (S10), and an input signal is sampled (S12). Left and right processing windows are set near a position, where the existence of white line is estimated, on the left and right sides of image (S14) and its luminance value is inputted (S16). The average luminance values of respective horizontal lines are added and divided with the number of horizontal lines and an average luminance value A in the processing window is calculated (S18-S26). Continuously, the maximum luminance values of respective horizontal lines are detected and a maximum luminance value M in the processing window is found out of these values (S28-S40). Next, a threshold value TH (for binarization) is determined from the average luminance value A and the maximum luminance value M (S42). Processing from S16 to S42 is separately performed for the left and right processing windows and the threshold value TH is separately determined.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for detecting a white line of a vehicle, and more particularly to an apparatus for accurately detecting a white line on a highway or the like.

[0002]

2. Description of the Related Art Recently, as one of vehicle running control technologies,
It has been proposed that an image sensor (imaging means) such as a CCD camera be mounted on a vehicle to image a traveling path ahead of the vehicle on a highway or the like and detect a white line. In such a technique, a threshold value is determined from a histogram created in a predetermined area (window) in a captured road surface image, and binarization is performed. For example, JP-A-5-205
The technology described in Japanese Patent Publication No. 93 discloses a threshold value for binarization,
The value is a value obtained by subtracting a certain value from the maximum value of the luminance.

[0003]

As described above, in the prior art, the threshold value is obtained from the histogram relating to the luminance of the pixels. However, since the road surface is generally not smooth,
The brightness of the road surface changes due to the reflection of light or the like, and a histogram having a regular shape is not always obtained.

Further, in order to obtain a histogram of an image, the entire image is quantized to 8 bits and temporarily stored in a memory. Therefore, the amount of memory increases, the time required to access the memory increases, and the processing time increases. There was an inconvenience.
Further, there is also a problem that the circuit scale becomes large and the configuration becomes complicated even if it is implemented as hardware.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to solve the above-mentioned disadvantages of the prior art. In order to binarize an image, a threshold value is not affected by a change in environment and a road structure pattern, and It is an object of the present invention to provide a vehicle white line detection device capable of reducing the amount of memory and performing high-speed calculations and reducing the circuit scale when implementing hardware.

[0006] Further, in recognition of a road structure from a moving vehicle by image recognition, generally, a point at which a gray value changes from dark to bright and from light to dark from a two-dimensional observation image obtained by imaging an external environment including a white line. Detect and detect an area in which the area formed by the set of detected points is equal to or less than a predetermined width as a white line detection candidate portion, classify the white line pattern from the area of the center point of the area, and classify the classified pattern. Is subjected to Hough transform to extract white lines.

However, in such a method, it is difficult to set a threshold value for separating a white line from an edge other than the white line. May be extracted. Further, since the Hough transform is performed for all the white line candidate portions, the calculation process becomes enormous, and the circuit scale may be increased when hardware is implemented.

Accordingly, a second object of the present invention is to solve the above-mentioned disadvantages, and to reduce the amount of calculation and perform high-speed processing by performing processing only on binarized images without using Hough transform. Another object of the present invention is to provide a vehicle white line detection device which has a simplified configuration.

[0009]

In order to achieve the above object, a vehicle white line detecting apparatus according to the present invention is provided.
In the paragraph, imaging means for imaging a traveling road including a traveling road surface in the traveling direction of the vehicle, window setting means for setting a window in an image captured by the imaging means, an average value of luminance values in the window Average brightness value calculating means for calculating the maximum brightness value in the window, and a threshold value for binarization based on the calculated average brightness value and the maximum brightness value. It is configured to have a threshold setting means for setting.

According to a second aspect of the present invention, a correlation value calculating means for comparing the image binarized by the set threshold value with a template and calculating a correlation value, and calculating the calculated correlation value. A comparison means for comparing with a threshold value and a white line detection means for detecting a white line based on the comparison result are provided.

According to a third aspect of the present invention, the white line detecting means includes a template updating means for updating a template based on the comparison result, and a threshold increasing means for increasing the threshold. .

According to a fourth aspect of the present invention, the template updating means, when it is determined based on the comparison result that the calculated correlation value is not more than the threshold value, shifts the template in the horizontal direction. A template vertical moving means for moving the template, and a template vertical moving means for moving the template in the vertical direction when the calculated correlation value is determined to be equal to or larger than the threshold value based on the comparison result. did.

According to a fifth aspect of the present invention, the template updating means includes a search window setting means for setting a search window, and the template horizontal moving means moves the template horizontally in the search window. It was configured to be moved.

[0014]

According to the first aspect of the present invention, the threshold value for binarizing an image is not affected by a change other than light / dark changes in the environment or other than a road structure pattern. The circuit scale can be reduced even when the hardware is implemented.

In the second aspect, since the processing is executed only for the binary image without using the Hough transform, the amount of calculation is reduced to enable high-speed processing, and the configuration is simplified. be able to.

According to the third aspect, since the template is updated based on the latest data and the threshold is also updated, the processing speed and the detection accuracy can be improved, and the distance can be reliably increased. The white line can be recognized.

Since the template is moved in the horizontal or vertical direction based on the comparison result, the white line can be recognized more reliably.

According to the fifth aspect, since the template is moved in the horizontal direction within the search window, the processing speed can be further increased.

[0019]

Embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a top view of a vehicle showing the entire vehicle white line detecting device according to the present invention, and FIG. 2 is a block diagram functionally showing the device.

As shown in the figure, this apparatus comprises one CCD camera (monochrome TV camera) 1
0, an image processing unit 12, and a display unit 14. CC
The D camera 10 is fixed near the rearview mirror mounting position above the driver's seat, captures a road image in the vehicle traveling direction with a single eye, and converts the road image into an analog luminance signal.

The camera parameters of the CCD camera 10 are as follows:
CCD size: 2/3 inch, lens: with auto iris function, lens focal length: 16 mm, lens angle of view: 38
Degree, camera installation height: 1200 mm, camera depression angle: about 5 degrees. The image plane of the CCD camera 10 is 480 pixels vertically by 640 pixels horizontally as shown in FIG.

FIG. 4 shows a lens angle of view, a white line detection range determined according to the angle of view, and a road environment assumed by the apparatus. The device according to the present invention, as shown in FIG.
It is assumed that a road environment such as a highway in which a driving route (lane) is displayed as a white line (long line or broken line) 18.
It should be noted that the road environment may be such that the travel path is specified by only one of the long line and the broken line among the white lines.

The video signal output from the CCD camera 10 is
It is sent to the image processing unit 12.

The image processing unit 12 includes the microprocessor 2
2, image processing board 24, DC / AC inverter 26,
And a VTR / SCAN inverter 28. The microprocessor 22 converts the input video signal into a digital luminance signal corresponding to the road surface brightness at a predetermined sampling cycle, stores the digital luminance signal in the VTR / SCAN converter 28, performs image processing as described later, and (White line) is detected.

The detection result of the image processing unit 12 is sent to an external device such as a vehicle control device (not shown), and is used for automatic traveling control and the like and sent to the display unit 14. The display unit 14 includes a display (CRT display device) 32 and a keyboard 34, and displays detection results of the image processing unit 12.

Next, the binarization threshold value determination operation in the operation of the vehicle white line detection device according to the present invention will be described with reference to the flowchart of FIG. The illustrated program is started, for example, every 33 ms (30 frames per second).

The operation shown in the flow chart of FIG.
However, specifically, the processing is performed via an image processing board 24 including a hardware circuit shown in FIG. Hereinafter, FIGS. 5 and 6
This will be described with reference to FIG.

First, in step S10, an analog video signal from the CCD camera 10 is input. Subsequently, the process proceeds to S12, where the input signal is sampled, converted into a digital brightness value corresponding to the brightness of the road surface, and stored in the VTR / SCAN converter 26.

Then, the process proceeds to S14, where the processing window 40
Set. That is, as shown in FIG. 7, the left and right processing windows 40a and 40b are set near the estimated position where the white line exists on the left and right of the image. The processing window 40 is 120 pixels in length and 160 pixels in width in both left and right.

Then, the process proceeds to S16, where the luminance value in the window is input, and the process proceeds to S18, where the luminance value of the horizontal line n, and in the first program loop, the luminance value of the first line is added to the adder 44 as shown in FIG. Is added using. Then, the process proceeds to S20, where the added value is divided by the number of pixels (160 pixels) of the horizontal line n via the divider 46 shown in FIG. 6 to calculate the average luminance value An of the horizontal line n.

The calculated value An is stored in the corresponding address of 1o to 1n in the buffer 48 provided for each horizontal line.
Then, the process proceeds to S22 in which the horizontal line n is set to the processing window 4
It is determined whether or not it is the last row of 0 (120 rows). If the result is negative, the process proceeds to S24, where the number of horizontal lines to be searched is incremented by 1, and the process returns to S18 to repeat the above operation.

On the other hand, if the result in S22 is affirmative, the program proceeds to S26, in which the average luminance value An of each horizontal line is added via the adder 50 and the divider 52 shown in FIG. The average luminance value A within the processing window is calculated.

Then, the program proceeds to S28, in which the horizontal line n is detected, and in the case of the first program loop, the maximum luminance value Mn of the first line is detected. That is, as shown in FIG. 6, the luminance value of the horizontal line n of the processing window is also sequentially input to the comparator 54, where the value of the first column and the value of the second column in the vertical direction are larger. The maximum luminance value Mn of the horizontal line n is detected by selecting the value, comparing the selected value with the value in the third column, and performing the same processing thereafter. Stored in the corresponding address up to.

Subsequently, the process proceeds to S30, where it is determined whether the maximum luminance value Mn of the horizontal line n exceeds the maximum luminance value Mn-1 of the horizontal line n-1 (previous line) via the comparator 58, and the result is affirmative. If so, the process proceeds to S32, where the maximum luminance value Mn of the horizontal line n is set to the maximum luminance value Mn, and if not, the process proceeds to S34 to set Mn-1 to Mn.

Subsequently, the flow advances to S36, where the horizontal line n is 12
It is determined whether or not the number exceeds 0, and if not, the process proceeds to S38, where the number of horizontal lines is incremented by one.
36 is repeated, and if affirmative, S40
And Mn is set to the maximum luminance value M in the processing window.

Then, the process proceeds to S42, where a threshold value TH (for binarization) is determined from the average luminance value A and the maximum luminance value M.
Specifically, any value between A and M, more specifically, the average luminance value A and the maximum luminance value M are added by the adder 60, and are obtained by dividing by 2 through the divider 62. Average value to threshold T
H. The processes from S16 to S42 are separately performed in the left and right processing windows 40a and 40b, and the threshold value TH is determined separately.

FIG. 8 shows the threshold value TH determined in this way. In this embodiment, the threshold value TH is not obtained from the histogram, but is shown in the form of a histogram for convenience of understanding.

It is considered that the average luminance value A obtained as described above corresponds to the vicinity of the upper limit of the brightness of the road surface portion 16 from which the histogram is obtained, and the maximum luminance value M corresponds to the vicinity of the upper limit of the brightness of the white line 18. . That is, since the white line 18 naturally has a higher luminance than the road surface 16, the processing window 40 is appropriately set, more specifically, by setting the processing window so that the ratio of the white line to the processing window is relatively small, the average luminance value A can be regarded as the luminance of the road surface, and the maximum luminance value M can be regarded as the luminance value of the white line.

Therefore, the threshold value TH is obtained between the average luminance value A and the maximum luminance value M and binarization is performed, so that the threshold value TH can be easily determined, and the white line can be reliably formed. It is possible to cut off from the road surface and improve the white line recognition rate. FIG. 9 shows a case where the input image processing windows 40a and 40b are binarized by the threshold value TH.

Further, since the calculation is performed while taking in the image, the processing can be executed at a high speed. In addition, since the threshold value TH is determined by a simple algorithm, it is easy to implement hardware and can be realized by a small-scale circuit.

Further, an average luminance value is calculated and stored for each horizontal line, and the stored value is added and divided by the number of lines to calculate the average luminance value A of the window, and is larger than the value of the previous line. Since the maximum luminance value M is calculated by storing only the value of the other, not only the calculation can be speeded up, but also the memory capacity can be reduced.

That is, since a histogram is not required, it is not necessary to quantize to 8 bits and store it once. Therefore, the amount of memory can be reduced accordingly, the time for accessing the memory can be reduced, and the operation can be performed at high speed. it can.

Next, a white line detection (recognition) process performed based on the binarized image thus obtained will be described.

FIG. 10 is a flow chart for explaining the operation (operation of the microprocessor 22).
Is a hardware circuit (image processing board 2)
It is a block diagram which shows 4).

First, in step S100, the threshold value TH determined in the processing of the flow chart of FIG. 5 is read, and in step S102, the processing window portion of the input image is binarized, and the image shown in FIG. obtain. Then S
Proceeding to 104, the template 66 is set. That is, in this apparatus, matching between the binarization processing area and the template is determined to detect a white line.

As shown in FIG.
It is composed of 2 pixels × 32 pixels, and has two values, 0 indicating a road surface equivalent portion (black) and 1 indicating a white line equivalent portion (white). The illustrated template is for processing on the left side.

Subsequently, the process proceeds to S106, where the template 66 is placed at the lowermost position near the center of the binarization processing area (processing windows 40a and 40b) as indicated by the imaginary line in FIG.
The horizontal line n of the image data in the template (32 lines from the bottom end in the case of the first program loop) is compared with the template, and the process proceeds to S108 to calculate and store the correlation value. For convenience of understanding, the vicinity of the template 66 in the binary processing area is shown in white.

That is, as shown in FIG. 11, the color of each pixel (32 pixels) of the uppermost horizontal line of the template 66 and the pixel at the corresponding position in the binarization processing area are compared one by one by the comparator 68. Then, the value is set to +1 if they match (both black or white) and 0 if they differ, and the value is added to the adder 7.
0 is added, and the added value is stored in a corresponding address among 3o to 3n of the buffer 72. This added value is called a correlation value, and the correlation value for the horizontal line n is Vn.

Subsequently, the flow advances to S110 to determine whether or not the horizontal line n exceeds the last line (32 lines) of the template. If the result is negative, the flow advances to S112 to increment n, and the flow returns to S106 to perform the same processing. repeat.

On the other hand, if the result in S110 is affirmative, S11
Then, as shown in FIG. 11, the correlation value Vn is added via the adder 74 to calculate the sum V of the correlation values in the template as shown in FIG. In the case of the example, the correlation value (sum) is 1024 if all the matching points are the same.

Subsequently, the program proceeds to S116, in which it is determined whether or not the correlation value (sum) V obtained via the comparator 76 is equal to or greater than the recognition rate threshold value VTH. The recognition rate threshold is as shown in FIG.
The initial value is 70% to 80% of the perfect score, for example, 750). When the result in S116 is affirmative, the process proceeds to S120, in which it is determined that something similar to the template at the position of the template 66, that is, the presence of the white line is recognized.

On the other hand, if the result in S116 is NO, S1
In step S18, the template 66 is moved by one pixel in the left-right direction from the initial position without changing the position in the vertical direction.
Returning to 106, the same processing is performed.

If the result in S116 is NO again, S
Returning to step 118, the same processing is performed by moving one more pixel in the same direction, and thereafter, the processing is repeated until the leftmost (right) end is reached.

When the result in S116 is affirmative, the flow proceeds to S120, and if a white line is detected, the flow proceeds to S122 to set the search window 80. The size of the search window 80 is the same as that of the template 66 (32 pixels) in the vertical direction and m times (m: about 3 to 4 times, specifically, 96 to 128 pixels) the horizontal direction.

Then, the process proceeds to S124, where the recognition rate threshold value V
TH is increased by multiplying by a coefficient k (for example, 1.02), and is increased to 72% to 82% of the full score.

Then, the process proceeds to S126, in which it is determined whether or not the horizontal line n exceeds the number of horizontal lines 120 in the binarization processing area. If the result is negative, the process proceeds to S128 to add the template vertical pixels to the horizontal line n. The template 66 is moved in the vertical direction as shown in FIG.
Returning to step 6, the above processing is repeated. 16 shows the correlation values obtained by FIG.

As shown in the drawing, the template 66 is located in the left search window, and the recognized white line 1
Follow 8 and move diagonally to the upper left rather than directly above. In S118, the template 66 is moved in the horizontal direction only within the range of the search window.

As described above, once the white line 18 is recognized, a search window 80 is set thereon, and
Since the template 66 is moved only in that position, the white line can be quickly detected and recognized, and the amount of calculation is reduced to enable high-speed processing. Further, since the recognition rate threshold value VTH is gradually increased, the detection accuracy can be improved.

Although the template 66 is continuously moved in the vertical direction in FIG. 14, it may be moved in a discrete manner.

As described above, in the white line detection described with reference to FIGS. 10 and 11, since the processing is executed only for the binary image without using the Hough transform, the amount of calculation is reduced and the high-speed processing is performed. And the configuration can be simplified.

Further, since the template is updated based on the latest data and the threshold value is also updated, the processing speed and the detection accuracy can be improved, and the white line can be reliably recognized over a long distance. .

Further, since the template is moved in the horizontal direction or the vertical direction based on the comparison result, the white line can be recognized more reliably, and the template can be moved in the horizontal direction in the search window. Therefore, the processing speed can be further increased.

In this embodiment, as described above, the image pickup means (CCD camera 10) for picking up an image of the traveling road including the traveling road surface in the traveling direction of the vehicle, and a window (processing window 40) in the image picked up by the image pickup means. ), And an average luminance value calculating means (S16) for calculating an average value A of the luminance values in the window.
To S28), a maximum luminance value calculating means (S16, S26 to S4) for calculating the maximum value M of the luminance value in the window.
0) and threshold value setting means (S42) for setting a threshold value TH for binarization based on the calculated average luminance value and maximum luminance value.

Further, a correlation value calculating means (S100 to S114) for comparing the binarized image with the set threshold value and the template 66 and calculating a correlation value V, and calculates the correlation value V Threshold (recognition rate threshold) V
TH comparing means (S116), and white line detecting means (S1) for detecting the white line 18 based on the comparison result.
20).

Further, the white line detecting means includes a template updating means (S118, 128) for updating the template 66 based on the comparison result, and the threshold value VT.
It is configured to include a threshold increasing means (S124) for increasing H.

The template updating means includes means for moving the template in the horizontal direction when the calculated correlation value is determined to be not greater than the threshold value based on the comparison result. (S118), and a template vertical direction moving means (S128) for moving the template in the vertical direction when the calculated correlation value is determined to be not less than the threshold value based on the comparison result. .

The template updating means includes a search window setting means (S122) for setting a search window 80, and the template horizontal direction moving means moves the template horizontally in the search window. Configured.

[0069]

According to the first aspect of the present invention, the threshold value for binarizing an image is not affected by a change other than light and dark changes in the environment or other than a road structure pattern. In addition, the circuit scale can be reduced even when hardware is used.

According to the second aspect, since the processing is executed only for the binary image without using the Hough transform, the amount of calculation is reduced to enable high-speed processing, and the configuration is simplified. be able to.

According to the third aspect, since the template is updated based on the latest data and the threshold value is also updated, the processing speed and the detection accuracy can be improved, and the distance can be reliably increased. The white line can be recognized.

According to the fourth aspect, since the template is moved in the horizontal direction or the vertical direction based on the comparison result, the white line can be recognized more reliably.

According to the fifth aspect, since the template is moved in the horizontal direction in the search window, the processing speed can be further increased.

[Brief description of the drawings]

FIG. 1 is an explanatory perspective view generally showing a vehicle white line detection device according to the present invention.

FIG. 2 is an explanatory block diagram functionally showing the apparatus shown in FIG. 1;

FIG. 3 is an explanatory diagram showing all images obtained by a CCD camera of the apparatus shown in FIG. 1;

4 shows an angle of view and a white line detection range of a CCD camera of the apparatus of FIG. 1 in a driving environment assumed by the apparatus of FIG. 1,
It is a road top view.

FIG. 5 is a flowchart showing an operation of determining a binarization threshold TH in the operation of the apparatus in FIG. 1;

6 realizes the operation shown in the flow chart of FIG. 5,
FIG. 2 is a block diagram illustrating a specific hardware circuit of an image processing board in the apparatus in FIG. 1.

FIG. 7 is an explanatory diagram showing a processing window set for an input image in the processing of the flowchart in FIG. 5;

8 is an explanatory graph showing a binarization threshold value TH obtained by the processing of the flow chart of FIG. 5 with respect to a histogram obtained hypothetically.

FIG. 9 is an explanatory diagram showing an image when binarization is performed on the input image of FIG. 7;

FIG. 10 is a flowchart showing white line detection / recognition in the operation of the apparatus in FIG. 1;

11 is a block diagram showing a specific hardware circuit of the image processing board in the apparatus shown in FIG. 1 for realizing the operation shown in the flowchart of FIG. 10;

FIG. 12 is an explanatory diagram showing a template used in the processing of the flow chart of FIG. 10;

13 is an explanatory graph for explaining a correlation value and a recognition rate threshold value VTH obtained in the processing of the flowchart of FIG. 10;

FIG. 14 is an explanatory diagram showing template movement performed in the processing of the flow chart of FIG. 10;

FIG. 15 is an explanatory graph showing a correlation value obtained by the processing of the flow chart of FIG. 10;

[Explanation of symbols]

 Reference Signs List 10 CCD camera (imaging means) 12 Image processing unit 14 Display unit 16 Road surface 18 White line 22 Microprocessor 24 Image processing board 26 DC / AC inverter 28 VTR / SCAN inverter 40 Processing window 44, 50, 60, 70, 74 Adder 46 , 52, 62 Divider 48, 56, 72 Buffer 54, 58, 68, 76 Comparator 66 Template 80 Search window

Claims (5)

[Claims] 1. A method according to claim 1, Imaging means for imaging a traveling road including a traveling road surface in the traveling direction of the vehicle, b. Window setting means for setting a window in the image picked up by the image pickup means; c. Average brightness value calculating means for calculating an average brightness value in the window; d. Maximum luminance value calculating means for calculating a maximum luminance value in the window; and e. A white line detecting device for a vehicle, comprising: threshold value setting means for setting a threshold value for binarization based on the calculated average luminance value and maximum luminance value. 2. The method according to claim 2, wherein f. Correlation value calculating means for comparing the image binarized by the set threshold value with the template and calculating a correlation value; g. Comparing means for comparing the calculated correlation value with a threshold value; and h. The white line detecting device for a vehicle according to claim 1, further comprising a white line detecting unit configured to detect a white line based on the comparison result. 3. The method according to claim 2, wherein the white line detecting means comprises: i. Template updating means for updating a template based on the comparison result; j. 3. The white line detecting device for a vehicle according to claim 2, further comprising a threshold value increasing unit that increases the threshold value. 4. The method according to claim 1, wherein the template updating unit includes: k. Based on the comparison result, when it is determined that the calculated correlation value is not greater than or equal to the threshold value, a template horizontal moving means for moving the template in a horizontal direction; Based on the comparison result, when the calculated correlation value is determined to be equal to or greater than the threshold value, a template vertical moving unit that moves the template in the vertical direction,
The vehicle white line detection device according to claim 3, further comprising: 5. The template updating means: 5. The vehicle white line detection according to claim 4, further comprising a search window setting means for setting a search window, and wherein the template horizontal moving means moves the template in the search window in the horizontal direction. apparatus.