JP2850608B2 - Roadway detection device for vehicles - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a travel path detecting apparatus for recognizing, by image processing, a road area for automatic traveling in, for example, an automobile or an automatic guided vehicle.

[0002]

2. Description of the Related Art A conventional road area recognition apparatus is disclosed, for example, in JP-A-63-142478. In the above device, after inputting an image from a camera installed in front of the vehicle and extracting white line candidate points,
A straight line is detected by gh conversion, and the straight line is recognized as a white line indicating the end of the traveling road. Further, in this apparatus, as a pre-process for detecting a white line candidate point, a point having a vertical edge is removed as a point other than the white line, thereby increasing the accuracy.

[0003]

However, in the above-described conventional road region recognition apparatus, a point at which black changes to white and then black is set as a white line candidate point, and Hough transform is performed over the entire candidate point. Thus, a method of detecting a straight line has been used. Therefore, (1) it is necessary to perform the Hough transform on the white line candidate points on the entire screen, so that the processing time becomes long. (2) Road edges without white lines cannot be detected. (3) Only linear approximation is performed, and the degree of curve cannot be detected. Therefore, particularly at a sharp curve, an error becomes large when a straight line is approximated. There was a problem.

[0004] In order to solve the above problems, the present applicant has
We have already set up a window based on the result of the previous calculation, tracked an edge with a predetermined slope in the window, and modified the linear expression by linear approximation and time filtering. An application has been filed (Japanese Patent Application No. 2-141269). In this vehicle traveling path detection device, the processing time is short, the response is high, and the lane marker and the traveling lane can be recognized with high reliability. However, in the above-described vehicle travel path detection device, two or three straight lines (a straight line indicating the left end of the road, a lane boundary line, a center line, etc.) on the travel road are independently detected, It may be difficult to accurately determine a line that is partially hidden by other vehicles traveling in front of the vehicle, or a broken line such as a center line or a lane boundary line. If it is vibrating, there is a fear that it cannot be obtained with high accuracy.
There is also a problem that the curve degree cannot be detected in the above (3).

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and is a vehicle traveling path detecting device which is resistant to vibration of a vehicle body, has a high processing speed, and can reliably detect broken lines. The purpose is to provide. Another object of the present invention is to provide a vehicular travel path detecting device capable of detecting a travel path with a small error even on a sharp curve by detecting the degree of a curve of the travel path in addition to the above-mentioned objects. It is to be.

[0006]

Means for Solving the Problems In order to achieve the above object, the present invention is configured as described in the claims. That is, according to the first aspect of the present invention, an image input unit that captures a road image ahead of a vehicle, a preprocessing unit that extracts an edge point, and a window are set near a straight line expression at the time of the previous calculation. A straight line fitting means for measuring coordinates of a plurality of edge points in each set window and obtaining a plurality of straight line formulas by straight line approximation, and an error between the straight line formulas based on the plurality of straight line formulas obtained above. Xy of the vanishing point so that the sum of squares of
And a straight line / vanishing point determining means for estimating the coordinates and the amount corresponding to the inclination of each straight line passing through the vanishing point and obtaining the result of this estimation. The image input means is, for example, as shown in FIG.
The pre-processing means, the straight-line fitting means, and the straight-line / vanishing-point determining means correspond to, for example, an image processing unit 2 in FIG. 2 described later, and are constituted by, for example, a microcomputer. According to the second aspect of the present invention, in addition to the components described in the first aspect,
Edge point tracking means for extracting an edge point corresponding to a lane marker based on the result of the preprocessing means and the detection result of the straight line fitting means, and applying a curve equation to the extracted edge point to calculate a curvature of the curve. A curve fitting means for detecting;
And a smoothing means for smoothing the result of the curve fitting means. The above-mentioned edge point tracking means, curve fitting means and smoothing means are included, for example, in the image processing unit 2 shown in FIG.

[0007]

FIG. 1 is a block diagram showing the functions of the present invention, wherein (a) corresponds to the first aspect of the present invention, and (b) corresponds to the second aspect of the present invention. First, in FIG. 1A, an image input unit 101 captures a road image ahead of a vehicle, and is, for example, a video camera. Further, the preprocessing means 102 is for inputting a signal from the image input means 101 and extracting an edge point. Note that the edge point is a point at which the lightness / darkness expected to correspond to a lane marker or a road edge changes, specifically, a point at which the lightness changes, for example, “dark → bright → dark” or “bright → dark” It is. In the present invention, a lane marker is a linear mark indicating an end of a traveling road, a center line, or the like, such as a white line drawn on a road. In addition, the road edge means a groove, a step, a median strip (grass, tree plant, or the like) serving as a boundary between the edge of the road and another area, which can be distinguished between light and dark on an image. Next, the straight-line fitting unit 103 sets a window near the straight-line equation at the time of the previous calculation, measures the coordinates of a plurality of edge points in each of the set windows, and calculates a plurality of straight-line equations by straight-line approximation. Ask. Note that the straight line fitting is to find a best fit straight line formula by fitting straight lines to a plurality of edge points. Next, the straight-line / vanishing-point determining means 104 calculates the xy coordinates of the vanishing point and the vanishing point based on the plurality of obtained linear equations so as to minimize the sum of squares of the error between the linear equations. An amount corresponding to the slope of each straight line passing is estimated and the result is taken as the current result. Since the output of the straight line / vanishing point determining means 104, that is, the detected straight line corresponds to a lane marker or a road edge, the straight line indicates a running path ahead of the vehicle, and a subsequent display device, automatic steering device, or the like is used. Give to.

In this description, a two-lane road is assumed, and
The process proceeds assuming that the number of lane markers is determined, but the number of lane markers to be determined may be any number as long as it is two or more. Assuming that the road edges are parallel, three (two lanes) lane markers intersect one point on the image. The horizontal coordinate x of the vanishing point (infinity point, a point at which a plurality of straight lines should intersect) is the most important parameter corresponding to the relative angle between the vehicle and the road. However, when three lane markers are independently obtained as in the above-described prior art, they do not always intersect at one point, and therefore, the detection results of the respective lane markers also include errors.
According to the present invention, three lane markers are parallel and the lane width and the lane marker width do not greatly vary with time in order to more accurately estimate the lane marker position with respect to the lane marker or noise hidden by another vehicle or the like. By taking advantage of this fact and assuming that the vehicle makes a large and fast movement in the vertical direction due to the vertical movement of the vehicle, there is no fast movement in the horizontal direction (corresponding to the steering of the vehicle). The left and right ends of the lane markers are fitted with straight lines, and based on the six straight line formulas obtained, the straight line and the vanishing point are obtained by performing temporal and spatial smoothing in consideration of the above assumptions. Things. As a result, in the present invention, it is possible to detect a traveling path that has good consistency and is resistant to fluctuations in the pitch angle and height of the vehicle body.

Next, in the invention according to claim 2,
The edge point is tracked by using the straight line formula and the vanishing point obtained by the configuration according to claim 1, and a curve is applied to the coordinate value of the tracking point. Then, the curvature of the road is calculated from the plurality of curve formulas. It is configured to be required. As described above, according to the second aspect of the present invention, a straight line is detected first, and then the edge direction image is tracked along the edge direction with a point on the straight line as a starting point. Curve information (runway) and curvature information that are resistant to dirt can be obtained.

[0010]

FIG. 2 is a block diagram showing an embodiment of the present invention. In FIG. 2, reference numeral 1 denotes a video camera which is installed in front of a vehicle and captures a road image ahead of the vehicle and converts it into an electric signal. Further, the image processing unit 2 is configured by, for example, a microcomputer, receives a video signal from the video camera 1, performs image processing, and detects a traveling road. The detection result of the image processing unit is sent to an external device such as a vehicle control device (not shown) and used for automatic traveling control and the like. The display unit 3 displays a detection result of the image processing unit 2, and is, for example, a CRT display device or a liquid crystal display device. The processing contents of the image processing unit are the same as those in FIG.
FIG. 3 is a block diagram illustrating the above contents in more detail, as shown in FIGS. FIG.
Are roughly divided into "image input", "pre-processing", "measurement of screen vibration (pitch angle fluctuation) component", "straight line fit",
It can be divided into each part of "determination of each linear equation and vanishing point" and "display". It is basically the same as FIG. 1 (a),
"Measurement of screen vibration component" has been added. Although this part is not essential, the provision of this part can improve the followability of lane marker detection.

Hereinafter, each processing shown in FIG. 3 will be described in detail. In the present embodiment, a case will be described in which a two-lane road is assumed and three lane markers (linear markers drawn on the road) drawn by white lines or yellow lines are obtained. May be any number as long as it is two or more. The image is a 256 × 240 monochrome light and shade image, which is a continuously processed image. Then, the current time is represented by k. Further, the six straight lines are represented by a straight line i or a straight line formula i (i = 1 to 6),
Lines 1 (i = 1) and 2 (i = 2) are the left and right ends of the leftmost lane marker, lines 3 and 4 are the left and right ends of the center lane marker, and lines 5 and 6 are the rightmost lane markers. The left and right ends of the lane markers are respectively represented. The detection results of the three lane markers are obtained as an average value of two straight lines at the left end and the right end, respectively, and are represented by a subscript j in this case. Therefore, j = 1 to 3.

(1) Image input and pre-processing In pre-processing, the direction of an edge is determined by a density gradient. That is, for the input image A (x, y) at each time k, one of the x direction and the y direction is
The second derivative is obtained, and is set as S _x (x, y) and S _y (x, y), respectively. The edge direction image G (x, y) is expressed as G (x, y) = sign { _Sx (x, y)} · tan ~ ¹ { _Sx (x, y) / _Sy (x, y)}. Given. However, it is determined that the point of | S _x (x, y) | + | S _y (x, y) | <C _s (C _s = threshold) is not an edge point. Since the pre-processing as described above can be executed at a high speed in a pipeline type image processing apparatus, the calculation may be performed simultaneously with the image input.

(2) Measurement of Screen Vibration Component Assuming that the gradient of the road surface does not change, it is considered that the lane marker position on the image moves slowly in the x-axis direction with the steering of the vehicle. However, the lane marker position moves sharply also in the y-axis direction due to the screen vibration caused by the pitch angle fluctuation. Therefore, in order to improve the followability of the lane marker detection, the screen vibration component is measured independently of the lane marker detection, and is used in estimating a vanishing point and each straight line later. At time k = 0, the edge direction image G (x,
y) is scanned from bottom to top along the y axis, and the y coordinate of the edge point found first is set as an initial value as Y (x, 0). Also, R (x, 0) representing the number of continuous edges is cleared to zero. At time k, Y (x, k- 1) is also scanned from bottom to top the G (x, y) within the range of the width C _y around the, edge points y coordinate Y (x, k) Get. Then, R (x, k) = R (x, k-1). If no edge points are present in the range of C _y, determine the Y (x, k) is scanned again from the lowest point, R
(x, k) = 0. Edges that have a direction parallel to the traveling direction of the vehicle, such as lane markers and the upper ends of guardrails, are considered to have apparently no change in position on the screen even while the vehicle is running, so only movement in the y-axis direction due to screen vibration is observed. Is done. R (x, k) is used to select such an edge point, that is, an edge point whose position changes little with time, and indicates how many frames of the edge appear continuously. Y (x, k) and R
After (x, k) is obtained over the entire screen, Y (x, k) −Y (x, k−1) is averaged at x where R (x, k)> C _c with a threshold value C _c. To be the screen vibration speed v (k). That is, v (k) = Y (x, k) -Y (x, k-1) for {x | R (x, k)> C _c }. It is not necessary to scan the y-axis densely.
Thinning scanning is possible for each _p line. Since the screen vibration component appears on the y coordinate of the vanishing point (infinity point, a point where a plurality of straight lines should intersect at one point), v (k) is used as one parameter for obtaining the vanishing point.

(3) Straight Line Fit First, in the tracking area as shown in FIG.
A candidate point of each straight line is searched from (x, y). Then, straight line fitting is performed from the coordinates of the candidate points, and each straight line equation is obtained.
Incidentally, in FIG. _{4, (x s (k-} 1), y s (k-1))
Is the coordinates of the vanishing point obtained in the previous calculation (previous frame). The second lane marker from the left indicates a broken line such as a center line. As shown in FIG. 4, the search area of the candidate point is performed in three areas based on the result of the lane marker detection of the previous frame. That is, the straight line 1
And the candidate point of the straight line 2 are searched in the same area (j = 1). Lines 3 and 4 (region j = 2), lines 5 and 6 (region j =
The same applies to 3). The three search areas are distinguished by a subscript j. Also, the boundary in the x-axis direction between two adjacent regions is
As shown in FIG. 4, straight lines XM1 and XM2 connecting the midpoints of the two lane marker detection results of the previous frame are set. In addition, the upper limit of the y-axis of each area is represented by the coordinates of the vanishing point of the previous frame.
if _s (k-1), and _{y s (k-1) +} v (k) + y tj (j = 1~3). An n _j line is searched for every y _pj lines from each upper limit. y _tj , y _pj and n _j are set for each area. Assuming that the vehicle is traveling in the left lane, the rightmost lane marker has a nearly horizontal inclination, and the front of the lane marker protrudes from the screen. As captured reliably even in such a lane marker, y _pj corresponding search area is set to the upper than the other regions, the scanning is closely performed. The search scan is performed outward from the center of the screen. This is to mitigate the influence of mixed edge points of other vehicles and noise outside the road.
That is, the left area (j = 1) goes from right to left,
The right area (j = 3) goes from left to right, and the central area (j = 2) goes from left to right while traveling in the left lane and from right to left while traveling in the right lane. The candidate points are edge points that first appear in the scanning of each line and satisfy the following conditions, and their coordinate values are stored. The above conditions are as follows. The straight line 1 and the straight line 2 rise to the right, and are respectively (π / 8) <G (x, y) <(3π / 8) (Line 1) (−7π / 8) <G (x, y) <(− 5π / 8) (Line 2) is an edge point having a gradient, and Lines 3 and 4 are (π / 6) <G (x, y) <(5π / 6) (Line 3) (−5π / 6) ) <G (x, y) <(− π / 6) (Line 4) An edge point having a gradient of: (5π / 8) <G (x, y) <(7π / 8) (Line 5) (−3π / 8) <G (x, y) <(− π / 8) (Line 6) An edge point having a gradient:

After six sets of candidate points are obtained from the three search areas as described above, straight line fitting is performed to obtain six straight line equations. That is, any two points are selected from the candidate points, and how many other candidate points are on the line connecting the two points is counted. Then, of the combinations of all two points, the two points giving the largest value of the count number are determined as the end points of the straight line. Further, normalized number of counts by the constant n _c, and the probability p _i of the straight line i (0 ≦ p _i
≦ 1). As a result of the straight-line fit such above, to obtain a linear equation _{x = a i '(k)} · y + b i' (k) (i = 1~6). Hereinafter, a _i '(k) and b _i' (k) merely a _i for ', b _i' referred to.

(4) Determination of vanishing point and straight line formula Assuming that the three lane markers are parallel on the road surface and the lane marker width is constant, the coordinates of the point at infinity on the image, that is, the vanishing point Is used to estimate each straight line and the vanishing point so that the six straight lines always intersect. Assuming the above, since each straight line passes through the vanishing point, the parameter to be obtained is
The coordinates x _s (k) and y _s (k) of the vanishing point, and the slope a _i (k) as parameters of the linear expression i. Here, d
_i (k) = put with a _i (k) · y _c to estimate the d _i (k). This is for unifying the unit of the variable to the pixel. The vanishing point and each straight line are estimated to best satisfy the following conditions. The equation of the result of the straight-line fitting is represented by x = a _i ′ · y + b _i ′.

The conditions are as follows (a) to (f). (A) The x-coordinate value a _i ′ · y _s (k) + b _i ′ of the result of the straight line fitting at y = y _s (k) coincides with x _s (k). (B) x of the line fitting result at y = y _s (k) + y _c
The coordinates a _i ′ ｛y _s (k) + y _c ｂ + b _i ′ are x _s (k) + d
_i (k) (see FIG. 5). (C) x _s time variation x _s of _{(k) (k) -x s} (k-1)
Is small (ie, the steering of the vehicle is assumed to be smooth). (D) Time change amount d _i (k) −d _i (k−
1) is small (that is, it is assumed that changes in road shape and vehicle steering are smooth). (E) a unit time variation of _{y s (k) y s (} k) -y s (k-
1) coincides with the screen vibration speed v (k). (F) The difference d _{i + 1} (k) −d _i (k) between the inclinations of adjacent straight lines is temporally constant (that is, the lane width and the lane marker width are assumed to be constant).

Next, the error for each of the above conditions is defined as follows. Error _{E ai = x s (k)} -a i '· y s (k) -b i' error E _bi = x _s of (b) (k) + d i (k) -a i of (a) '{ y
_{s (k) + y c}} -b i ' error E _ci = d _i of the error in _{(c) E x = x s} (k) -x s (k-1) (d) (k) -d i (k -1) (error of _{e) E y = y s (} k) -y s (k-1) -v (k) ( the error in _{f) E di = d i +} 1 (k) -d i (k) −g _{i + 1} ,
_i (k) Note that g _{i + 1} , _i (k) in the above error (f) is d
_{i + 1} (k) is a temporal average value of _{-d i (k), g i} + 1, i (k) = (1-w g) {d i + 1 (k-1) -d i (k−1)} + w _g · g _{i + 1} , _i (k−1). However, i = 1 to 6, but when i = 6, i + 1 = 1. W _g is a smoothing time constant (0 <w _g <1).

Next, the above errors are squared, weighted and summed. If the value is set to E ² , E ² is expressed by the following (Equation 1).

[0020]

(Equation 1)

In the above equation (1), w _pi is made to correspond to the probability p _i of the straight line fit of a straight line i. Also, w _q and w
_x corresponds to a time constant of temporal smoothing. w _y is the weight of the screen vibration component measurement result with respect to the straight line fitting result. And w _w is the weight of the lane width and lane marker width, so to speak a spatial smoothing coefficients, respectively set to an appropriate value. Next, _di (k), x _s (k), and y _s (k) are estimated so as to minimize the sum E ² of the errors represented by the above (Equation 1).
This minimization can be performed by partially differentiating each variable to zero. As a result, a simultaneous linear equation expressed by the following equation (2) is obtained.

[0022]

(Equation 2)

Where w _i = w _pi + w _q + 2w _w w _ai = −w _pi · a _i ′. The simultaneous linear equation expressed by the above (Equation 2) can be easily solved, and the vanishing point x _s (k) at time k,
y _s (k) and a straight line i passing through the vanishing point are obtained.

The detection result of the three lane markers is an average of the detection results of two straight lines corresponding to the left and right ends of each lane marker. That is, the inclination of the lane marker j is expressed by f
When _{j (k) / y c,} {(d 2i-1 (k) + d 2i (k)) / 2} is _{-x s (k) i = 1~3} . The values f _j (k) (j = 1 to 3) corresponding to the vanishing points x _s (k) and y _s (k) and the inclination of the lane marker are defined as lane marker detection results.

(5) Identification of Traveling Lane Although the identification of the traveling lane is not described in FIG. 3, the identification of the currently traveling lane can be easily performed as follows. The identification of the traveling lane is performed by determining the scanning direction of the center area in the candidate point search and the upper limit (y
_tj) and search line number and (n _j), is used as information for determining. The identification is simply the center lane marker (j =
This is performed by the sign of the slope f ₂ (k) in 2). That is,
If f ₂ (k) is positive, it can be determined that the vehicle is traveling in the left lane, and if f ₂ (k) is negative, it can be determined that the vehicle is traveling in the right lane.

Next, a second embodiment of the present invention whose block diagram is shown in FIG. 1B will be described. In this embodiment, a configuration for detecting the degree (curvature) of a curve of a curved lane marker is added to the embodiment of FIG. In FIG. 1B, the image input unit 101, the preprocessing unit 102, the straight line adaptation unit 103, and the straight line / vanishing point determination unit 104 are the same as those described in FIG. 1A and FIGS. Are identical. In this embodiment, the above 10
1 to 104, an edge point tracking means 105,
A curve fitting means 106 and a smoothing means 107 are provided. Hereinafter, the added portion will be described.

(1) Edge Point Tracking Edge point tracking is for finding candidate point coordinates for curve fitting, and is performed in order to minimize the influence of disturbance (edge points of other vehicles or dirt on the road surface). This is performed based on the edge points obtained by the preprocessing unit 101 and the straight line formula obtained by the straight line fitting unit 103. FIG. 6 is a diagram showing an image for explaining edge point tracking. In FIG.
Is a left end line of the first lane marker, is a straight line corresponding to the right end of the first lane marker, is a straight line corresponding to the left end of the second lane marker, is a right end line of the second lane marker,
Is a straight line corresponding to the left end of the third lane marker, and is a right end line of the third lane marker. Note that the second lane marker is a center line and is a broken line as shown in the figure. In addition, thick arrows indicate edge point tracking. In a certain straight line, the value of x at y _{max at the} lower part of the screen is obtained from a straight line equation and is set to x ₀ . That is, x ₀ = a
_i y _max + b _i (i = 1 to 6). Then, it is checked whether or not B (x ₀ , y _max ) is an edge point in the preprocessed image B (x, y). If it is not an edge point, B (x ₀ +
1, y _max ) and B (x ₀ -1, y _max ). If there is an edge point, the coordinates of that point are x ₁ , y ₁
To be stored. If there is no edge point, N representing the number of edge points is set to 0, and the process is terminated. Next, x ₁ , y ₁
Is used as a starting point to perform edge tracking. B (x _1, y ₁₎
Represents the directional angle of the edge, so that the image is tracked upward in the direction indicated by the edge direction. Specifically, assuming that the current coordinates are x _j and y _j ,
X _{j + 1} and y _{j + 1 given} by the equation are obtained, and a point B (x _{j + 1} ,
Check whether y _{j + 1} ) is an edge point.

(Equation 3) 0 ° ≦ B (x _j , y _j ) ≦ 45 ° When x _{j + 1} = x _j + m ｛−180 ° ≦ B (x _j , y _j ) <− 135 ° y _{j + 1 = y j -m ·} tanB (x j, y j) 45 ° <B (x j, y j) ≦ 90 ° x j + 1 = m · tan {90 ° -B (x j, y j )} + X _j ｛−135 ° ≦ B (x _j , y _j ) <− 90 ° y _{j + 1} = y _j− m 90 ° <B (x _j , y _j ) ≦ 135 ° x _{j + 1} = x _j −tan {B (x _j , y _j ) −90 °} ｛−90 ° ≦ B (x _j , y _j ) <− 45 ° y _{j + 1} = y _j −m 135 ° <B (x _j , y _j ) ≦ 180 ° x _{j + 1} = x _j −m ｛−45 ° ≦ B (x _j , y _j ) <0 ° y _{j + 1} = y _j −m · tanB (x _j , y _j ) In the expression (3), j is a suffix indicating the number of an edge point with j = 1 as an initial value. It is appropriate that the constant m is about 2 to 8 pixels. As a result of the above, if B (x _{j + 1} , y _{j + 1} ) is an edge point, the coordinates x _{j + 1} , y
_{j + 1} is stored, and the process is repeated with j + 1 as j. If it is not an edge point, the neighboring pixels are checked within a range of ± 1 pixel. If there is still no edge point, the process is terminated with N = j. When there is an edge point in the vicinity, its coordinates are x
_{j + 1} and y _{j + 1} are stored and j = j + 1 is repeated.
In the above processing, the lane marker and the road edge are
Considering that the bend is not large, the edge point can be restricted. For example _{B (x j, y j)} -α <B (x j + 1, y j + 1) <B (x j, y j) only when the range of + alpha, it is determined that the edge point. Α is suitably about 15 ° to 30 °. Due to this limitation, edge point tracking that is resistant to noise and mixing of other vehicles can be performed. In the case of the example shown in FIG. 6, since the second lane marker is a broken line, the tracking of the edge point is interrupted.

(2) Curve Fitting When the above processing is performed for each straight line, three sets of edge point coordinate sequences are obtained. Curve fitting is performed for each set. The curve fit is as follows (Equation 4)
Where coefficients c _i , d _i , e _i (i = 1 to 1)
6).

[0030]

(Equation 4)

[0031] Using the above curve equation, the coefficient d _i is determined as an amount in proportion to the actual curvature of the road. The above curve fitting can be easily obtained by the least square method. That is, assume that the edge point sequence is x _j , y _j, and there are N points of data. Also, placing r _j = y _j -y _s, and _{s j = 1 / (y j} -y s). Y _s in the above equation is the y coordinate of the vanishing point previously determined. For short subscript _{i, e j = c · r} j + d · s j + believed error of e-x _j, c which minimizes the following equation (5), d, e
Should be obtained.

[0032]

(Equation 5)

In order to obtain c, d, and e that minimize the above equation (5), the following equation (7) is obtained from the following equation (6). I just need to solve it.

[0034]

(Equation 6)

[0035]

(Equation 7)

By solving the above equation (7), c,
d and e are obtained as shown in the following equation (8).

[0037]

(Equation 8)

In the above equation (8), D is as shown in the following equation (9).

[0039]

(Equation 9)

Through the above processing, the curve parameters c _i , d _i , and e _i (i = 1 to 6) for each lane marker are obtained. In addition, as the likelihood, each edge point number _Ni is used.

Next, the degree of curvature (curvature) is determined. If three lane marker is assumed to be drawn in parallel, the value of the quantity d _i corresponding to the curvature on the image is an approximation equal _{(d 1 = d 2 = d} 3). In the edge point tracking described above, a broken line such as a center line cannot be traced to a distant place, so that a curve-type matching result is not always obtained with good accuracy. Therefore, the curvature by averaging the values of the respective three curves d _i. In this case using N _i as probability.

[0042] The averaged curvature of the d _a result, d _a is calculated by the following equation (10) below.

[0043]

(Equation 10)

Note that averaging the curvatures of the three curves as described above corresponds to spatial smoothing.

(3) Smoothing Next, taking advantage of the fact that the curvature does not greatly change over time, temporal smoothing is performed. That is, the smooth curvature at the time t is d (t), and the value represented by the following equation (11) is N (t).

[0046]

[Equation 11]

Next, by normalizing the above N (t), P
Find (t). For example However, if T ₁ and T _{2 are} appropriate thresholds set in advance, 0 ≦ P (t) ≦ 1. Smoothed d
(t) is the d (t) = {1- P (t)} · d (t-1) · β + P (t) · d a. Here, β is a constant close to 1 (appropriate value of about 0.9 to 0.99). When P (t) is not large continuously, instead of using the curvature obtained previously, , So as to gradually correct the curvature to zero, that is, to approach a straight line. That is, if the edge points can be tracked sufficiently, a faithful curvature can be obtained, and in an environment where the edge points cannot be tracked, the result of the straight line detection is output. When β = 1, if P (t) = 0, the previous value d (t−1) is output as it is.

Finally, three final curve equations are obtained by combining the curvature d (t) and the result of the straight line detection. The reason why the curve fitting result (c _i , e _i ) is not used as the curve equation is that the reliability of edge point tracking is lower than that of straight line detection. The linear equation and the curve equation have a relationship as shown in FIG. In FIG.
Indicates a straight line, and indicates a curve. As shown in FIG. 7, in the range at the lower part of the screen, the linear equation and the curve equation coincide. _Let y _p be the approximate center coordinate of the y coordinate when determining the linear equation. This y _p may be an average value of the upper limit (y _min ) and the lower limit (y _max ) of the tracking area. Further, when the x-coordinate of y _p in the linear i and x _pi, linear equation becomes _{x pi = a i · y p} + b i (i = 1~6). The curve to be determined passes through the point (x _pi , y _p )
Assuming that the inclination at that point is a _i and the curvature is d (t) (hereinafter simply referred to as d), the equation to be obtained is shown by the following (Equation 12).

[0049]

(Equation 12)

The coefficients f _i and g _i of the curve equation can be obtained by performing the calculation represented by the following equation (13) from the equation (12).

[0051]

(Equation 13)

As shown in FIG. 8, g _i should be essentially equal in the three curves. That is,
Three straight lines x = f _i (y−y _s ) + g _i (i = 1 to 6) converge at a point (g _i , y _s ). Therefore, g _i
May be averaged to obtain g. g is represented by the following equation (14).

[0053]

[Equation 14]

[0054] _{Further, f 2 -f 1 = f 3} -f 2 corresponds to the width of the road. Therefore, if the most reliable curve among the three is obtained from the certainty, and the obtained curve is, for example, the curve 1, the curves 1, 2, and 3 may be expressed by the following (Equation 15).

[0055]

(Equation 15)

In equation (15), w is a constant corresponding to the road width (lane width).

[0057]

As described above, in the present invention, after extracting the edge points by the preprocessing, a total of six straight lines (in the case of two lanes) corresponding to the left and right ends of each lane marker are first obtained, and thereafter, Since each straight line is determined so that all the straight lines intersect one point at the vanishing point, the pre-processing is simple in hardware and can be executed at high speed. The post-processing (processing after the pre-processing) has a simple processing algorithm, so that the processing time can be reduced. That is,
It is not necessary to scan the entire screen for tracking candidate points, etc., straight line fitting can be performed at high speed if the number of candidate points is small, and the linear equation can be determined by solving an 8-ary simultaneous linear equation. This is a fast process. At the vanishing point, it is possible to perform detection with high accuracy and high resistance to the influence of other vehicles and interruption or blurring of the lane marker. And other excellent effects.

In the apparatus for detecting a curve according to the second aspect, after detecting a straight line, an edge point is tracked, and a curve is applied to a coordinate value of the tracking point, and then the curvature of the road is calculated from a plurality of curve expressions. With this configuration, a straight line is detected first, and then the edge direction image is tracked along the edge direction starting from a point on the straight line. (Runway) and curvature information can be obtained. By setting the curve equation to be of the type x = ay + (b / y) + c, the value of b becomes almost equal to the curvature of the road, and by averaging over a plurality of lanes, it is possible to obtain a certain curvature. In the time smoothing of the curvature, a certainty is used, and when the certainty is small, a straight line is approached, so that reliability can be maintained when used for control of an autopilot vehicle or the like. And other excellent effects.

[Brief description of the drawings]

FIG. 1 is a block diagram showing functions of the present invention.

FIG. 2 is a block diagram of one embodiment of the present invention.

FIG. 3 is a functional block diagram of one embodiment of the present invention.

FIG. 4 is a view showing an image for explaining straight line fitting;

FIG. 5 is an xy coordinate diagram for explaining a linear equation and a vanishing point.

FIG. 6 is a diagram showing an image for explaining edge point tracking.

FIG. 7 is a view showing an image for explaining smoothing.

FIG. 8 is a view showing an image for explaining smoothing.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Video camera 2 ... Image processing part 3 ... Display part 101 ... Image processing means 102 ... Preprocessing means 103 ... Straight line fitting means 104 ... Straight line / vanishing point determination means 105 ... Edge point tracking means 106 ... Curve fitting means 107 ... Smoothness means

Claims (2)

(57) [Claims] 1. An image input means for capturing a road image ahead of a vehicle, a preprocessing means for inputting a signal from the image input means and extracting an edge point, and a window near a straight-line equation at the time of a previous calculation. Is set, the coordinates of a plurality of edge points in each of the set windows are measured, and straight line fitting means for obtaining a plurality of straight line formulas by straight line approximation. Straight line / vanishing point determining means for estimating the xy coordinates of the vanishing point and the amount corresponding to the slope of each straight line passing through the vanishing point so as to minimize the sum of squares of the error between the equations; A travel path detection device for a vehicle, comprising: 2. An image input means for capturing a road image ahead of a vehicle, a preprocessing means for inputting a signal from the image input means and extracting an edge point, and a window near a straight line expression at the time of the previous calculation. Is set, the coordinates of a plurality of edge points in each of the set windows are measured, and straight line fitting means for obtaining a plurality of straight line formulas by straight line approximation. Straight line / vanishing point determining means for estimating the xy coordinates of the vanishing point and the amount corresponding to the slope of each straight line passing through the vanishing point so as to minimize the sum of squares of the error between the equations; An edge point tracking means for extracting an edge point corresponding to a lane marker based on a result of the preprocessing means and a detection result of the straight line fitting means; and applying a curve equation to the extracted edge point to obtain a curvature of the curve. Detect A curve fitting unit for a vehicle traveling road detecting apparatus characterized by comprising a smoothing means for smoothing the results of the curve fitting means.