JP6404748B2 - Traveling line recognition device - Google Patents

Description

  The present invention relates to a travel lane marking recognition device for recognizing a road lane marking based on an image taken by an in-vehicle camera.

  In a device that recognizes a travel lane line based on an image taken by an in-vehicle camera, in a road environment where it is difficult to recognize a travel lane line such as a snowy road, Techniques for suppressing control using lines have been proposed.

  For example, in the lane boundary detection device described in Patent Document 1, it is determined whether or not the road is a snow road with a hail based on the likelihood of a snow hail. When it is determined that the lane boundary detection device is a snowy road with a fence, the lane boundary detection device uses the detection of the lane marking, the output of the detected lane marking, and the detected lane marking. Suppressed control.

JP2013-250874A

  In the lane boundary line detection device, in order to stably determine the snowfall, the snowfall likelihood is calculated using the history of the snowfall likelihood. Therefore, even if the road environment is improved and a clear lane marking suddenly appears, a time lag occurs until the lane marking is recognized and used for driving support control, and the user feels uncomfortable. .

  In view of the above circumstances, the present invention has as its main object to provide a travel lane marking recognition device capable of improving the maintainability of a state in which a travel lane line recognition result is output.

  In order to solve the above problems, the invention according to claim 1 extracts and extracts lane line candidates that are candidates for travel lane lines that divide a road lane from an image taken by a camera mounted on the vehicle. A lane marking recognition device for recognizing the lane marking candidate as the lane marking; first determination means for determining the likelihood of the lane marking of the lane line candidate extracted from the image; Second determination means for determining the likelihood of the lane marking of the lane marking candidate, and road surface determination means for determining, from the images of a plurality of frames, disorder of the road surface condition of the road that prevents recognition of the lane marking. And the road surface determination means determines that the road surface state is not messy, and the first determination means determines that the lane line candidate is likely to be the travel lane line. Output when the road surface determination means determines that the road surface state is messy, and outputs or stops the output of the recognition result. The second determination means determines the likelihood of the travel lane line based on the feature amount including the feature amount of the travel lane line related to the clarity of the travel lane line, and determines the likelihood of the travel lane line. The road surface is provided on condition that an instantaneous determination condition at the time of determination is stricter than that of the first determination unit, and the second determination unit determines that the lane line candidate is likely to be the travel lane line. It is easy to make a transition from a state where the road surface state is determined to be messy by the determination means to a state where it is determined that the road surface state is not messy, or the road surface state is messy by the road surface determination means. Be determined, to output the recognition result by said output means.

  According to the first aspect of the present invention, the first determination unit determines whether the extracted lane line candidate is a running lane line. Further, the second determination unit determines the likelihood of the lane line candidate of the lane line candidate using an instantaneous determination condition that is stricter than that of the first determination unit, based on the feature amount including the feature amount relating to the clarity. Further, the road surface determination means determines the disorder of the road surface state of the road from the images of a plurality of frames.

  Then, when the road surface determination means determines that the road surface state is not messy and the first determination means determines that the lane line candidate is likely to be a travel lane line, the recognition result of recognizing the lane line candidate as a travel lane line Is output. Therefore, when the road surface state is not in a state that prevents recognition of the lane marking, the recognition result can be used for control. In addition, when the road surface determination unit determines that the road surface state is messy, the recognition result is not output. Therefore, when the road surface state is a state that hinders recognition of the travel lane marking, it is possible to suppress the recognition result that may be erroneously recognized from being used for the control.

  In addition, only when the second determination unit determines that the lane line candidate is likely to be a traveling lane line, the second determination unit can easily transition from the state in which the road surface state is determined to be messy to the state in which it is determined not to be messy. Or even if it is determined that the road surface state is messy, the recognition result is output. Therefore, when a clear traveling lane line suddenly appears from a messy road surface state, it is possible to quickly shift from a state where the recognition result of the traveling lane line is not output to a state where the recognition result is output. As a result, the state in which the recognition result is output can be maintained for a longer time.

  According to the second aspect of the present invention, a lane line candidate that is a candidate for a lane line that divides a road lane is extracted from an image taken by a camera mounted on the vehicle, and the extracted lane line candidate is extracted. Is a travel lane line recognition device that recognizes the lane line candidate of the lane line candidate extracted from the image, and the lane line candidate extracted from the image. The recognition result is determined to be abnormal based on a comparison between a second determination unit that determines the likelihood of the travel lane line and a recognition result obtained by recognizing the lane line candidate as the travel lane line and an abnormal threshold. When it is determined that the lane marking candidate is likely to be the travel lane marking by the abnormality determination means and the first determination means, and the abnormality determination means determines that the lane marking candidate is not abnormal, An output unit that outputs a recognition result recognized as a row lane line, and stops or holds the output of the recognition result when the abnormality determination unit determines that there is an abnormality, and the second determination unit includes Determining the likelihood of the travel lane line based on the feature amount including the feature amount of the travel lane line relating to the clarity of the travel lane line, and instantaneously determining the likelihood of the travel lane line The determination condition is stricter than the first determination means, and the abnormality determination means determines that the recognition result is more abnormal as the traveling block line is more likely to be determined by the second determination means. The abnormality threshold is changed so as to make it difficult to perform.

  According to the second aspect of the present invention, the first determination unit determines whether the extracted lane line candidate is a running lane line. Further, the second determination unit determines the likelihood of the lane line candidate of the lane line candidate using an instantaneous determination condition that is stricter than that of the first determination unit, based on the feature amount including the feature amount relating to the clarity. Further, the abnormality determination means determines that the recognition result is abnormal based on a comparison between the recognition result obtained by recognizing the lane line candidate as the travel lane line and the abnormality threshold value.

  Then, when the first determination unit determines that the lane line candidate is likely to be a travel lane line, and the abnormality determination unit determines that the recognition result is not abnormal, the recognition result of recognizing the lane line candidate as the travel lane line is Is output. Therefore, when it is determined that the recognition result is normal, the recognition result can be used for control. Further, when the recognition result is determined to be abnormal by the abnormality determination means, the recognition result is not output. Therefore, when it is determined that the recognition result is abnormal, it can be suppressed that the recognition result is used for control.

  Furthermore, the abnormality threshold is changed so that the recognition line is less likely to be determined to be abnormal as the lane marking candidate determined by the second determination unit is more likely to travel. Therefore, when a clear lane line is seen, there is a low risk of misrecognizing the lane line, so the abnormality threshold is tightened and the output of the lane line recognition result is less likely to be stopped or suspended. The Therefore, when a clear travel lane line is seen, the output of the recognition result of the travel lane line can be maintained.

  According to a third aspect of the present invention, a lane line candidate that is a candidate for a lane line that divides a road lane is extracted from an image photographed by a camera mounted on a vehicle, and the extracted lane line candidate is extracted. Is a travel lane line recognition device that recognizes the lane line candidate of the lane line candidate extracted from the image, and the lane line candidate extracted from the image. Second determination means for determining the likelihood of the travel lane line, recognition means for calculating a parameter representing the state of the travel lane line from the lane line candidate, and recognizing the lane line candidate as the travel lane line; An abnormality determining means for determining that the parameter calculated by the recognizing means is abnormal; a case where the abnormality determining means determines that the parameter is abnormal; and When the parameter is not calculated, the prediction unit that predicts the parameter based on the parameter when the abnormality determination unit determines that the parameter is not abnormal, and the first determination unit determines the lane line candidate as the parameter. When it is determined that it is likely to be a lane marking, and when it is determined by the abnormality determination means that there is no abnormality, the parameter calculated by the recognition means is output, and when the abnormality determination means determines that it is abnormal When the parameter is no longer calculated by the recognition unit, the parameter predicted by the prediction unit is output continuously for a predetermined time, and then the output of the parameter predicted by the prediction unit is stopped. Output means, wherein the second determination means is related to the clarity of the lane marking. Based on the feature amount including a feature amount of a row lane line, the likelihood of the travel lane line is determined, and an instantaneous determination condition when determining the likelihood of the travel lane line is the first determination unit. The output means increases the predetermined time as the likelihood of the travel lane line determined by the second determination means when the parameter is calculated by the recognition means is higher.

  According to the invention described in claim 3, the first determination unit determines whether the extracted lane line candidate is a running lane line. Further, the second determination unit determines the likelihood of the lane line candidate of the lane line candidate using an instantaneous determination condition that is stricter than that of the first determination unit, based on the feature amount including the feature amount relating to the clarity. In addition, a parameter indicating the state of the travel lane line is calculated from the lane line candidate, and the lane line candidate is recognized as the travel lane line. Also, the abnormality of the parameter calculated by the abnormality determining means is determined. Further, when it is determined that the parameter calculated by the abnormality determination unit is abnormal, and when the parameter is not calculated, the parameter is determined based on the parameter when it is determined normal by the abnormality determination unit. Is predicted.

  Then, when the first determination unit determines that the lane line candidate is likely to be a traveling lane line and the parameter calculated by the abnormality determination unit is determined to be normal, the calculated parameter is output. Further, when it is determined that the calculated parameter is abnormal by the abnormality determination unit, and when the parameter is not calculated by the recognition unit, the predicted parameter is output continuously for a predetermined time, Thereafter, the output of the predicted parameter is stopped. This predetermined time is made longer as the lane marking candidate running lane line determined by the second determination means is higher. Therefore, when a clear travel lane line is seen, the accuracy of the prediction parameter is high, so the output time of the prediction parameter can be extended and the parameter output can be maintained.

The figure which shows the mounting position of a vehicle-mounted camera. The block diagram which shows the function of the white line recognition apparatus which concerns on 1st Embodiment. The figure which shows the road surface where the clear white line appeared suddenly. The bird's-eye view which shows the road surface where the clear white line appeared suddenly. The figure which shows a clean road surface. The figure which shows the road surface where snow accumulated. The figure which shows the wet road surface. The flowchart which shows the process sequence which outputs a white line parameter. A subroutine showing a second processing procedure for calculating the likelihood of white lines. The subroutine which shows the 1st process sequence which calculates white line likelihood. The block diagram which shows the function of the white line recognition apparatus which concerns on 2nd Embodiment. The block diagram which shows the function of the white line recognition apparatus which concerns on 3rd Embodiment.

  Hereinafter, each embodiment which embodies the travel lane marking recognition device will be described with reference to the drawings. In the following embodiments, parts that are the same or equivalent to each other are denoted by the same reference numerals in the drawings, and the description of the same reference numerals is used.

(First embodiment)
First, with reference to FIG.1 and FIG.2, the traveling lane marking recognition apparatus which concerns on this embodiment is demonstrated. The travel lane marking recognition apparatus according to the present embodiment recognizes a white line (travel lane marking) that divides the left and right lanes from a front image captured by the in-vehicle camera 10 and outputs a recognition result to the travel support device 30. It is an in-vehicle device. The travel lane marking recognition apparatus according to the present embodiment is configured by an ECU 20. In the present embodiment, a running line such as yellow other than white is also included in the white line.

  The in-vehicle camera 10 includes at least one of a CCD image sensor, a CMOS image sensor, and the like. As shown in FIG. 1, the in-vehicle camera 10 is mounted, for example, in the vicinity of the upper end of the windshield of the vehicle 50, and images a region that extends in a predetermined angle range toward the front of the vehicle. That is, the in-vehicle camera 10 captures the surrounding environment including the road ahead of the vehicle 50.

  The driving support device 30 is a departure warning device that warns a lane departure based on the recognition result of the white line output from the ECU 20 or a driving support device that performs driving support. When the driving support device 30 is a departure warning device, it is configured as a human machine interface such as a display, a speaker, and a vibrator, and outputs a warning to the driver when the vehicle 50 departs from the lane. In addition, when the driving support device 30 is a driving support device, the driving support device 30 is configured as a steering actuator or a braking actuator, and performs steering control and brake control of the vehicle 50.

  The ECU 20 is a computer that includes a CPU, a RAM, a ROM, an I / O, a storage device, and the like. The CPU implements various functions stored in the ROM, thereby realizing the functions of the first determination unit 21, the second determination unit 22, the road surface determination unit 23, the recognition unit 24, and the output unit 25.

  The first determination unit 21 (first determination unit) and the second determination unit 22 (second determination unit) extract a white line candidate (division line candidate) from the front image captured by the in-vehicle camera 10, and extract the extracted white line candidate Determine the white line likelihood. The first determination unit 21 and the second determination unit 22 determine the white line likelihood of the white line candidate based on the degree to which the white line candidate has a white line feature amount.

  The first determination unit 21 determines that a white line candidate corresponding to a white line that appears stably is likely to be a white line in order to suppress erroneous recognition due to disturbance or the like. Therefore, as shown in FIG. 3 and FIG. 4, when the white line on the right side of the lane is covered with snow, when there is suddenly no snow and a clear white line appears, a clear white line appears. A time lag occurs until the first determination unit 21 determines that a white line candidate corresponding to a clear white line is likely to be a white line.

  However, when a clear white line is visible on the road surface, the possibility of misrecognizing the white line is low. Therefore, it is desirable to immediately recognize the clear white line and use it for controlling the driving support device 30. Therefore, in the present embodiment, in addition to the first determination unit 21, the second determination unit 22 determines the white line likelihood of the white line candidate. When the road surface condition suddenly recovers and a clear white line suddenly appears, the second determination unit 22 immediately determines that a white line candidate corresponding to the white line that appears appears to be a white line. The clear white line means that the contrast between the road surface and the white line is high, the direction of the left and right edges of the white line is the same direction, and the direction of the left and right edges is consistent between the front side and the back side of the traveling direction. It is a white line that is not blurred or dirty with exhaust gas.

  Specifically, the second determination unit 22 determines the likelihood of a white line mainly based on the feature amount of the white line related to clarity. The white line feature amount used when the second determination unit 22 determines the likelihood of a white line includes the contrast to the road surface (the luminance ratio of the white line to the road surface brightness), the amount of change in contrast, and the direction of the left and right edges and At least one of consistency. In addition, white line candidates corresponding to clear white lines are consistent in color, thickness, and luminance on the near side and the far side in the traveling direction. In addition, white line candidates corresponding to clear white lines have the same number of edge points included in the left and right edges, and it is difficult to change line types such as solid lines and broken lines, and variations in shapes such as straight lines and curves are small. Therefore, the second determination unit 22 further includes color uniformity, thickness consistency, consistency of thickness changes, consistency of brightness, consistency of brightness changes, The uniformity of the number of edge points included in the edge, the line type of the white line, the change of the line type, the shape, and the change of the shape may be used.

  On the other hand, the first determination unit 21 uses at least one of the lane width consistency, the lane width variation consistency, and the continuous distance (the length of one white line candidate) as the white line feature amount. The white line likelihood of the white line candidate is determined. Since the three feature values are feature values related to the continuity of the white lines, only the first determination unit 21 uses the three feature values to determine the white line likelihood, and the second determination unit 22 uses the three feature values. Do not use the amount to determine the likelihood of a white line. The first determination unit 21 further uses at least one of the white line feature amounts used by the second determination unit 22.

  Here, the second determination unit 22 is stricter than the first determination unit 21 in instantaneous determination conditions when determining the white line likelihood of a white line candidate, that is, determination conditions other than the duration. For example, when the feature amount of the white line is contrast, the second determination unit 22 determines that the white line candidate is likely to be a white line for the contrast when the contrast exceeds a threshold value of 170%. On the other hand, the first determination unit 21 determines that the contrast seems to be a white line when the contrast exceeds the threshold value 110%. Further, when the feature amount of the white line is the consistency of the edge direction, the second determination unit 22 determines that the consistency of the edge direction seems to be a white line when the variation in the edge direction is less than Va. On the other hand, when the variation in the edge direction is less than Vb (Vb> Va), the first determination unit 21 determines that the consistency of the edge direction seems to be a white line.

  Then, the second determination unit 22 determines the white line likelihood of the white line candidate for the plurality of feature amounts, and determines that the white line candidate is likely to be a white line when it is determined that the predetermined number or more feature amounts are white lines. Similarly, the first determination unit 21 also determines that a white line candidate is likely to be a white line when it is determined that a predetermined number or more of feature quantities are likely to be white lines.

  Alternatively, for example, when the feature amount of the white line is the contrast, the second determination unit 22 calculates the white line likelihood (white line likelihood) when the contrast is 170% as X%, and calculates the white line likelihood higher as the contrast increases. . The white line likelihood may be calculated as a continuous value or a stepwise value. On the other hand, the first determination unit 21 calculates the white line likelihood when the contrast is 110% as X%, and calculates the white line likelihood higher as the contrast increases. Further, when the feature amount of the white line is the consistency of the edge direction, the second determination unit 22 sets the white line likelihood when the edge direction variation is Va as X%, and the white line becomes smaller as the edge direction variation becomes smaller. The likelihood is calculated high. On the other hand, the first determination unit 21 sets the white line likelihood when the edge direction variation is Vb to X%, and calculates the white line likelihood higher as the edge direction variation becomes smaller.

  That is, the second determination unit 22 calculates a white line likelihood lower than that of the first determination unit 21 for white line candidates having a predetermined amount of feature. Then, each of the first determination unit 21 and the second determination unit 22 integrates the likelihood of the white line calculated for each feature amount, and determines that the white line candidate is likely to be a white line when the integrated white line probability exceeds a threshold value. .

  Furthermore, the second determination unit 22 determines the likelihood of a white line in a shorter time than the first determination unit 21. Specifically, the first determination unit 21 includes a learning unit 21a (learning unit) that learns the feature amount of the white line including the lane width based on the images of a plurality of frames. And the 1st determination part 21 determines the white line likelihood of a white line candidate based on the feature-value of the white line learned by the learning part 21a. On the other hand, the second determination unit 22 determines the white line likelihood of a white line candidate based on an instantaneous image or a feature amount of a white line calculated from an image having a smaller number of frames than the first determination unit 21. Further, the learning unit 21a increases the learning speed of the white line feature amount as the white line likelihood determined by the second determination unit 22 increases.

  When the first determination unit 21 or the second determination unit 22 determines that the white line candidate is likely to be a white line, the recognition unit 24 (recognition unit) recognizes the white line candidate as a white line. Specifically, the recognition unit 24 calculates a white line parameter of a white line candidate determined to be a white line by an extended Kalman filter using a predetermined white line model. The white line parameters are lane curvature, lane offset position, lane width, yaw angle, pitch angle, and the like.

  The road surface determination unit 23 (road surface determination means) determines the complexity of the road surface state of the road that prevents the recognition of the white line from the images of a plurality of frames. As shown in FIG. 5, on the road surface where there is no disturbance factor and a clear white line is seen, there is a low possibility that the white line is erroneously recognized. Further, as shown in FIG. 6, there is a high possibility that a white line is erroneously recognized on a road surface on which snow is piled up or on a road surface where light is reflected due to rain as shown in FIG. Therefore, the road surface determination unit 23 determines that the state of the road surface roughened by snow or wet with rain is messy.

  Specifically, the road surface determination unit 23 compares the feature amount representing the road surface randomness with a threshold value to determine whether the road surface is random. For example, on the road surface where the white line is hidden due to snow, the frequency of repeating the white line becomes high. On the road surface that is wet and shining due to rain, the variation of the white line parameter becomes large. The road surface determination unit 23 determines the magnitude of temporal variation of the white line parameter calculated by the recognition unit 24, the frequency of the variation, the frequency of resuming white line recognition and repetition of lost, the way in which edge points are arranged, and the contrast change of white line candidates with respect to the road surface. And the threshold value are compared with each other to determine whether or not the road surface state is messy. The road surface determination unit 23 calculates an average value of each feature amount in the images of a plurality of frames, and determines the messiness using the calculated average value of each feature amount.

  Further, the road surface determination unit 23 may determine the disorder of the road surface state using information acquired by the sensor group 11. The sensor group 11 includes a GPS receiver and a storage device in which map information is stored, a wheel speed sensor, a gyro sensor, a road-vehicle communication device, and the like. For example, the road surface determination unit 23 acquires attributes of a road such as an expressway, a general road, and a mountain road from the map information of the current position, and determines that the road surface is messy. Moreover, the road surface determination part 23 determines with it being messy, when the shaking and side slip of the vehicle 50 are detected from the detection value of a wheel speed sensor and a gyro sensor. Moreover, the road surface determination part 23 determines with it being messy, when the information of snow accumulation or rainfall is acquired from the road-vehicle communication apparatus.

  When the road surface determination unit 23 determines that the road surface state is not disordered and the first determination unit 21 determines that the white line candidate is likely to be a white line, the output unit 25 (output unit) recognizes the recognition result by the recognition unit 24, That is, the white line parameter is output to the driving support device 30. The output unit 25 stops or holds the output of the white line parameter when the road surface determination unit 23 determines that the road surface state is messy.

  However, the output unit 25 outputs the white line parameters only when the second determination unit 22 determines that the white line candidate is likely to be a white line, even if the road surface determination unit 23 determines that the road surface state is messy. . In this case, the output unit 25 outputs white line parameters regardless of the white line likelihood determination result by the first determination unit 21 on the condition that the second determination unit 22 determines that the white line candidate is likely to be a white line.

  Next, a processing procedure for outputting white line parameters will be described with reference to the flowchart of FIG. This processing procedure is executed by the ECU 20 every time an image of one frame is taken by the in-vehicle camera 10.

  First, the information of the image image | photographed with the vehicle-mounted camera 10 is acquired (S10). Subsequently, white line likelihood determination 2 is performed based on the image information acquired in S10 (S20). That is, the second determination unit 22 determines the white line likelihood of the white line candidate. Details of white line likelihood determination 2 will be described later. Subsequently, white line likelihood determination 1 is performed based on the image information acquired in S10 and the image information of a plurality of frames acquired in the past (S30). That is, the first determination unit 21 determines the white line likelihood of the white line candidate. Details of white line likelihood determination 1 will be described later.

  Subsequently, based on the image information acquired in S10, the image information of a plurality of frames acquired in the past, and the information acquired by the sensor group 11, the disorder of the road surface state is determined (S40).

  Subsequently, when it is determined in S20 or S30 that the white line candidate is likely to be a white line, the white line candidate is recognized as a white line, and white line parameters are calculated (S50).

  Subsequently, when it is determined in S40 that the road surface state is not messy and the white line candidate is determined to be a white line in S30, or the white line candidate is determined to be a white line in S20, the white line parameter calculated in S50 is output. To do. This process is complete | finished above.

  Next, the second white line likelihood determination by the second determination unit 22 will be described with reference to the subroutine of FIG. Here, the feature amount of the white line is assumed to be color uniformity, contrast of the white line with respect to the road surface, consistency of the edge direction, uniformity of the number of edge points included in the left and right edges, and consistency of the thickness.

  First, a color is calculated from the acquired image information using a known calculation formula (S21). Subsequently, a luminance value is calculated from the acquired image information using a known calculation formula (S22).

  Subsequently, using the luminance value calculated in S22, the contrast of the white line candidate with respect to the road surface, that is, the luminance ratio of the white line candidate with respect to the road surface is calculated (S23). Subsequently, the left and right edges of the white line candidate are calculated using the luminance value calculated in S22 (S24). Specifically, the image is scanned in the horizontal direction, edge points whose luminance changes greatly are extracted, and the left and right edges are calculated by applying Hough transform or the like to the extracted edge points.

  Subsequently, the number of edge points included in each of the left and right edges of the white line candidate calculated in S24 is calculated (S25). Subsequently, the thickness of the white line candidate is calculated from the interval between the left and right edges calculated in S24 (S26).

  Next, based on the strength of contrast, the uniformity of the number of left and right edge points, the uniformity of the color of the white line candidate from the front side to the back side of the white line candidate, the consistency of the edge direction, and the consistency of the thickness Then, the white line likelihood of the white line candidate is calculated (S27). The white line likelihood may be calculated as a probability of a continuous value, or may be calculated in two stages, i.e., white line-like or not white line-like. Further, the white line likelihood may be calculated step by step in three or more steps. Thereafter, the process returns to S30.

  Next, the first white line likelihood determination by the first determination unit 21 will be described with reference to the subroutine of FIG. Here, the feature amount of the white line is the contrast of the white line to the road surface, the consistency of the lane width, and the continuous distance.

  First, the contrast is learned using the contrast calculated in S23 (S31). That is, a predetermined weighting is performed on the contrast calculated in S23 and the stored learning value of contrast, the learning value of contrast is updated, and the updated learning value of contrast is stored. At this time, the higher the likelihood of the white line calculated in S27, the greater the contrast weight calculated in S23 and the faster the learning speed.

  Subsequently, the lane width is calculated from the distance between the edge of the right white line candidate of the vehicle 50 calculated in S24 and the edge of the left white line candidate, and the lane width is learned (S32). That is, predetermined weighting is performed on the calculated lane width and the stored learned lane width value, the learned lane width value is updated, and the updated learned lane width value is stored. At this time, the higher the white line likelihood calculated in S27, the larger the lane width weight calculated in S24, and the faster the learning speed.

  Subsequently, the continuous distance of the white line candidate is calculated from the edge length of the white line candidate calculated in S24 (S33).

  Subsequently, the white line likelihood of the white line candidate is calculated based on the contrast learned value, the lane width learned value, and the continuous distance (S34). Similar to white line likelihood determination 2, the white line likelihood may be calculated as a probability of a continuous value or may be calculated in stages. Thereafter, the process returns to S40.

  According to 1st Embodiment described above, there exist the following effects.

  When the road surface determination unit 23 determines that the road surface state is not messy and the first determination unit 21 determines that the white line candidate is likely to be a white line, a recognition result in which the white line candidate is recognized as a white line is output. Therefore, when the road surface state is not in a state where it is easy to erroneously recognize the white line, the recognition result of the white line can be used for the control of the driving support device 30. If the road surface determination unit 23 determines that the road surface state is messy, the white line recognition result is not output. Therefore, when the road surface state is a state in which the white line is easily misrecognized, it is possible to suppress the recognition result having the possibility of misrecognition being used for the control of the driving support device 30. Furthermore, only when the second determination unit 22 determines that the white line candidate is likely to be a white line, the white line recognition result is output even if the road surface state is determined to be messy. Therefore, when a clear white line suddenly appears from a messy road surface state, it is possible to quickly shift from a state where the white line recognition result is not output to a state where the white line recognition result is output. As a result, the state in which the recognition result of the white line is output can be maintained for a longer time.

  -When the white line is faded, the color of the road surface is mixed with the white line, the contrast is lowered, and the amount of change in contrast is increased. Further, when the white line is blurred, the edge of the white line is directed in a direction other than the traveling direction, and the consistency is lowered. Therefore, a clear white line can be appropriately determined by using the contrast between the road surface and the white line, the amount of contrast change, and the edge direction and the consistency thereof as the characteristic amount of the white line related to the clarity.

  By using the lane width, the amount of change in the lane width, and the continuous distance as the feature amount of the white line, it is possible to determine that a white line candidate that appears stably and stably appears to be a white line. As a result, erroneous recognition of white lines can be suppressed.

  ・ Clear white lines are consistent in color, thickness, brightness, line type, and shape from the front side to the back side in the direction of travel, so color, thickness, brightness, line type, shape, etc. By using it as a quantity, a clear white line can be determined appropriately.

  The second determination unit 22 can instantaneously determine that a white line candidate is likely to be a white line even when a clear white line suddenly appears. As a result, the state in which the recognition result of the white line is output can be maintained for a longer time. On the other hand, the first determination unit 21 can determine the white line-likeness of the white line candidate with high stability and suppress white line misrecognition.

  -The higher the white line-likeness determined by the second determination unit 22, the faster the learning speed of the white line feature amount used by the first determination unit 21 for determination. That is, when a clear white line is seen, the possibility of misrecognizing the white line is low. Therefore, the learning speed of the white line feature amount is increased, and the responsiveness of the white line-likeness determination by the first determination unit 21 is increased. Can be improved.

(Modification of the first embodiment)
In the first embodiment, only when the second determination unit 22 determines that the white line candidate is likely to be a white line, the white line parameter is output even if the road surface determination unit 23 determines that the road surface state is messy. I did it. Instead, only when the second determination unit 22 determines that the white line candidate is likely to be a white line, the state where the road surface determination unit 23 determines that the road surface state is random is changed to the state where it is determined that the road surface state is not random. You may make it easy to make a transition. Specifically, on the condition that the second determination unit 22 determines that the white line candidate is likely to be a white line, the road surface determination unit 23 reduces the number of frames used for the determination of randomness, and from the image with a small number of frames, Judging. Further, the threshold value for determining the randomness may be tightened on the condition that the second determination unit 22 determines that the white line candidate is likely to be a white line.

  When the second determination unit 22 determines that the white line candidate is likely to be a white line and the road surface determination unit 23 determines that the road surface state is not messy, the output unit 25 performs the first determination unit 21. Regardless of the determination result of white line likelihood, the white line parameter is output.

  Even in this case, when a clear white line suddenly appears from a rough road surface state, it is possible to quickly shift from a state in which no white line parameter is output to a state in which the white line parameter is output. As a result, the state in which the white line parameter is output can be maintained for a longer time.

(Second Embodiment)
Next, a difference between the travel lane marking recognition apparatus according to the second embodiment and the travel lane marking recognition apparatus according to the first embodiment will be described with reference to FIG. The ECU 20 according to the second embodiment includes a function of the abnormality determination unit 26 instead of the function of the road surface determination unit 23.

  The abnormality determination unit 26 (abnormality determination unit) determines that the recognition result is abnormal based on the comparison between the recognition result of the white line by the recognition unit 24 and the abnormality threshold value. Specifically, the abnormality determination unit 26 compares each white line parameter calculated by the recognition unit 24 and the temporal variation amount of each white line parameter with the corresponding abnormal threshold value, and exceeds the abnormal threshold value. If there is something, it is determined that the recognition result is abnormal.

  Furthermore, the abnormality determination unit 26 changes each abnormality threshold according to the white line-likeness of the white line candidate determined by the second determination unit 22. Specifically, the abnormality determination unit 26 changes the abnormality threshold so that the white line candidate of the white line candidate determined by the second determination unit 22 is more likely to be determined to be abnormal as the recognition result is abnormal. That is, the abnormality determination unit 26 makes the abnormality threshold stricter as the white line likelihood of the white line candidate determined by the second determination unit 22 is higher.

  When the first determination unit 21 determines that the white line candidate is likely to be a white line, and the abnormality determination unit 26 determines that the recognition result is not abnormal, the output unit 25 outputs a white line parameter that is a recognition result. The output unit 25 stops or holds the output of the white line parameter when the abnormality determination unit 26 determines that the recognition result is abnormal.

  Further, the output unit 25 determines that the recognition result by the abnormality determination unit 26 is not abnormal regardless of the determination result by the first determination unit 21 on the condition that the second determination unit 22 determines that the white line candidate is likely to be a white line. If it is determined, the white line parameter is output. That is, the output unit 25 outputs a white line parameter when the first determination unit 21 or the second determination unit 22 determines that the white line candidate is likely to be a white line and the abnormality determination unit 26 determines that the recognition result is not abnormal. To do.

  According to the second embodiment described above, the higher the white line likelihood of the white line candidate determined by the second determination unit 22, the less likely the abnormality determination unit 26 determines that the white line recognition result is abnormal. That is, when a clear white line is seen, there is a low possibility that the white line is erroneously recognized. Therefore, the abnormality threshold is tightened so that the output of the white line recognition result is hardly stopped or suspended. Therefore, when a clear white line is seen, the output of the white line recognition result can be maintained.

(Third embodiment)
Next, a difference between the traveling lane marking recognition apparatus according to the third embodiment and the traveling lane marking recognition apparatus according to the second embodiment will be described with reference to FIG. The ECU 20 according to the third embodiment further includes a prediction unit 27.

  The prediction unit 27 (prediction means) is abnormal when the abnormality determination unit 26 determines that the white line parameter is abnormal, and when the white line suddenly disappears and the recognition unit 24 no longer calculates the white line parameter. The white line parameter is predicted based on the white line parameter when it is determined by the determination unit 26 that there is no abnormality. For example, the prediction unit 27 holds the white line parameter calculated at the previous processing timing and sets it as the predicted parameter. Alternatively, the prediction unit 27 predicts the next white line parameter from the white line parameter calculated or predicted at the previous and previous processing timings.

  The output unit 25 outputs the white line parameter calculated by the recognition unit 24 when the first determination unit 21 determines that the white line candidate is likely to be a white line and the abnormality determination unit 26 determines that the white line candidate is not abnormal. Further, the output unit 25 determines that the white line parameter is not abnormal by the abnormality determination unit 26 regardless of the determination result by the first determination unit 21 on the condition that the second determination unit 22 determines that the white line candidate is likely to be a white line. When the determination is made, the white line parameter calculated by the recognition unit 24 is output.

  The output unit 25 continues the predetermined time and predicts by the prediction unit 27 when the white line parameter is determined to be abnormal by the abnormality determination unit 26 and when the white line parameter is no longer calculated by the recognition unit 24. Output white line parameters. Then, the output unit 25 outputs the predicted white line parameter continuously for a predetermined time, and then stops outputting the predicted white line parameter.

  Furthermore, when the white line parameter is calculated by the recognition unit 24, the output unit 25 continuously outputs the predicted white line parameter as the white line probability of the white line candidate determined by the second determination unit 22 increases. Increase time.

  According to the third embodiment described above, when the abnormality determination unit 26 determines that the white line parameter calculated by the recognition unit 24 is abnormal, and when the recognition unit 24 no longer calculates the white line parameter. The predicted white line parameter is output for a predetermined time. Then, the output of the predicted white line parameter is stopped. The predetermined time during which the predicted white line parameter is continuously output is lengthened as the white line candidate determined by the second determination unit 22 is more likely to be a white line. Therefore, when a clear white line is seen, the accuracy of the predicted white line parameter is high, so that the output time of the predicted white line parameter can be extended and the output of the white line parameter can be maintained.

(Other embodiments)
The recognition unit 24 may increase the responsiveness of the filter used for calculating the white line parameter as the white line likelihood determined by the second determination unit 22 increases. That is, the higher the clarity of the white line candidate, the higher the response of a filter such as an extended Kalman filter. Thereby, a white line parameter having high responsiveness to the actual white line shape is calculated.

  If the white line is indistinct and the filter response is high, a straight composite line may be misrecognized as a curved white line, or a straight lane may be misrecognized as a curve due to a disturbance such as coal tar or guardrail. It is easy to do. Therefore, when a clear white line is not observed, erroneous recognition of the white line can be suppressed by making the response of the filter relatively low. Further, when a clear white line is seen, the white line can be properly recognized even in a place where the state of the white line such as a sharp curve changes suddenly by making the response of the filter relatively high. . The filter used for calculating the white line parameter is not limited to the extended Kalman filter, but may be any filter that can adjust the responsiveness. For example, a state space filter such as an H∞ filter may be used.

  -The 1st determination part 21 may determine the white line likelihood of a white line candidate using the value which averaged the feature-value of the white line calculated from the image of several frames.

  The second determination unit 22 may also use a white line feature amount other than the white line feature amount related to the white line clarity if the white line feature amount related to the white line clarity is mainly used. Further, the feature amount of the white line is not limited to that described in the above embodiments.

  In the flowchart of FIG. 8, the road surface state determination may be performed before the white line likelihood determinations 1 and 2.

  10: In-vehicle camera, 20: ECU, 50: Vehicle.

Claims (8)

A lane line candidate that is a candidate for a lane marking that divides a road lane is extracted from an image captured by a camera (10) mounted on the vehicle (50), and the extracted lane line candidate is used as the lane marking. A lane marking recognition device (20) that recognizes as
First determination means for determining the likelihood of the running lane line of the lane line candidate extracted from the image;
Second determination means for determining the likelihood of the lane marking of the lane marking candidate extracted from the image;
Road surface determination means for determining from the images of a plurality of frames, the road surface state disorder of the road that prevents recognition of the travel lane markings;
When the road surface determination means determines that the road surface state is not messy and the first determination means determines that the lane line candidate is likely to be the travel lane line, the lane line candidate is used as the travel lane line. Output a recognized recognition result, and when the road surface determination means determines that the road surface state is messy, output means for stopping or holding the output of the recognition result,
The second determination means determines the likelihood of the travel lane line based on the feature amount including the feature amount of the travel lane line related to the clarity of the travel lane line, and determines the likelihood of the travel lane line. The instantaneous determination condition at the time of determination is stricter than the first determination means,
A state in which it is determined that the road surface state is not messy from the state in which the road surface state is determined to be messy on the condition that the lane marking candidate is determined to be the travel lane line by the second determination unit. A travel lane marking recognition device characterized in that, even if the road surface determination means determines that the road surface state is messy, the output means outputs the recognition result. Feature quantity of the traveling partition lines regarding the clarity, the traveling partition lines in contrast, the variation of the contrast, and according to claim 1 comprising at least one orientation and its consistency edge with respect to the road surface of the road Lane marking recognition device. Among the first determination means and the second determination means, only the first determination means uses the characteristic amount of the travel lane line when determining the likelihood of the travel lane line, the lane width consistency, consistency variation of the width of the lane, and the travel lane mark recognition apparatus according to claim 1 or 2 comprising at least one continuous length. The feature amount of the travel lane line used when the second determination unit determines the likelihood of the travel lane line is color uniformity, thickness consistency, consistency change of the thickness, brightness consistency. The consistency of the brightness change, the uniformity of the number of edge points included in the left and right edges, the consistency of the line type, the consistency of the line type change, the consistency of the shape, and the consistency of the shape change. The travel lane marking recognition device according to any one of claims 1 to 3 , comprising at least one. Said second judging means, wherein in a shorter time than the first determination means, the travel lane line recognition apparatus according to any one of determining claims 1-4 said traveling partition lines ness. The recognition result of the said travel lane line is output irrespective of the determination result by the said 1st determination means on the condition that the said lane line candidate was determined with the said 2nd determination means to be the said lane line. traveling partition line recognition system according to any one of 5. The first determination unit includes a learning unit that learns a feature amount of the travel lane line including the width of the lane based on the images of a plurality of frames, and the feature amount learned by the learning unit Based on the travel lane marking,
It said learning means, the higher is said traveling partition lines likelihood determination by the second determination means, the travel division line recognition apparatus according to any one of claims 1 to 6 to increase the learning speed of the feature quantity. Recognizing means for recognizing the lane line candidate as the travel lane line by calculating a parameter representing the state of the travel lane line from the lane line candidate;
The travel lane line according to any one of claims 1 to 7 , wherein the recognition unit increases the responsiveness of a filter used for calculating the parameter as the likelihood of the travel lane line determined by the second determination unit is higher. Recognition device.