JP5957315B2 - In-vehicle lane recognition system - Google Patents

Description

  The present invention relates to an apparatus that is mounted on a vehicle and recognizes a lane in which the vehicle is traveling.

  Conventionally, the lane in which the vehicle is traveling is recognized by detecting the white line (lane line) drawn on the road surface to distinguish the lane from the image that is mounted on the vehicle and captured by the camera. An apparatus has been proposed. With respect to such an apparatus, an edge point corresponding to a white line is extracted from the photographed image, and the white line is a solid line or a broken line depending on whether or not the time-series variance value of the number of measurements of the edge point is equal to or less than a threshold value. A technique for determining whether or not there is known (Patent Document 1).

JP 2010-244382 A

  Even if the white line is a solid line, the number of edge points to be extracted is not constant, for example, when the white line is partially thin in some places or when the white line is interrupted. In such a case, the technique described in Patent Document 1 may erroneously determine that the white line is a broken line even though the white line is a solid line. In addition, for example, when it is difficult to extract an edge point because the captured image is generally dark and the contrast is low, noise due to a slight change in contrast on the road surface may be extracted as the edge point. In such a case, with the technique described in Patent Document 1, a broken white line that does not actually exist may be erroneously detected. As described above, the conventional technology cannot correctly detect the lane markings drawn with broken lines on the road surface.

An in-vehicle lane recognition device according to the present invention includes a photographing device that photographs a road surface around a vehicle and obtains a photographed image, and a plurality of photographed images acquired in time series by the photographing device, and the vehicle travels on the road surface. A feature point extraction unit that extracts a plurality of feature points corresponding to lane markings drawn on at least one of the left and right sides of the lane, and a periodicity of the number of feature points extracted from each captured image by the feature point extraction unit A periodicity analysis unit that determines whether or not the change in the number of feature points is periodic, and the lane when the change in the number of feature points is determined to be periodic by the periodicity analysis unit A lane line detection unit that detects a lane line as a broken line , and the periodicity analysis unit, based on a frequency waveform that represents a change in the number of feature points on the detected lane line on the frequency axis, Analyze the periodicity of the number of .

  According to the present invention, it is possible to correctly detect a lane marking drawn with a broken line on the road surface.

It is a block diagram which shows the structure of the vehicle-mounted lane recognition apparatus by one Embodiment of this invention. It is a figure which shows the imaging | photography area | region of a camera. It is a figure which shows the example of the attachment position of a camera. It is a control block diagram regarding the classification judgment of a lane line. It is a flowchart of the process performed by the classification judgment of a lane line. It is a figure which shows the example which extracts a feature point from a picked-up image. It is explanatory drawing of the method of judging the presence or absence of periodicity from the frequency waveform of the number of feature points. It is explanatory drawing when there exists a false detection factor of a white line immediately after the broken line interrupted.

  FIG. 1 is a block diagram showing a configuration of an in-vehicle lane recognition device 100 according to an embodiment of the present invention. An in-vehicle lane recognition device 100 shown in FIG. 1 is used by being mounted on a vehicle, and includes an attached camera 1, a control unit 2, an alarm output unit 3, and an operation state notification unit 4. Prepare.

  The camera 1 is installed toward the rear of the vehicle, and images an image in an imaging region including a road surface behind the vehicle at predetermined time intervals. For this camera 1, for example, an image sensor such as a CCD or a CMOS is used. A captured image acquired by the camera 1 is output from the camera 1 to the control unit 2.

  FIG. 2 is a diagram showing a shooting area of the camera 1 and shows the camera 1 viewed from the lateral direction. The camera 1 captures an image including a road surface behind the vehicle in this imaging region. Here, the imaging region (angle of view) of the camera 1 is set relatively wide so that the road surface behind the vehicle can be imaged in a sufficiently wide range in the left-right direction.

  FIG. 3 is a diagram illustrating an example of the attachment position of the camera 1. A number plate 21 is installed on the vehicle body 20 at the rear portion of the host vehicle. The camera 1 is attached at a position directly above the number plate 21 so as to face obliquely downward. In addition, since the attachment position shown here is an example to the last, you may attach the camera 1 to another position. As long as the road surface behind the vehicle can be photographed within an appropriate range, the attachment position of the camera 1 may be determined in any manner.

  The control unit 2 performs predetermined image processing using the captured image from the camera 1 and performs various controls according to the processing result. By the control performed by the control unit 2, for example, a function called LDW (Lane Departure Warning) is realized in the in-vehicle lane recognition device 100. The LDW is a function that outputs a warning when the host vehicle is likely to deviate from the traveling lane by detecting a white line (lane marking) on the road surface from the captured image.

  The alarm output unit 3 is a part for outputting an alarm by an alarm lamp, an alarm buzzer or the like to the driver of the vehicle. The operation of the alarm output unit 3 is controlled by the control unit 2. For example, when it is determined in the aforementioned LDW that the host vehicle is about to depart from the lane in which the vehicle is traveling, or when a vehicle that may collide with the host vehicle is detected in the BSW, according to the control of the control unit 2 An alarm is output from the alarm output unit 3.

  The operation state notification unit 4 is a part for notifying the driver of the vehicle of the operation state of the in-vehicle lane recognition device 100. For example, when the in-vehicle lane recognition device 100 is in a non-operating state without satisfying a predetermined operating condition, a lamp installed near the driver's seat of the vehicle as the operating state notifying unit 4 under the control of the control unit 2 Lights up. This notifies the driver that the in-vehicle lane recognition device 100 is in a non-operating state.

  Next, the lane line classification determination performed in the in-vehicle lane recognition device 100 will be described. The on-vehicle lane recognition device 100 determines whether the lane line is a broken line or a solid line when detecting the white line (lane line) on the road surface from the captured image in the LDW described above. Perform the appropriate control. For example, if it is determined that the lane marking is a broken line, the detection of the lane marking between the broken lines is suppressed, or even when the vehicle is likely to deviate from the traveling lane between the broken lines, a warning is surely issued. To be able to output. This makes it possible to stably use the LDW even when a lane marking using a broken line that is difficult to detect compared to a solid line is drawn on the road surface.

  The white line (lane line) used in the above determination is a marking line drawn on the road surface to divide the lane, and corresponds to a road lane boundary line, a road center line, a road outer line, and the like. This is not limited to white, but may be drawn in other colors such as yellow. Moreover, what installed the circular or other shape road fence (Bottsdots) on the road at predetermined intervals may be used as a lane marking. In the following description, these are collectively referred to as lane markings.

  FIG. 4 is a control block diagram of the control unit 2 regarding the type determination of the lane line. The control unit 2 relates to the type determination of the lane line, each of the feature point extraction unit 201, the lane line detection unit 202, the periodicity analysis unit 203, the reliability calculation unit 204, the lane departure detection unit 205, and the alarm control unit 206. Has a control block. In the control unit 2, for example, each control block of FIG. 4 is realized by executing a program recorded in the memory corresponding to each of these control blocks with a microcomputer.

  The feature point extraction unit 201 extracts feature points corresponding to lane markings drawn on the left and right of the traveling lane of the host vehicle from a plurality of photographed images acquired in time series by the camera 1. The lane line detection unit 202 detects a straight line in which many feature points extracted from each captured image by the feature point extraction unit 201 are arranged as lane line candidates, and the lane line is either a broken line or a solid line. Determine whether. Specifically, the final line is selected from the straight lines in which the feature points are arranged on the line based on the parallelism of the left and right lane markings, the time-series lateral position change, the number of feature points, and the like. The periodicity analysis unit 203 analyzes the periodicity of the number of feature points arranged on the finally selected line. The reliability calculation unit 204 determines the number of feature points extracted by the feature point extraction unit 201 and the periodicity analyzed by the periodicity analysis unit 203 with respect to the detection result of the lane line by the lane line detection unit 202. The reliability based on the presence or absence of is calculated.

  The result passed to the periodicity analysis is preferably data after removing the noise factor.For example, if the lens is contaminated and white line feature points always exist in the same place, this After removing the known noise components, select the white line candidate, remove the noise component on the white line candidate, check the number of feature points, and apply this to the periodic analysis. Stabilize.

  The lane departure detection unit 205 recognizes the traveling lane of the host vehicle based on the lane marking detected by the lane marking detection unit 202, and determines whether the host vehicle is likely to deviate from the traveling lane. Thus, the deviation running state of the host vehicle is detected. When the departure running state of the host vehicle is detected, an alarm output instruction is issued to the alarm control unit 206. The warning control unit 206 outputs a warning output signal to the warning output unit 3 in response to an instruction from the lane departure detection unit 205. In response to the output of the alarm output signal, an alarm is output from the alarm output unit 3 to the driver. LDW is realized in the in-vehicle lane recognition device 100 by the operations of the lane departure detection unit 205 and the alarm control unit 206 as described above.

  The warning control unit 206 determines whether or not to output a warning output signal based on the reliability calculated by the reliability calculation unit 204 when a warning output instruction is given from the lane departure detection unit 205. . If it is determined that the alarm output signal should not be output, the alarm output signal is not output to the alarm output unit 3. As a result, even if a warning output instruction is issued from the lane departure detection unit 205, a warning output signal is not output from the warning control unit 206 to the warning output unit 3. Thereby, when the reliability with respect to the detection result of a lane marking is not enough, the warning output by the warning output part 3 is suppressed.

  FIG. 5 is a flowchart of processing executed in the above-described lane line classification determination. The processing shown in this flowchart is performed by the control unit 2 during execution of the LDW.

  In step S <b> 10, the control unit 2 uses the camera 1 to take an image of a predetermined imaging area including a road surface around the host vehicle, and obtain a captured image. This captured image is output from the camera 1 to the control unit 2 and used in the subsequent processing.

  In step S20, the control unit 2 extracts feature points corresponding to the lane markings from the captured image acquired in step S10. Here, the feature point extraction unit 201 sets an extraction region at a predetermined portion in the photographed image, and extracts an edge point whose luminance change is equal to or greater than a predetermined threshold in the extraction region as a feature point.

  FIG. 6 is a diagram illustrating an example of extracting feature points from a captured image. The captured image 30 shown in FIG. 6 is divided into a road surface image area 32 where a road surface is captured and a background image area 33. In step S20, as shown in FIG. 6, the extracted image 34 corresponding to the inside of the lane marking on the right side of the host vehicle and the inside of the lane marking on the left side of the host vehicle are corresponded to the captured image 30. An extraction area 35 is set. Then, edge points are detected by comparing the brightness of adjacent pixels in these extraction regions 34 and 35 and extracted as feature points 36. Thereby, a plurality of feature points 36 are extracted along the inner contour lines of the left and right lane markings.

  In step S30, the control part 2 detects a lane line based on the feature point extracted by step S20 by the lane line detection part 202. FIG. For example, a plurality of straight lines are set on the left and right according to the arrangement state of the feature points extracted from the left and right extraction regions 34 and 35, respectively, and one straight line that passes through the most feature points is left and right. Identify one by one. By recognizing the pair of left and right straight lines thus identified as lane markings, it is possible to detect lane markings on the left and right of the traveling lane of the host vehicle. In addition, you may detect a lane marking from a feature point by methods other than this.

  By executing the processing of step S20 and step S30 described above, the lane markings drawn on the left and right of the traveling lane of the host vehicle can be detected from the captured image acquired by the camera 1 respectively. . When the lane marking is drawn only on either the left side or the right side of the traveling lane, no feature point is extracted for the side on which the lane marking is not drawn, and therefore no lane marking is detected.

  In step S40, the control unit 2 uses the feature point data extracted in step S20 to update the feature point history data recorded in a memory (not shown) or the like. In the history data of the number of feature points updated here, data of the number of feature points of each captured image acquired in time series by the camera 1 is recorded for each of the left and right lane markings. Thus, each time feature points are extracted from the captured image, the data of the number of feature points is newly recorded, and the history data of the number of feature points is updated.

  In step S50, the control unit 2 determines whether or not feature points have been extracted from a predetermined number or more of captured images. When feature points are extracted from a predetermined number of photographed images, that is, the number of photographed images acquired after the start of the process of FIG. 5 is equal to or greater than a predetermined number, and the number of feature points extracted from each of the photographed images. If history data is recorded, the process proceeds to step S60. On the other hand, if feature points have not been extracted from a predetermined number or more of photographed images, that is, if the number of photographed images acquired after starting the processing of FIG. 5 is less than the predetermined number, the process returns to step S10. In this case, the processes of steps S10 to S40 described above are repeated until feature points are extracted from a predetermined number or more of the captured images. Thereby, a plurality of feature points corresponding to the lane markings are respectively extracted from a plurality of captured images acquired in time series by the camera 1, and history data of the number of feature points is recorded.

  In step S <b> 60, the control unit 2 causes the periodicity analysis unit 203 to perform fast Fourier transform on the history data of the number of feature points recorded so far. With this fast Fourier transform, the change in the number of feature points represented on the time axis as time-series data of the number of feature points for each of the left and right lane markings is converted into a frequency waveform represented on the frequency axis. be able to. The fast Fourier transform can be executed by processing using a known algorithm or the like.

  In step S70, the control unit 2 uses the periodicity analysis unit 203 to perform the following fast Fourier transform in step S60, and based on the frequency waveform of the number of feature points converted from the original time series data, the period of the feature points Analyze sex.

Maximum number of past data retention frames for white line feature points on a straight line: N
Past data of the number of white line feature points on a straight line: x (n) (n = 0,1,2, ..., N-1)
Discrete Fourier transform: X (k) (k = 0,1,2, ..., N-1)

  Here, the periodicity is analyzed for the change in the number of feature points extracted from the captured image by determining the presence or absence of periodicity from the frequency waveform of the number of feature points using the method described below.

  FIG. 7 is an explanatory diagram of a method for determining the presence or absence of periodicity from the frequency waveform of the number of feature points. The periodicity analysis unit 203 can determine the presence or absence of periodicity with respect to changes in the number of feature points using three types of determination methods as shown in FIGS. 7A, 7B and 7C, respectively.

  FIG. 7A is an explanatory diagram of a method for determining the presence or absence of periodicity by comparing two peak frequencies in the frequency waveform of the number of feature points. For example, as shown in FIG. 7A, it is assumed that there are a peak P1 centered on the peak frequency f1 and a peak P2 centered on the peak frequency f2 in the frequency waveform of the number of feature points. In this case, the periodicity analysis unit 203 can determine the presence or absence of periodicity by comparing a value obtained by doubling the peak frequency f1 with the peak frequency f2. That is, when the value obtained by doubling the peak frequency f1 and the peak frequency f2 substantially coincide with each other, the peak P2 corresponds to the harmonic component of the peak P1, and it is determined that they exhibit the same periodicity. it can. Therefore, when such a condition is satisfied, it can be determined that the change in the number of feature points is periodic.

  FIG. 7B is an explanatory diagram of a method for determining the presence or absence of periodicity by comparing the peak frequency in the frequency waveform of the number of feature points with the past peak frequency. For example, as shown in FIG. 7B, it is assumed that there is a peak P1 centered at the peak frequency f1 in the frequency waveform of the number of feature points. In this case, the periodicity analysis unit 203 can determine the presence or absence of periodicity by comparing the current peak frequency f1 with the average value f1a of past peak frequencies stored in a memory (not shown) or the like. it can. That is, when the difference between the average value f1a of the peak frequencies obtained when analyzing the periodicity of the number of feature points in the process of step S70 executed in the past and the current peak frequency f1 is within a predetermined range, It can be determined that a peak corresponding to the peak P1 has also existed in the past. Therefore, when such a condition is satisfied, it can be determined that the change in the number of feature points is periodic.

  FIG. 7C is an explanatory diagram of a method for determining the presence or absence of periodicity based on whether or not the peak frequency in the frequency waveform of the number of feature points is within an allowable range. For example, as shown in FIG. 7C, it is assumed that there is a peak P1 centered at the peak frequency f1 in the frequency waveform of the number of feature points. In this case, the periodicity analysis unit 203 can determine the presence or absence of periodicity based on whether or not the peak frequency f1 is within an allowable range Fr determined according to the traveling speed of the host vehicle. That is, the allowable range Fr is calculated based on the traveling speed of the host vehicle and the distance between the broken lines of the lane markings drawn on the road surface at regular intervals. If the peak frequency f1 is within the allowable range Fr, the peak P1 It can be determined that represents the periodicity of the broken line of the lane line. Therefore, when such a condition is satisfied, it can be determined that the change in the number of feature points is periodic. For example, the allowable range Fr can be calculated using the traveling speed of the host vehicle and the distance between the broken lines of the lane markings defined by rules such as the Road Traffic Law.

  In step S70, the periodicity of the number of feature points can be analyzed for the left and right lane markings by the method described above, and it can be determined whether the change in the number of feature points is periodic. Note that any one of the above three types of determination methods may be used alone, or a plurality of methods may be used in any combination.

  In step S80, the control unit 2 makes a determination based on the analysis result in step S70. That is, if it is determined in step S70 that the change in the number of feature points is periodic, an affirmative determination is made in step S80 and the process proceeds to step S100. If it is determined that there is no periodicity, a negative determination is made in step S80. Proceed to S90. This determination is preferably performed separately for the left and right lane markings.

  In step S90, the control unit 2 determines whether or not the lane marking detected in step S30 satisfies the predetermined solid line condition by the periodicity analysis unit 203. For example, when the feature points extracted in step S20 are continuously arranged in a line in the captured image, or when the history data of the number of feature points recorded so far is stable at a predetermined value or more, the lane section It is determined that the line is a solid line, and the process proceeds to step S110. On the other hand, when these solid line conditions are not satisfied, it is determined that the lane marking is not a solid line. In this case, the process returns to step S10, and acquisition of the captured image is continued.

  In step S100, the control unit 2 determines that the lane line detected in step S30 is a broken line by the lane line detection unit 202.

  In step S110, the control unit 2 determines that the lane line detected in step S30 is a solid line by the lane line detection unit 202.

  The lane line detection unit 202 executes the process of step S30 described above, and then executes the process of step S100 or S110 based on the result of analysis of the number of feature points by the periodicity analysis unit 203, so that the traveling lane of the host vehicle The left and right lane markings can be detected as solid lines or broken lines, respectively. When the lane marking is a botsdot, if the installation interval is constant, it is determined that there is periodicity in step S80, and then step S100 is executed to determine a broken line. If step S100 or S110 is performed, the control part 2 will progress to step S120.

  In step S120, the control unit 2 determines whether to execute reset. This determination is made based on whether or not a predetermined reset condition is satisfied. For example, when the host vehicle suddenly stops, suddenly starts, suddenly steers, etc., it can be determined that the reset is executed assuming that the reset condition is satisfied. If it is determined that resetting is to be executed, the process proceeds to step S130. Otherwise, the process proceeds to step S140.

  In step S130, the control unit 2 executes the reset by clearing the history data of the number of feature points recorded so far. If step S130 is performed, it will return to step S10 and the acquisition of a picked-up image will be performed again from the beginning.

  In step S140, the control unit 2 uses the reliability calculation unit 204 to calculate the reliability for the detection result of the lane markings performed in step S100 or S110. Here, the reliability with respect to the detection result of the lane marking is calculated based on the number of feature points extracted in step S20. For example, by multiplying the number of feature points extracted for the left and right lane markings by a predetermined conversion coefficient, the reliability corresponding to the number of feature points is set to 0% to 100 for the left and right lane markings. % Can be calculated respectively. In addition, when the lane marking is detected as a solid line in step S110, the reliability may always be 100%.

  In addition, in spite of being determined to have periodicity in S80, the number of feature points is equal to or greater than a predetermined value for a predetermined period or more, or from the past feature points stored when it is determined to have periodicity. If feature points that are equal to or greater than the average of the feature points are not extracted, it is determined that the broken line is interrupted, and the false alarm is suppressed by resetting the reliability to zero.

  If the white line has been detected stably for a long time, it is considered that there is a certain level of reliability, and even if a situation where the white line cannot be seen somewhat continues, the recognition status can be easily reset and no alarm is generated. difficult. If the recognition state is forcibly reset or a warning is not given, a warning is likely to occur immediately if the lane mark is difficult to see, such as when the broken line is faint, or when the lane mark is difficult to see. For this reason, even when the broken line is blurred or when the recognition is interrupted at the time of bottling or darkness, it is difficult to make a false alarm in an instant. However, if the periodic white line that had been stable until now is suddenly interrupted, the periodic broken line is suddenly interrupted even though the white line was periodically visible on a clean road. It can be considered that there is a high possibility that the broken line disappeared. It is very rare that a clean road suddenly changes to a dirty road, and if the condition that the feature point does not enter in a certain period is satisfied, the reliability is lowered or the false alarm is suppressed by resetting to 0 . As shown in FIG. 8, this is effective in suppressing false alarms when there is a white line false detection factor such as road surface paint immediately after a broken line breaks off at a toll booth, during construction, or at a branch. . If road surface paint, arrows, etc. enter the processing area immediately after the periodic broken line breaks, there is a risk of false detection. In addition, since a periodic broken line has been observed so far, a false alarm will result if an erroneous detection is performed while maintaining a certain degree of reliability. By using periodic analysis, it is determined that the road surface condition is good, and the fact that the white line feature point suddenly disappears despite the fact that it is good determines that the broken line is broken, By resetting, false alarms in front of the toll gate, branching and joining are suppressed.

  In addition, when a lane line is detected as a broken line in step S100, it is preferable that the reliability calculation part 204 suppresses the fall of the reliability calculated between the broken lines of the lane line. For example, based on the periodicity analysis result of step S70, the timing at which the broken line breaks in the imaging region is predicted, and the reliability calculated for the captured image acquired at this timing is multiplied by a predetermined correction coefficient. Therefore, the reliability is improved. Alternatively, when a lower limit value of reliability is determined in advance and the reliability calculated between the broken lines is less than this lower limit value, the reliability may be set as the lower limit value regardless of the calculation result. In addition to this, any method may be used as long as it is possible to suppress a decrease in reliability between the lane markings between the broken lines.

  In step S150, the control unit 2 performs lane departure determination by the lane departure detection unit 205. Here, based on the positions of the left and right lane markings detected in step S30 in the captured image, the positional relationship between the host vehicle and the travel lane is recognized, and the host vehicle deviates from the travel lane from the recognition result. It is determined whether or not it is likely. For example, the distance from the camera 1 to the lane line is obtained based on the position of the lane line in the captured image, and this distance is converted into the distance from the front wheel of the host vehicle to the lane line. If the distance from the front wheel thus determined to the lane line is less than a predetermined value, it is determined that the host vehicle is about to depart from the traveling lane, and this state is detected as a deviated traveling state.

  In step S160, the control unit 2 determines whether or not a departure traveling state is detected by the lane departure determination in step S150. If the departure running state is detected, the process proceeds to step S170. If not detected, the process returns to step S10 to continue acquiring the captured image.

  In step S170, the control unit 2 determines whether or not the reliability calculated in step S140 is greater than or equal to a predetermined value by the alarm control unit 206. If the reliability is equal to or higher than the predetermined value, it is determined that an alarm output signal should be output, and the process proceeds to step S180. On the other hand, if the reliability is less than the predetermined value, it is determined that the alarm output signal should not be output, and the process returns to step S10 to continue acquiring the captured image.

  In step S180, the control unit 2 instructs the alarm output unit 3 to output an alarm using the alarm control unit 206. At this time, the alarm control unit 206 issues a warning output instruction by outputting a predetermined warning output signal to the warning output unit 3. A warning is output from the warning output unit 3 in response to the warning output signal, thereby notifying the driver that the host vehicle is in a deviating running state that is likely to deviate from the driving lane. If step S180 is performed, it will return to step S10 and acquisition of a picked-up image will be continued.

  In the flowchart of FIG. 5 described above, when the lane marking is determined to be a broken line in step S100 and the process returns to step S10, when extracting feature points from the captured image in step S20, It is preferable to suppress the extraction between the broken lines of the lane markings. For example, based on the analysis result of the previous step S70, a range where the broken line is interrupted in the captured image is estimated, and the extraction region is not set for the range, or the processing region width itself is set narrow. By setting the processing region so as not to allow a large change in the horizontal position, erroneous detection that greatly shifts the horizontal position is suppressed. Even if a false detection is made, the next cycle is set to return to the broken line side when a broken line appears.

  Alternatively, while the broken line is interrupted, the threshold value of the luminance change used for detecting the edge point is increased. In addition to this, any method may be used as long as feature point extraction can be suppressed. By doing in this way, it can prevent beforehand that a feature point is extracted accidentally between the broken lines of a lane line, although a lane line does not exist.

  Also, when detecting the lane markings, using the period of the broken line, predicting the length of the line that will be captured in the current frame, and extracting the target straight line from the candidate lines, False detection is suppressed at intervals where there is no broken line, and it is easy to select a lane stably while the broken line is visible. When the type of line such as the boundary between the broken line and the solid line changes, a solid line is selected immediately because a line with an overwhelming number of feature points rises as a candidate line at the moment when the solid line enters.

  According to the embodiment described above, the following operational effects can be obtained.

(1) The in-vehicle lane recognition device 100 has at least one of the left and right sides of the traveling lane of the vehicle on the road surface around the vehicle, from a plurality of captured images acquired by the feature point extraction unit 201 in time series by the camera 1. A plurality of feature points corresponding to the lane markings drawn on one side are respectively extracted (step S20). Then, the periodicity analysis unit 203 analyzes the periodicity of the number of feature points extracted from each captured image, and determines whether the change in the number of feature points is periodic (step S70). As a result, when it is determined that the change in the number of feature points has periodicity, the lane line detection unit 202 detects the lane line as a broken line (steps S30 and S100). Since it did in this way, the lane line drawn with the broken line on the road surface can be detected correctly.

(2) In step S70, the periodicity analysis unit 203 analyzes the periodicity of the number of feature points based on the frequency waveform representing the change in the number of feature points on the detected lane marking on the frequency axis. . Since it did in this way, it can be judged correctly whether the change of the number of feature points has periodicity.

(3) As shown in FIG. 7A, the periodicity analysis unit 203, when the value obtained by doubling the peak frequency f1 in the frequency waveform and the peak frequency f2 in the frequency waveform substantially match, It may be determined that the change in number is periodic. Thereby, it is possible to easily determine whether or not the change in the number of feature points has periodicity by using the harmonic component of the frequency waveform.

(4) As shown in FIG. 7B, when the difference between the current peak frequency f1 in the frequency waveform and the average value f1a of the past peak frequencies is within a predetermined range, the number of feature points It may be determined that the change has periodicity. Thereby, it is possible to easily determine whether or not the change in the number of feature points has periodicity by using the average value of the past peak frequencies.

(5) Further, as shown in FIG. 7C, when the peak frequency f1 in the frequency waveform is within the allowable range Fr according to the traveling speed of the vehicle, the change in the number of feature points is periodic. You may judge. Thereby, it is possible to easily determine whether or not the change in the number of feature points has periodicity by using the traveling speed of the vehicle.

(6) In step S20, the feature point extraction unit 201 determines the luminance difference between the feature points with respect to the lane line between the broken lines when the lane line is detected as a broken line by the lane line detection unit 202 in the previous process. By increasing the threshold value, it is possible to suppress erroneous extraction of feature points. In this way, it is possible to prevent the feature points from being erroneously extracted between the broken lines of the lane markings in spite of the absence of the lane markings, and to prevent false alarms.

(7) The in-vehicle lane recognition device 100 calculates the reliability of the detection result of the lane line based on the number of feature points extracted in step S20 by the reliability calculation unit 204 (step S140). Further, the lane departure detection unit 205 detects the departure traveling state when the vehicle is likely to depart from the traveling lane (step S150). As a result, when the reliability calculated in step S140 is equal to or greater than a predetermined value and a deviation running state is detected in step S150, an alarm is output from the alarm output unit 3 (step S180). Furthermore, when calculating the reliability in step S140, if the lane marking is detected as a broken line in step S100, a decrease in reliability between the broken lines of the lane marking can be suppressed. If it does in this way, it can prevent that a warning will not be output by the fall of reliability between the broken lines of a lane line. Therefore, it is possible to reliably output an alarm when the vehicle is likely to deviate from the traveling lane while suppressing an erroneous alarm.

(8) In step S20, the feature point extraction unit 201 performs the process of extracting feature points between the lane markings when the lane marking is detected by the lane marking detection unit 202 in the previous process. By reducing the area, it is possible to suppress erroneous extraction of feature points. In this way, it is possible to prevent the feature points from being erroneously extracted between the broken lines of the lane markings in spite of the absence of the lane markings, and to prevent false alarms.

(9) In step S20, the feature point extraction unit 201 determines the length of the lane line based on the periodicity when the lane line is detected as a broken line by the lane line detection unit 202 in the previous process. It is possible to preferentially detect a lane line that is estimated and close to the estimated length. In this way, it is possible to prevent the feature points from being erroneously extracted between the broken lines of the lane markings in spite of the absence of the lane markings, and to prevent false alarms.

(10) When it is determined that there is periodicity in S80, it is determined that the broken line is interrupted when a predetermined number or more of feature points are not extracted in the feature point extraction unit 201 for a predetermined period or more, By resetting or reducing the reliability to 0, a false alarm can be suppressed by determining the break of the broken line before the toll gate, branching, merging, etc. and controlling the reliability.

  In the embodiment described above, the camera 1 photographs the road surface behind the vehicle, but the road surface ahead of the vehicle may be photographed. As long as the road surface around the vehicle can be photographed, the photographing range of the camera 1 may be set in any way.

  Moreover, in the said embodiment, you may set the time limit with respect to the detection of a lane marking based on the periodicity of the number of feature points calculated | required by step S70. For example, the limit time can be set to twice the period in which the number of feature points changes. At this time, when the feature point is not extracted and the state where the lane line cannot be detected continues beyond the set time limit, for example, a reset by the process of step S130 is executed to clear the history data of the number of feature points. It is preferable to suppress alarm output by the alarm output unit 3.

  The embodiment and various modifications described above are merely examples, and the present invention is not limited to these contents as long as the features of the invention are not impaired.

DESCRIPTION OF SYMBOLS 1 Camera 2 Control part 3 Alarm output part 4 Operation | movement state alerting | reporting part 100 Vehicle-mounted lane recognition apparatus 201 Feature point extraction part 202 Lane lane marking detection part 203 Periodicity analysis part 204 Reliability calculation part 205 Lane departure detection part 206 Alarm control part

Claims (9)

A photographing device for photographing a road surface around the vehicle and acquiring a photographed image;
A plurality of feature points corresponding to lane markings drawn on at least one of the left and right sides of the traveling lane of the vehicle on the road surface are extracted from a plurality of captured images acquired in time series by the imaging device. A feature point extraction unit;
Analyzing the periodicity of the number of feature points extracted from each captured image by the feature point extraction unit, and determining whether the change in the number of feature points has periodicity; and
A lane marking detection unit that detects the lane marking as a broken line when the periodicity analysis unit determines that the change in the number of feature points is periodic ;
The periodicity analysis unit analyzes the periodicity of the number of feature points based on a frequency waveform representing a change in the number of feature points on the detected lane marking on the frequency axis. In-vehicle lane recognition device. The in-vehicle lane recognition device according to claim 1 ,
The periodicity analyzing unit periodically changes the number of feature points when a value obtained by doubling the first peak frequency in the frequency waveform and a second peak frequency in the frequency waveform substantially coincide with each other. An in-vehicle lane recognition device characterized by determining that there is a vehicle. The in-vehicle lane recognition device according to claim 1 ,
The periodicity analysis unit determines that the change in the number of feature points is periodic when the difference between the current peak frequency in the frequency waveform and the average value of the past peak frequencies is within a predetermined range. An in-vehicle lane recognition device characterized by: The in-vehicle lane recognition device according to claim 1 ,
The periodicity analysis unit determines that the change in the number of feature points is periodic when a peak frequency in the frequency waveform is within an allowable range corresponding to a traveling speed of the vehicle. In-vehicle lane recognition device. The in-vehicle lane recognition device according to any one of claims 1 to 4 ,
The feature point extraction unit increases a threshold value of a luminance difference for extraction of the feature point with respect to a portion between the lane markings when the lane marking is detected as a broken line by the lane marking detection unit. In-vehicle lane recognition device. The in-vehicle lane recognition device according to any one of claims 1 to 4 ,
A reliability calculation unit that calculates a reliability for the detection result of the lane marking based on the number of feature points extracted by the feature point extraction unit;
A lane departure detection unit for detecting a departure traveling state when the vehicle is likely to depart from the traveling lane;
An alarm output unit that outputs an alarm when the reliability is equal to or greater than a predetermined value and the lane departure detection unit is detected by the lane departure detection unit;
The reliability calculation unit suppresses a decrease in the reliability of the lane line between the broken lines when the lane line is detected as a broken line by the lane line detection unit. Lane recognition device. The in-vehicle lane recognition device according to any one of claims 1 to 4 ,
The feature point extraction unit reduces a processing area for extracting the feature point between the broken lines of the lane line when the lane line is detected as a broken line by the lane line detection unit. A vehicle lane recognition device. The in-vehicle lane recognition device according to any one of claims 1 to 4 ,
The feature point extraction unit estimates a length of a lane line based on periodicity when the lane line is detected as a broken line by the lane line detection unit, and a lane line close to the estimated length An in-vehicle lane recognition device that detects a line preferentially. The in-vehicle lane recognition device according to claim 6 ,
When the lane line is detected as a broken line by the lane line detection unit, and when a predetermined number or more of feature points are not extracted for a predetermined period or more from the feature point extraction unit, a broken line is displayed. An in-vehicle lane recognition device characterized by determining that there is a break and resetting or reducing the reliability.