JP5172422B2 - Driving support system - Google Patents

Description

  The present invention relates to a driving support system, and more particularly to a driving support system that temporarily stops driving support control when an imaging environment is deteriorated.

  In recent years, image processing is performed on an image captured by an imaging unit such as a camera mounted on the host vehicle, a preceding vehicle traveling in front of the host vehicle is detected, and the preceding traveling control for the preceding vehicle is performed based on the detected preceding vehicle. ACC devices that perform cruise control with vehicle following function (Adaptive Cruise Control, hereinafter abbreviated as ACC) have been developed. In addition, a lane tracking control device having a lane tracking control function that detects the left and right lanes of the host vehicle by image processing of the captured image and controls the host vehicle to follow the lane is developed. ing. In the present invention, a continuous line or a broken line marked on a road surface such as a road center line such as an overtaking prohibition line, a vehicle traffic zone boundary line, a lane line dividing a roadside zone and a roadway is referred to as a lane.

  In the ACC device, when the preceding vehicle does not exist, the host vehicle travels at a set speed at a set speed, and when the preceding vehicle exists, the accelerator throttle or brake of the own vehicle follows the preceding vehicle. Control for automatically operating the mechanism and the like is performed automatically. Therefore, the information on the preceding vehicle used in the ACC device is required to be detected in a situation where the preceding vehicle can be accurately detected.

  In the lane tracking control device, control for automatically performing the steering operation of the host vehicle is performed so that the host vehicle travels following the lane without departing from the left and right lanes. Therefore, the lane information used in the lane tracking control device is required to be detected in a situation where the lane can be accurately detected.

By the way, for example, under the backlight imaging environment as shown in FIG. 24, flare caused by strong light from the sun Su (flare) is also included in the image T _O captured by the imaging means. Smear (smear, also referred to as overexposure) Sm may occur. When image processing such as stereo matching processing is performed on the image T _O or the like captured in such an imaging environment, the preceding vehicle Vah and illustration are omitted due to the influence of the flare Fl and smear Sm extremely brighter than the surrounding image area. The lane to be detected may not be detected.

In addition, for example, a backlight is incident on only one camera, such as a reference image T _O or a comparative image T _C captured by a stereo camera illustrated in FIGS. In some cases, the images T _O and T _C are picked up in a state where the brightness balance is lost. Even in such an imaging environment, there may be cases where it is impossible to detect the preceding vehicle Vah or a lane that is not shown.

  Then, if the ACC device continues to be operated, for example, when the preceding vehicle Vah cannot be detected (ie lost), the host vehicle approaches the preceding vehicle Vah and makes a rear-end collision. Become. Further, if the lane tracking control is continued in a state where the lane cannot be detected, for example, the host vehicle deviates from the lane without permission.

In order to avoid the occurrence of such a situation, in the invention described in Patent Document 1, the overall luminance (for example, the average of luminance values) of each image region corresponding to the position of the reference image T _O and the comparative image T _C is determined. It has been proposed to make a fail determination when they are significantly different from each other. This corresponds to the case where the balance of the brightness of the reference image T _O and the comparison image T _C is lost as shown in FIGS. 25 (A) and 25 (B).

Further, in the invention described in Patent Document 2, there is a possibility that a flare Fl, a smear Sm, or the like may be erroneously recognized when an object is not stably imaged for several frames when the exposure amount of the camera is large. It has been proposed to eliminate as. In addition, the luminance value of all the pixels of the captured image or the luminance value of each pixel in the monitoring area set in the image is monitored, and there is a pixel in which the luminance value is saturated or close to it. In some cases, an apparatus for determining the occurrence of smear has been developed (see, for example, Patent Documents 3 to 6).
JP 2002-374524 A JP 2007-96684 A JP 2000-207563 A JP 2001-28745 A JP 2001-43377 A JP 2002-22439 A

  By the way, in the inventions described in these patent documents, when the imaging environment deteriorates due to the influence of flare Fl or smear Sm or the balance of the brightness of a pair of images, the image recognition performance does not deteriorate. Even in a state where the preceding vehicle Vah and the lane can be detected sufficiently accurately, the driving support control such as the preceding vehicle following control and the lane following control is automatically paused.

  However, even if the imaging environment deteriorates, the degree of deterioration is not serious, and if the preceding vehicle Vah or lane can be detected sufficiently accurately without degrading the image recognition performance, the information is valid. It is preferable to continue driving support control until the image recognition performance reaches a limit and the preceding vehicle Vah or lane cannot be accurately detected.

  The present invention has been made in view of such circumstances, and is capable of accurately pausing the driving support control when the image recognition performance reaches the limit depending on the degree of deterioration of the imaging environment. The purpose is to provide a support system.

In order to solve the above problem, the first invention provides:
A detection apparatus comprising: an imaging unit that captures an image of the surroundings of the host vehicle; and a detection unit that detects a specific target from an image captured by the imaging unit;
A driving support device that performs driving support control on the host vehicle based on the information on the specific target detected by the detection unit;
In a driving support system comprising:
Counting means for counting the number of valid data used for detection of the specific object by the detection means;
An imaging environment determination unit that determines whether the imaging environment is deteriorated based on an image captured by the imaging unit, and evaluates the degree of deterioration of the imaging environment in three stages;
When the imaging environment determination means evaluates that the degree of deterioration of the imaging environment is the worst stage, or when the degree of deterioration of the imaging environment is evaluated as an intermediate stage, and Control means for temporarily stopping the driving support control for the host vehicle by the driving support device when the number of the valid data counted by the counting means is less than a preset threshold;
Equipped with a,
The imaging environment determination unit includes a counter that increases a count number in a frame that is determined that the imaging environment is deteriorated, and that decreases a count number in a frame that is not determined that the imaging environment is deteriorated, A first threshold value and a second threshold value set to a value larger than the first threshold value are provided for the count number of the counter, and the degree of deterioration of the imaging environment when the count number of the counter is less than the first threshold value Is evaluated as the most improved stage, and when the count number of the counter is greater than or equal to the first threshold and less than the second threshold, the degree of deterioration of the imaging environment is evaluated as an intermediate stage, When the count number of the counter is greater than or equal to the second threshold value, the degree of deterioration of the imaging environment is evaluated to be the worst stage,
When the driving support control for the host vehicle is temporarily stopped, the driving support device notifies the driver of the host vehicle that the driving support control has been stopped,
The control unit evaluates that the imaging environment determination unit evaluates that the degree of deterioration of the imaging environment is most improved, or that the degree of deterioration of the imaging environment is an intermediate stage. If the number of the valid data that is counted by and said counting means in a case where is is preset threshold or more, characterized that you unpause the driving support control for the vehicle And

The second invention is the driving support system of the first invention,
The specific object is a preceding vehicle,
The driving support device performs preceding vehicle follow-up control for controlling the host vehicle to follow the preceding vehicle based on the information on the preceding vehicle detected by the detecting unit.

According to a third aspect of the present invention, in the driving support system of the second aspect , the threshold value relating to the number of valid data is set so as to become smaller as the distance from the host vehicle to the preceding vehicle becomes longer. Features.

A fourth invention is the driving support system according to any one of the first to third inventions,
The specific target is the left and right lanes of the host vehicle,
The driving support device performs lane tracking control for controlling the host vehicle to follow the lane based on the lane information detected by the detection unit.

A fifth invention is the driving support system according to any one of the first to fourth inventions,
The imaging means is a stereo camera that images the environment around the host vehicle with a pair of cameras and outputs a reference image and a comparison image,
The detection unit divides the reference image into pixel blocks having a predetermined number of pixels, performs a stereo matching process with the comparison image for each pixel block, and calculates a parallax calculated for each pixel block of the reference image or Based on distance images formed by associating distances with each other, a specific target is detected from the images captured by the imaging unit,
The imaging environment determining unit is configured to detect a distance of the reference image to which the pixel block to which a disparity equal to or greater than a distant disparity threshold corresponding to the distant disparity threshold corresponding to the distant disparity threshold is calculated as the disparity including the distant distance threshold including infinity as the distance. A frame in which the luminance value of each pixel in the pixel column extending in the vertical direction is searched, and the number of pixels having a luminance value equal to or higher than a predetermined luminance value is greater than or equal to a predetermined ratio of the number of pixels belonging to the pixel column On the other hand , it is determined that the imaging environment is deteriorated.

A sixth invention is the driving support system according to any one of the first to fifth inventions,
The imaging means is a stereo camera that images the environment around the host vehicle with a pair of cameras and outputs a reference image and a comparison image,
The detection means includes
The reference image or an image formed by performing predetermined preprocessing on the reference image is divided into pixel blocks having a predetermined number of pixels, and the predetermined image is divided into the comparison image or the comparison image for each pixel block. A first stereo matching unit that performs a stereo matching process with the image formed by the processing and associates the calculated first parallax with each pixel block to form a first distance image;
An image formed by subjecting the reference image to another preprocessing different from the preprocessing is divided into pixel blocks having the predetermined number of pixels, and the separate preprocessing is performed on the comparison image for each pixel block. A second stereo matching unit that performs a stereo matching process with the image formed by applying the second parallax calculated to each of the pixel blocks, and forms a second distance image.
The imaging environment determination unit compares the first parallax and the second parallax of corresponding pixel blocks of the first distance image and the second distance image, and an absolute value of the difference is equal to or greater than a predetermined threshold value. It is determined that the imaging environment is deteriorated for a frame in which a predetermined number or more of pixel blocks exist.

A seventh invention is the driving support system according to any one of the first to sixth inventions,
The imaging means is a stereo camera that images the environment around the host vehicle with a pair of cameras and outputs a reference image and a comparison image,
The detection unit divides the reference image into pixel blocks having a predetermined number of pixels, performs a stereo matching process with the comparison image for each pixel block, and calculates a parallax calculated for each pixel block of the reference image or Based on distance images formed by associating distances with each other, a specific target is detected from the images captured by the imaging unit,
The imaging environment determination unit sets a rectangular monitoring area extending in the horizontal direction of the image at the upper center of the image, calculates a luminance average value of the monitoring area, and in the monitoring area in the vertical direction of the image The luminance difference between each pixel pair is calculated, the number of pixel pairs having a luminance difference equal to or greater than a predetermined value is calculated as the number of edges, the average luminance value is greater than a predetermined first determination value, and the number of edges is It is determined that the imaging environment is deteriorated for a frame smaller than a predetermined second determination value.

  According to the first invention, whether or not the imaging environment is deteriorated based on the number of effective data used for detection of the specific target by the detection unit of the detection device and the image captured by the imaging unit. The driving support control for the host vehicle by the driving support device is temporarily stopped based on the information of the stage where the degree of deterioration of the imaging environment is determined and evaluated in three stages.

  Therefore, when the imaging environment is clearly deteriorated, it becomes possible to temporarily stop the driving support control, and even if the imaging environment is deteriorated, the degree of the deterioration is not a serious situation. In situations where the degree is intermediate between the worst and most improved stages above, the number of valid data used to detect a particular target is monitored to ensure that the number of valid data is sufficient. As long as it is obtained, the driving support control is accurately continued without being paused, and the driving support control is appropriately paused when the image recognition performance reaches the limit and the number of valid data cannot be obtained sufficiently. It becomes possible to make it.

According to the first invention, in addition to the prior SL effect of, when the driver of the subject vehicle, for example, adaptive cruise control is unaware that has been suspended, and fallacy the adaptive cruise control is activated For example, there is a possibility that a situation such as neglecting a braking operation on the host vehicle and a rear-end collision with a preceding vehicle may occur. Therefore, when the control means of the driving support device temporarily stops driving support control for the host vehicle, the driving support control is temporarily performed by, for example, making the driver speak a voice, display it on the screen, or turn on the lamp. By notifying the driver that the vehicle has been stopped, the driver's attention can be alerted, and the occurrence of the above situation can be avoided.

According to the first invention, in addition to the prior SL effects of, it becomes possible to immediately released to suspend the driving support control in situations where the imaging environment is clearly improved, the imaging environment is deteriorated However, when the degree of deterioration is not a serious situation and the degree of deterioration of the imaging environment is intermediate between the worst stage and the most improved stage, it is used to detect a specific target. The number of valid data is monitored, and while the number of valid data is not sufficient, it continues accurately without releasing the suspension of driving support control, and the image recognition performance is restored and effective. When the number of data is sufficiently obtained, the suspension of the driving support control can be accurately canceled.

According to the first invention, to increase the count in a frame it is determined that the imaging environment has deteriorated, by decreasing the number of counts in the frame imaging environment is not determined to be deteriorated, count By evaluating the degree of deterioration of the imaging environment step by step, it is possible to easily evaluate and classify into three stages: the stage where the degree of deterioration of the imaging environment is most improved, the intermediate stage, and the worst stage It becomes possible.

At the same time, the count number increases as the frequency of frames determined to have deteriorated the imaging environment increases, and the count number decreases as the frequency decreases. It is possible to accurately represent whether the degree of environmental deterioration is worsening or improved, and the degree of deterioration of the imaging environment can be clearly and graded step by step. Therefore, the effect of the previous SL is more accurately demonstrated.

According to the second invention, the specific target is the preceding vehicle, and the driving support device performs the preceding vehicle follow-up control for controlling the own vehicle to follow the preceding vehicle based on the information on the preceding vehicle. also in the support system, before Symbol onset Ming effect is properly exhibited.

According to the third invention, when the specific target is a preceding vehicle, the size of the preceding vehicle captured in the image decreases as the distance from the host vehicle to the preceding vehicle increases. The total number of data in the image area corresponding to is reduced. Therefore, it is effective in detecting the preceding vehicle by setting the threshold value of the number of effective data used for detecting the preceding vehicle to be smaller as the distance from the host vehicle to the preceding vehicle becomes longer accordingly. It is possible to accurately determine whether or not the number of data has been obtained, and the effects of the above-described inventions can be accurately exhibited.

According to the fourth aspect of the invention, the specific target is the left and right lanes of the host vehicle, and lane tracking control is performed to control the host vehicle to follow the lane based on the lane information by the driving support device. Also in the driving support system, the effects of the above-described inventions are accurately exhibited.

According to the fifth aspect , it is possible to accurately grasp whether or not a smear that gives a parallax corresponding to a far distance including infinity is generated in the image. Therefore, it is possible to accurately determine whether or not the imaging environment has deteriorated based on this, and the effects of the above-described inventions can be accurately exhibited.

According to the sixth aspect , by comparing the first parallax and the second parallax of the corresponding pixel blocks of the first distance image and the second distance image, the absolute value of the difference between them is a predetermined threshold value or more. It is possible to accurately detect that mismatching has occurred in the block. Therefore, it is possible to accurately grasp whether or not mismatching occurs frequently, and to accurately determine whether or not the imaging environment has deteriorated based on that, and the effects of the above-described inventions are accurately exhibited. The

According to the seventh aspect of the invention, by monitoring the average luminance value and the number of pixel pairs in the monitoring area provided at the upper center of the image and the number of edges, it is possible to detect poor visibility due to fog, snowstorm, etc. It becomes possible to accurately detect the deterioration of the imaging environment caused by fogging or dirt on the windshield of the vehicle. Therefore, it is possible to accurately determine whether or not the imaging environment has deteriorated due to them, and the effects of the above-described inventions are accurately exhibited.

  Embodiments of a driving support system according to the present invention will be described below with reference to the drawings.

  In the following description, the preceding vehicle follow-up control is performed in which the preceding vehicle is detected from the image captured by the imaging unit as the specific target, and the own vehicle is controlled to follow the detected preceding vehicle as the driving support control. Will be described.

[overall structure]
As shown in FIG. 1, the driving support system 1 according to the present embodiment includes a detection device 2, a preceding vehicle follow-up control device 3 that is a driving support device, and a notification device 4.

[Detection device]
As shown in FIG. 2, the detection apparatus 2 according to the present embodiment includes an imaging unit 21, a conversion unit 22, a detection unit 29, and the like. The detection apparatus 2 includes two systems of image processing means 25a and 25b.

  The configurations of the image pickup means 21, the conversion means 22, the image correction unit 23, the image data memory 24, and the first and second image processing means 25a and 25b are disclosed in Japanese Patent Application Laid-Open No. 5-1114099 previously filed by the applicant of the present application. JP-A-5-265547, JP-A-6-266828, JP-A-10-283461, JP-A-10-283477, JP-A-2006-72495, and the like. The explanation is left to those publications. A brief description is given below.

  In this embodiment, the imaging means 21 includes CCD (Charge Coupled Device) image sensors that are synchronized with each other, and are attached to the inside of the windshield, for example, at a predetermined interval in the vehicle width direction, that is, in the left-right direction. A stereo camera including a pair of main camera 21a and sub camera 21b is used.

The main camera 21a and the sub camera 21b are mounted at the same height from the road surface, and are configured to simultaneously capture the environment around the host vehicle and output information of the captured image at a predetermined sampling period. . The main camera 21a which is disposed closer to the driver outputs image data of the reference image T _O illustrated in FIG. 3, the sub-camera 21b disposed farther from the driver compares not shown and it outputs the image data of the image T _C.

The image data of the reference image T _O and the comparison image T _C output from the main camera 21a and the sub camera 21b is converted into, for example, 256 gradations for each pixel from the analog image by the A / D converters 22a and 22b, which are the conversion means 22, respectively. Each of the images is converted into a digital image having a predetermined luminance gradation such as a gray scale, and the image correction unit 23 performs image correction such as displacement and noise removal. Each image data subjected to image correction or the like is transmitted to and stored in the image data memory 24, and is transmitted to the first image processing means 25a and the second image processing means 25b.

  The first image processing means 25a includes a first stereo matching means 26a composed of an image processor or the like and a first distance data memory 27a.

In the present embodiment, the first image processing unit 25a, without being subjected to pre-processing the reference image T _O and the comparative image T _C, when performing a direct stereo matching with respect to the reference image T _O and the comparative image T _C Will be described. However, as in the case of the second image processing means 25b described later, the first image processing means 25a also applies to the two images formed by pre-processing the reference image T _O and the comparison image T _C. It is also possible to perform a stereo matching process. In that case, two images preprocessed by different preprocessing methods are input to the first image processing means 25a and the second image processing means 25b, respectively.

In the first stereo matching means 26a, as shown in FIG. 4, to set the pixel block PB _O of a predetermined number of pixels of 4 pixels such as 3 × 3 pixels or 4 ×, for example, on the reference image T _O, the pixel block PB _O For each pixel block PB _C having the same shape as the pixel block PB _O on the epipolar line EPL in the comparison image T _C corresponding to, a SAD value that is a difference in luminance pattern with the pixel block PB _O is calculated according to the following equation (1). and, so that the SAD value to identify the minimum pixel block PB _C.

Incidentally, p1s, t represents the intensity value of each pixel in the pixel block PB _O, p2s, t represents the luminance value of each pixel in the pixel block PB _C. Also, the sum is in the range of 1 ≦ s ≦ 3,1 ≦ t ≦ 3 in the case of pixel block PB _O or pixel block PB _C is set as an area of 3 × 3 pixels, for example, 4 × 4 pixels When set as a region, calculation is performed for all pixels in the range of 1 ≦ s ≦ 4 and 1 ≦ t ≦ 4.

Further, in the present embodiment, it is configured to use a SAD value as described above, the addition to, for example, the luminance value of each pixel in the pixel block PB _O p1s, t and in the pixel block PB _C It is also possible to perform a stereo matching process based on the sum of squares of the difference from the luminance value p2s, t of each pixel. Further, also be configured to perform stereo matching based as long as it can properly calculate the difference in luminance pattern of the pixel block PB _O and pixel block PB _C on a difference is calculated according to another calculation formula It is.

The first stereo matching means 26a, for each pixel block PB _O of the reference image T _O in this way, the position of the identified pixel block PB _{C on} the comparison image T _C and the reference image T _O of the pixel block PB _O The first parallax dp1 is calculated from this position. In the present embodiment, the first stereo matching means 26a associates the first parallax dp1 calculated as described above with each pixel block PB _O of the reference image T _O exemplified in FIG. 3, for example. .

Hereinafter, the reference image T _O of first parallax dp1 is associated to each pixel block PB _O of the first distance image T _Z 1. For example, the first distance image T _Z 1 in which the first parallax dp1 is formed in correspondence to each pixel block PB _O of the reference image T _O of FIG. 3 is an image as shown in FIG.

A point on the road surface directly below the center of the pair of main camera 21a and sub camera 21b is the origin, the vehicle width direction (left-right direction) of the host vehicle is the X-axis direction, and the vehicle height direction (height direction) is the Y-axis. direction, vehicle length direction (longitudinal direction) in the real space of the case of the Z-axis direction point and _{(X 1,} Y 1, _{Z 1),} on the first parallax dp1 and the first distance image T _Z 1 of the The point (i ₁ , j ₁ ) is associated one-to-one by coordinate transformation represented by the following equations (2) to (4) based on the principle of triangulation.
_{X 1 = CD / 2 + Z} 1 × PW × (i 1 -IV) ... (2)
Y ₁ = CH + Z ₁ × PW × (j ₁ −JV) (3)
Z ₁ = CD / (PW × (dp1-DP)) (4)

In the above equations, CD is the distance between the pair of cameras, PW is the viewing angle per pixel, CH is the mounting height of the pair of cameras, and IV and JV are the first distance images T _{Z at} the infinity point in front of the host vehicle. The i-coordinate and j-coordinate on 1 and DP represent the vanishing point parallax.

Thus, since the first parallax dp1 and the first distance Z ₁ is intended to be associated in a one-to-one, in the first stereo matching means 26a, the first parallax dp1 calculated according to the above (4) into a first distance Z _1, it can be configured to form a first distance image T _Z 1 assigns a first distance Z ₁ which is converted into pixel blocks PB _O of the reference image T _O.

The first stereo matching means 26a transmits the information of the first distance image T _Z1 formed as described above to the first distance data memory 27a for storage, and also transmits it to the detection means 29. .

On the other hand, the second image processing means 25b receives two images formed by performing predetermined preprocessing on the reference image T _O and the comparison image T _C by the preprocessing means 28 (see FIG. 2). It has come to be. In the present embodiment, the preprocessing means 28 performs edge detection processing on the reference image T _O and the comparison image T _{C as} predetermined preprocessing to form a reference edge image and a comparison edge image. In addition, as pre-processing performed in the pre-processing unit 28, it is possible to perform processing other than edge detection processing.

In this embodiment, the data of the luminance values p1i, j and p2i, j of each pixel of the reference image T _O and the comparison image T _{C on} which image correction such as displacement and noise removal has been performed from the image correction unit 23 is the reference image. The horizontal lines of T _O and the comparison image T _C , that is, the epipolar lines having a width of one pixel are sequentially output and input to the preprocessing means 28, respectively. The luminance values p1i, j and p2i, j represent the respective luminance values of the pixels at the coordinates (i, j) on the reference image T _O and the comparative image T _C.

The preprocessing means 28 applies a first-order differential operator (−1) to the data of the luminance values p1i, j, p2i, j of each pixel inputted for each horizontal line of the reference image T _O and the comparison image T _C. / 0 / + 1) (to be precise, an operator having a value twice as large as the first-order differential operator (−0.5 / 0 / + 0.5)) is applied to perform edge detection processing.

That is, the preprocessing means 28 performs the edge detection processing shown in the following equation (5) on the data of the luminance value p1i, j of each pixel input for each horizontal line of the reference image T _O and is calculated. that edge value p3i, to form a reference edge image TE _O to luminance values of pixels in the j coordinate (i, j).
p3i, j = -p1i-1, j + p1i + 1, j (5)

Further, the preprocessing means 28 performs the edge detection processing shown in the following equation (6) on the data of the luminance value p2i, j of each pixel inputted for each horizontal line of the comparison image T _C and is calculated. that edge value P4i, the j coordinate (i, j) so as to form a comparison edge image TE _C to luminance values of pixels in.
p4i, j = -p2i-1, j + p2i + 1, j (6)

In the scene where the reference image T _O and the comparative image T _C as shown in FIGS. 25A and 25B described above with the main camera 21a and the sub camera 21b are captured, the edge detection of the preprocessing means 28 is performed. the process, FIG. 6 (a), the reference edge image TE _O and the comparative edge image TE _C as shown in (B) is obtained.

Preprocessing means 28, the luminance value (the edge value) in this manner p3i, j, P4i, with the reference edge image TE _O consisting of pixels having j comparative edge image TE _C and were respectively formed, the second image The data is transmitted to the processing means 25b.

The second image processing means 25b is configured in the same manner as the first image processing means 25a described above, and the second stereo matching means 26b of the second image processing means 25b also performs stereo matching processing by the method shown in FIG. Is to be done. However, the subject of stereo matching is a comparative edge image TE _C with the reference edge image TE _O. The second stereo matching means 26b is in the same manner as in the first stereo matching means 26a and the second parallax dp2 calculated for each pixel block in the reference edge image TE _O and, in each pixel block of the reference edge image TE _O, The second distance image T _Z 2 is formed by associating the calculated second parallax dp2.

Since each pixel block of the reference edge image TE _O is set at the same pixel position as each pixel block PB _O on the reference image T _O in the first stereo matching means 26a, the second distance image T _Z 2 is can also be expressed as a respective second parallax dp2 calculated for each pixel block PB _O of the reference image T _O is formed in correspondence.

Similarly to the above, the origin, the X-axis direction, the Y-axis direction, and the Z-axis direction are set, and the coordinates of the point in the real space calculated from the second parallax dp2 are (X ₂ , Y ₂ , Z ₂ ). In this case, the second parallax dp2 and the point (i ₂ , j ₂ ) on the second distance image T _Z 2 are coordinate transformed by the following formulas (7) to (9) based on the principle of triangulation 1-to-1 correspondence. In the formula, CD and the like are the same as described above.
X ₂ = CD / 2 + Z ₂ × PW × (i ₂ −IV) (7)
Y ₂ = CH + Z ₂ × PW × (j ₂ −JV) (8)
Z ₂ = CD / (PW × (dp2-DP)) (9)

Furthermore, since the second parallax dp2 and second distance _{Z 2} is intended to be associated in a one-to-one, the second stereo matching means 26b, the second distance and the second parallax dp2 calculated according to the above (9) converted into Z _2, in each pixel block PB _O of the reference edge image TE _O (reference image T _O), constituting the second distance Z ₂ converted to form a second distance image T _Z 2 in association with It is also possible.

The second stereo matching means 26b transmits the information of the second distance image T _Z2 formed as described above to the second distance data memory 27b for storage, and transmits it to the detection means 29. .

  The detection means 29 (see FIG. 2) is composed of a microcomputer in which a CPU, ROM, RAM, input / output interface, etc. (not shown) are connected to the bus. The detection means 29 is connected to sensors Q such as a vehicle speed sensor, a yaw rate sensor, and a steering angle sensor for measuring the steering angle of the steering wheel. It should be noted that a device for estimating the yaw rate from the vehicle speed of the host vehicle or the like can be used instead of the yaw rate sensor.

  The detection unit 29 includes an object detection unit 29A, a preceding vehicle detection unit 29B, a counting unit 29C, and an imaging environment determination unit 29D, and further includes a memory (not shown).

In the present embodiment, the object detection means 29A detects an object imaged in the reference image T _O based on the first distance image T _Z 1, but for example, the second distance image T _Z 2 It is also possible to configure to detect an object in the reference image T _O based on the above. Further, the object detection unit 29A is configured to detect an object based on, for example, the first distance image T _Z1 and the second distance image T _Z2 and output a good detection result among the detection results. It is also possible to do.

  In the present embodiment, the object detection means 29A is configured based on a vehicle exterior monitoring device or the like described in Japanese Patent Laid-Open No. 10-283461 previously filed by the applicant of the present application. Detailed explanations are left to those publications. The configuration will be briefly described below.

For example, the object detection unit 29A reads the first distance image T _Z1 formed as shown in FIG. 5 from the first distance data memory 27a, and first, as shown in FIG. 7, the first distance image T _Z1. Are divided into strip-shaped sections Dn extending in the vertical direction with a predetermined pixel width. Then, a histogram Hn as shown in FIG. 8 is created for each section Dn, and each first parallax dp1 belonging to each section Dn is voted for the histogram Hn of the section Dn to which the parallax dp1 belongs. Then, the class value of the class having the maximum frequency Fn is set as the parallax dpn of the object in the section Dn. This is performed for all sections Dn of the first distance image T _Z 1.

Subsequently, the object detection means 29A substitutes the parallax dpn obtained for each section Dn into dp1 of the above equation (4), and the first distance Z in the real space for each section Dn corresponding to the parallax dpn. ₁ is calculated as the distance Zn of the section Dn, plotted on the real space as shown in FIG. 9, and adjacent to each other based on the distance and direction between the plotted points as shown in FIG. The points are grouped together into groups G1, G2, G3,.

  In this embodiment, the object detection means 29A linearly approximates each point belonging to each group as shown in FIG. 11, and each point in each group is approximately in the vehicle width direction of the host vehicle A, that is, the X-axis direction. A group arranged in parallel is labeled “object” O, and a group in which each point is arranged substantially parallel to the vehicle length direction of the host vehicle A, that is, the Z-axis direction is labeled “side wall” S for classification. Yes. In addition, C is labeled as a corner point at a position that can be regarded as an intersection of “object” and “side wall” of the same object.

  In the example of FIG. 11, the object detection unit 29A includes [side wall S1], [object O1], [sidewall S2], [object O2 and corner point C and side wall S3], [sidewall S4], and [object O3]. , [Object O4], [side wall S5 and corner point C and object O5], [object O6], and [side wall S6] are detected as one object. As described above, “object” and “side wall” are used as labels for convenience, but “side wall” is also detected as an object.

Further, the object detecting means 29A detects each object detected in this manner so as to be surrounded by a rectangular frame line on the reference image T _O as shown in FIG. The object detection means 29A stores information on each object detected in this way in a memory and outputs it to the outside as necessary.

  As shown in FIG. 13, the preceding vehicle detection unit 29B estimates a trajectory that the host vehicle A will travel in the future as a travel trajectory Lest based on the behavior of the host vehicle A, and the host vehicle centering on the travel trajectory Lest. An area corresponding to the vehicle width of A is calculated as the traveling path Rest of the host vehicle A.

The travel locus Lest of the host vehicle A is expressed by the following formula (10) or the following (11) based on information such as the vehicle speed V, the yaw rate γ, and the steering angle δ of the steering wheel transmitted from the sensors Q. It can be calculated based on the turning curvature Cua of the host vehicle A calculated according to the equation (12). In the following equations, Re is a turning radius, Asf is a vehicle stability factor, and Lwb is a wheelbase.
Cua = γ / V (10)
Re = (1 + Asf · V ² ) · (Lwb / δ) (11)
Cua = 1 / Re (12)

  The preceding vehicle detection means 29B detects the object closest to the own vehicle A among the objects existing on the travel path Rest of the own vehicle A as the preceding vehicle Vah traveling in front of the own vehicle A. ing. For example, in FIGS. 12 and 13, the vehicle O3 is detected as the preceding vehicle Vah.

  The preceding vehicle detection means 29B stores the distance Z of the preceding vehicle Vah detected in this way from the host vehicle and other necessary information in a memory, and transmits it to the preceding vehicle follow-up control device 3 described later. Yes.

  In the present embodiment, the preceding vehicle detection means 29B calculates the probability that the preceding vehicle detected in the previous sampling cycle and the object detected as the preceding vehicle in the current sampling cycle are the same three-dimensional object. It is designed to track the preceding vehicle while maintaining consistency. The preceding vehicle detection means 29B also detects that the detected preceding vehicle has left the front of the host vehicle and that the preceding vehicle becomes a new preceding vehicle, or that another vehicle interrupts between the host vehicle and the preceding vehicle. Thus, the replacement of the preceding vehicle due to the other vehicle becoming a new preceding vehicle can be detected.

  The counting means 29C (see FIG. 2) counts the number of valid data used for detecting the preceding vehicle Vah as a specific target in the preceding vehicle detecting means 29B of the detecting means 29.

Specifically, in this embodiment, the counting unit 29C uses the frame line Fr set on the reference image T _O so as to surround the detected preceding vehicle Vah for the object detection and the preceding vehicle detection. by applying the same pixel position of the first distance image T _Z 1 as illustrated in FIG. 14, counts the number of pixel blocks PB _O of first parallax dp1 is associated indicating a preceding vehicle Vah in border Fr It is like that.

Counting means 29C is configured to store the number of pixel blocks PB _O of first parallax dp1 is associated indicating a valid data number i.e. the preceding vehicle Vah and counted as described above in the memory, the adaptive cruise control to be described later The data is transmitted to the device 3.

The imaging environment determination unit 29D (see FIG. 2) is based on the first distance image T _Z1 and the second distance image T _Z 2 formed from the reference image T _{O and the} like captured by the imaging unit 21 in the following manner. Thus, it is determined whether or not the imaging environment has deteriorated, and the degree of deterioration of the imaging environment is further evaluated in three stages.

In the present embodiment, the imaging environment judgment unit 29D, as the criteria for determining whether the imaging environment has deteriorated, the first distance image T _Z 1 and the first parallax dp1 of the second distance image T _Z 2 second The parallax dp2 is compared to determine whether or not mismatching has occurred. If mismatching has occurred to some extent, it is determined that the imaging environment has deteriorated.

Specifically, as shown in FIG. 15, the imaging environment determination unit 29D first reads the information of the first distance image T _Z 1 from the first distance data memory 27a and the second distance from the second distance data memory 27b. reads the information of the image T _Z 2, is adapted to compare the first distance image T _Z 1 and the second distance corresponding first parallax dp1 and second parallax dp2 pixel block PB _O image T _Z 2 .

The imaging environment judgment unit 29D, when the absolute value of the difference between the first parallax dp1 pixel block PB _O in the same pixel position and the second parallax dp2 is greater than a predetermined threshold value Derutadpth, i.e.,
| Dp1-dp2 | ≧ Δdpth (13)
If satisfying, mismatching in the pixel block PB _O is adapted to determine that has occurred.

The threshold Δdpth, when converted to the above (4) and (9) and the first parallax dp1 the second parallax dp2 a first distance _{Z 1} each second distance according to equation _{Z 2,} the difference, i.e. The distance difference in the real space is set to a value that is considered to exist clearly. Since values that are considered to have a difference in distance may vary depending on the values of the first distance Z ₁ and the second distance Z ₂ themselves, in this embodiment, the values of the first distance Z ₁ and the second distance Z ₂ The threshold Δdpth is, for example, 5 m when the value is about 10 m, and 20 m when the values of the first distance Z ₁ and the second distance Z ₂ are about 100 m, the first distance Z _1, etc. Is set to increase as the value of increases.

Then, if the imaging environment judgment unit 29D is as described above, the first distance image T _Z 1 on each pixel block PB _O for mismatching the same pixel position on the second distance image T _Z 2 is generated It determines whether, mismatching continue to count the number of pixel blocks PB _O it is determined to be occurring. When the number of such pixel blocks PB _O is found greater than a predetermined number, so as to determine that the imaging environment has deteriorated.

Incidentally, mismatching determination for each pixel block PB _O described above may but be configured to perform the first distance image T _Z 1 and each entire image region of the second distance image T _Z 2, each range image it takes time to determine process belong a number of pixel block PB _O in, it is susceptible to mismatching further have front scenery sky or the preceding vehicle Vah is imaged in the upper region of each range image However, even if mismatching occurs in such a portion, detection of the preceding vehicle Vah is not so much affected.

Therefore, in this embodiment, the imaging environment judgment unit 29D, among the first distance image T _Z 1 and all pixel blocks PB _O of the second distance image T _Z 2, the first distance image preceding vehicle Vah like is detected The mismatching determination as described above is performed for the pixel block PB _O belonging to a predetermined image region (not shown) set in the vertical center portion of T _Z 1 and the second distance image T _Z 2. It has become.

  In this way, the imaging environment determination unit 29D determines that the imaging environment has deteriorated once for each frame (that is, each sampling period), or determines that the imaging environment has not deteriorated. It has become.

  In addition, the imaging environment determination unit 29D includes a counter (not shown). In the present embodiment, as illustrated in FIG. 16, in the frame where it is determined that the imaging environment has deteriorated, the count number C of the counter is increased. In a frame in which it is determined that the imaging environment has not deteriorated, the count number C of the counter is decreased, and the history of the deterioration condition of the imaging environment is monitored.

  In the present embodiment, the first threshold value Cth1 and the second threshold value Cth2 set to a larger value are set as the count number C of the counter.

Then, the imaging environment determination unit 29D is configured such that the count number C of the counter is
(1) If it is less than the first threshold Cth1, it is evaluated that the degree of deterioration of the imaging environment is the most improved stage α,
(2) When the first threshold value Cth1 is equal to or greater than the second threshold value Cth2, the degree of deterioration of the imaging environment is evaluated as an intermediate stage β;
(3) When the second threshold value Cth2 or more, it is evaluated that the degree of deterioration of the imaging environment is the worst stage γ,
The degree of deterioration of the imaging environment is evaluated in three stages.

  FIG. 16 shows the case where the first threshold value Cth1 is set to 2 with the count number C of the counter and the second threshold value Cth2 is set to 10 with the count number C. However, the first threshold value Cth1 and the second threshold value are shown. Cth2 is set to an appropriate value.

  The imaging environment determination unit 29D transmits any of the stages α to γ of the degree of deterioration of the imaging environment evaluated as described above to the preceding vehicle follow-up control device 3 described later.

[Leading vehicle tracking control device]
The preceding vehicle follow-up control device 3 (see FIG. 1), which is the driving assistance device in the present embodiment, is based on the information on the preceding vehicle Vah detected and transmitted by the detecting means 29 of the detecting device 2 as driving assistance control. The preceding vehicle follow-up control is performed to control the own vehicle so as to follow the preceding vehicle Vah.

  The preceding vehicle follow-up control device 3 includes a control unit 31 composed of a microcomputer or the like, and a response unit 32 composed of an actuator or the like that operates an accelerator throttle or a brake mechanism of the host vehicle based on an instruction from the control unit 31. And a memory (not shown).

  The control means 31 of the preceding vehicle follow-up control device 3 instructs the responding unit 32 based on the information on the distance Z of the preceding vehicle Vah from the own vehicle transmitted from the preceding vehicle detection means 29B of the detection device 2. Thus, the host vehicle is made to follow the preceding vehicle Vah. At this time, if the host vehicle and the preceding vehicle Vah are traveling, the host vehicle is made to follow the preceding vehicle Vah while maintaining the inter-vehicle distance from the preceding vehicle Vah, and when the preceding vehicle Vah stops, the preceding vehicle The host vehicle is stopped at a position away from the preceding vehicle Vah behind Vah by a predetermined distance.

  In this way, the control means 31 performs known preceding vehicle follow-up control including the above processing on the own vehicle based on the information on the preceding vehicle Vah transmitted from the preceding vehicle detection means 29B of the detection device 2 in this way. To do.

  In addition, the control unit 31 determines the number of effective data used for detecting the preceding vehicle Vah transmitted from the counting unit 29C of the detection device 2 and the degree of deterioration of the imaging environment transmitted from the imaging environment determination unit 29D. On the basis of the stages α to γ, the preceding vehicle following control for the host vehicle by the preceding vehicle following control device 3 is temporarily stopped, or the temporary stopping of the preceding vehicle following control is released.

  In the present embodiment, basically, the control unit 31 counts when the deterioration level of the imaging environment transmitted from the imaging environment determination unit 29D is at the stage γ (see FIG. 16), which is the worst stage. Regardless of the number of valid data used for detecting the preceding vehicle Vah transmitted from the means 29C, the preceding vehicle following control for the host vehicle by the preceding vehicle following control device 3 is temporarily stopped.

  In addition, when the degree of deterioration of the imaging environment transmitted from the imaging environment determination unit 29D is the most improved stage α, the control unit 31 leads the preceding vehicle transmitted from the counting unit 29C. Regardless of the number of valid data used for the detection of Vah, the temporary stop of the preceding vehicle follow-up control for the host vehicle is canceled.

  On the other hand, if the degree of deterioration of the imaging environment transmitted from the imaging environment determination unit 29D is the intermediate stage β, the control unit 31 detects the preceding vehicle Vah transmitted from the counting unit 29C. The effective number of data N used in the above is referred to.

  When the preceding vehicle follow-up control for the host vehicle is continued (that is, the vehicle is not temporarily stopped), when the degree of deterioration of the imaging environment is the intermediate stage β, the control means 31 counts If the number N of valid data transmitted from the means 29C is equal to or greater than a preset threshold value Nth, the preceding vehicle follow-up control is continued. If the number N is less than the threshold value Nth, the preceding vehicle follow-up control is performed on the host vehicle. It is designed to pause.

  When the preceding vehicle follow-up control for the host vehicle is temporarily stopped and the degree of deterioration of the imaging environment is the intermediate stage β, the control means 31 is transmitted from the counting means 29C. If the number N of valid data is less than a preset threshold value Nth, the preceding vehicle follow-up control is temporarily stopped. If the number of valid data is equal to or greater than the threshold value Nth, the preceding vehicle follow-up control for the host vehicle is canceled. It is like that.

Note that the number N of valid data used for detecting the preceding vehicle Vah is, as shown in FIG. 14, within the frame line Fr set on the reference image T _O so as to surround the preceding vehicle Vah. prior first parallax dp1 indicative of vehicle Vah is the number of pixel blocks PB _O that is associated. Moreover, border Fr which surrounds the preceding vehicle Vah, since the distance Z from the vehicle to the preceding vehicle Vah is smaller as the longer (i.e. farther), the total number of pixel blocks PB _O belonging to the frame line Fr is reduced .

  Therefore, the threshold value Nth related to the number N of valid data is set to be smaller as the distance Z from the host vehicle to the preceding vehicle Vah is longer. In the present embodiment, a table that associates the distance Z with the threshold value Nth is stored in advance in the memory, and the control unit 31 receives information on the distance Z from the preceding vehicle detection unit 29B of the detection device 2. The threshold value Nth is determined with reference to the table.

  The concept of starting and releasing the temporary stop of the preceding vehicle follow-up control in the control means 31 is conceptually expressed as shown in FIG. In the present embodiment, regarding the number N of valid data, the case where the threshold value for temporarily stopping the preceding vehicle tracking control and the threshold value for canceling the temporary stopping of the preceding vehicle tracking control are set to the same value Nth has been described. These threshold values may be set to different values.

  By the way, if the driver of the own vehicle does not notice that the preceding vehicle follow-up control is temporarily stopped, the driver misunderstands that the preceding vehicle follow-up control is operating, for example, neglects the braking operation on the own vehicle, and collides with the preceding vehicle Vah. There is a possibility that such a situation will occur. Therefore, in the present embodiment, when the preceding vehicle follow-up control for the host vehicle is temporarily stopped, the control means 31 of the preceding vehicle follow-up control device 3 utters a sound from a speaker (not shown) of the notification device 4 or does not show it, for example. The driver is informed that the preceding vehicle following control has been temporarily stopped by displaying it on a screen or turning on a predetermined lamp (not shown).

[Action]
Next, the operation of the driving support system 1 according to the present embodiment will be described.

In the detection apparatus 2, the environment around the host vehicle is imaged by the imaging means 21 including the main camera 21 a and the sub camera 21 b, and the first image is based on the output reference image T _O and the comparison image T _C. in the first stereo matching means 26a processing means 25a, the first parallax dp1 respectively, for each pixel block PB _O of the reference image T _O is calculated, the first distance image T _Z 1 is formed.

Further, the outputted reference image T _O and the comparative image T _C is pre-processing means 28 in each example edge detection processing is performed with the reference edge image TE _O and comparative edge image TE _C is formed, those reference edge based on the image TE _O and the comparative edge image TE _C, the second stereo matching means 26b of the second image processing unit 25b, a second parallax dp2, respectively for each pixel block of the reference edge image TE _O is calculated, the second A distance image T _Z 2 is formed.

Based on the first distance image T _Z1 obtained by the stereo matching process, the object detection unit 29A and the preceding vehicle detection unit 29B of the detection unit 29 detect the object and the preceding vehicle Vah from the reference image T _O. Is done. When the object detection unit 29A is configured to detect an object based on each of the first distance image T _Z1 and the second distance image T _Z 2, a good detection result is output among these detection results. The

  This object detection and preceding vehicle detection are always performed regardless of the occurrence of mismatching, and such information is constantly transmitted to the control means 31 of the preceding vehicle tracking control device 3 and other devices such as an ECU as necessary. Is done.

Counting means 29C is in a frame line Fr which fit in the first distance image T _Z 1, by counting the number N of valid data used for the detection of the preceding vehicle Vah (pixel block PB _O), adaptive cruise It transmits to the control means 31 of the control apparatus 3.

The imaging environment judgment unit 29D, the first distance image T _Z 1 and the first distance image T _Z 1 and on the second distance image T that belong to the image area of the central portion of the second distance each vertical image T _Z 2 The first parallax dp1 and the second parallax dp2 of the pixel block PB _O at the same pixel position on _Z 2 are compared, and the absolute value of the difference between these values is equal to or greater than the predetermined threshold Δdpth that changes according to the distance described above. if, it is determined that the mismatch in the pixel block PB _O is occurring. In this case, whether the first distance image T _Z 1 to be occurring one in mismatching of the second distance image T _Z 2 not detected.

The imaging environment determination unit 29D performs this determination process for all the pixel blocks PB _O belonging to the image areas of the first distance image T _Z1 and the second distance image T _Z 2 and mismatching occurs. determines that counts the number of pixel blocks PB _O was determined, the imaging environment if found more than a predetermined number are worse.

Thus, in this embodiment, even if mismatching occurs in the allowable number of pixel blocks PB _O in the first distance image T _Z1 and the second distance image T _Z 2, the imaging environment is immediately set. It not is determined to be deteriorated, it is determined that the imaging environment has deteriorated when the mismatch in the number of pixel blocks PB _O beyond the range permitted occurs. In other words, the number of limit mismatching is acceptable as the number of pixel blocks PB _O generated is set as the predetermined number.

  In the present embodiment, the imaging environment determination unit 29D defines the degree of deterioration of the imaging environment from the viewpoint of the frequency at which it is determined that the imaging environment has deteriorated as described above. Then, the history of the imaging environment deterioration status is monitored based on the count number C of the counter that increases and decreases for each frame shown in FIG. 16, and the frequency at which it is determined that the imaging environment is deteriorated increases. The degree of deterioration of the imaging environment is divided into three stages by increasing the degree of deterioration of the imaging environment from the most improved stage α to the intermediate stage β and the worst stage γ. evaluate.

  Conversely, the frequency at which it is determined that the imaging environment has deteriorated decreases, and as the count number C decreases, the degree of deterioration of the imaging environment is changed from the worst stage γ to the intermediate stage β, The evaluation is made by lowering to the most improved stage α. Then, the imaging environment determination unit 29D transmits, for each frame, to the control unit 31 of the preceding vehicle follow-up control device 3, which of the stages α to γ is the level of the degree of deterioration of the imaging environment that is the evaluation result.

  On the other hand, on the side of the preceding vehicle follow-up control device 3 (see FIG. 1), which is a driving support device, the control unit 31 receives information from the preceding vehicle Vah detected and transmitted by the detecting unit 29 of the detecting unit 2. In response to the instruction, the responding unit 32 operates the accelerator throttle, the brake mechanism, and the like of the host vehicle to perform preceding vehicle follow-up control for controlling the host vehicle to follow the preceding vehicle Vah.

  Further, the control means 31 of the preceding vehicle follow-up control device 3 transmits the effective number of data used for detection of the preceding vehicle Vah transmitted from the counting means 29C of the detection device 2 and the imaging environment determination means 29D. Based on the stages α to γ of the degree of deterioration of the imaging environment coming, the preceding vehicle follow-up control is paused or the pause is released according to the standard as shown in FIG.

  According to this criterion, the imaging environment in which the count number C of the counter in the imaging environment determination unit 29D of the detection apparatus 2 is accumulated greatly and the imaging environment is evaluated as the worst stage γ is clearly deteriorated. In this case, the preceding vehicle follow-up control is immediately suspended.

In this embodiment, the cause of the deterioration of the imaging environment is the influence of flare Fl and smear Sm (see FIG. 24) generated in the image, or the balance of the brightness of the pair of images is lost (FIG. 25 (A ), (B) not known by either reference) Therefore, in any of the first distance image T _Z 1 and the second distance image T _Z 2 calculated in the first stereo matching means 26a or the second stereo matching means 26b Whether or not mismatching has occurred is not detected, but in any case, it is understood that the imaging environment is such that mismatching frequently occurs in stereo matching processing.

  It can be determined that the imaging environment where the mismatching occurs frequently in a plurality of frames is a very poor imaging environment. If such mismatching occurs frequently, for example, even if the distance Z of the preceding vehicle Vah from the host vehicle is calculated, the reliability of the distance Z is poor, and the preceding vehicle following control is continued according to the calculated distance Z. The vehicle may collide with the preceding vehicle Vah, or the host vehicle may unnecessarily accelerate or decelerate behind the preceding vehicle Vah that is traveling stably.

  Therefore, when the imaging environment where the count number C of the counter in the imaging environment determination unit 29D of the detection apparatus 2 is largely accumulated and the imaging environment in which the degree of deterioration of the imaging environment is evaluated as the worst stage γ is clearly deteriorated, Immediately stopping the preceding vehicle following control results in safe driving and stable driving of the host vehicle.

  However, in that case, if the driver of the own vehicle does not notice that the preceding vehicle follow-up control has been temporarily stopped, the driver misunderstands that the preceding vehicle follow-up control is operating and neglects the braking operation on the own vehicle, for example. Since a situation such as a collision with Vah may occur, it is preferable to notify the driver that the preceding vehicle follow-up control has been temporarily stopped via the notification device 4, for example.

  Further, according to the above criteria, the count number C of the counter in the imaging environment determination unit 29D of the detection apparatus 2 is 0 or a value close thereto, and the evaluation is a stage α where the degree of deterioration of the imaging environment is most improved. In such a situation where the imaging environment is clearly improved, the suspension of the preceding vehicle following control is immediately released. This is because it is not necessary to temporarily stop the preceding vehicle follow-up control due to the deterioration of the imaging environment when the imaging environment is favorable.

  Certainly, even when the imaging environment is good, that is, when mismatching does not occur or when it is only a small scale, for example, the preceding vehicle Vah or the like may be lost without being detected. is there. However, there is no clear structure on the back of the preceding vehicle Vah, or the stereo matching cannot be performed well because the brightness of the preceding vehicle Vah and the surrounding brightness are very close. It is caused by a cause other than environmental deterioration. Therefore, at least from the viewpoint of the deterioration of the imaging environment, if the imaging environment is clearly improved, the suspension of the preceding vehicle follow-up control by that should be canceled.

  Further, according to the above criteria, when the preceding vehicle follow-up control for the host vehicle is continued (that is, the vehicle is not temporarily stopped), the imaging environment is deteriorated at an intermediate stage β If the number N of valid data transmitted from the counting means 29C is equal to or greater than the threshold value Nth, the preceding vehicle follow-up control is continued. If the number N is less than the threshold value Nth, the preceding vehicle follow-up control is temporarily stopped. .

  On the contrary, when the preceding vehicle follow-up control for the host vehicle is temporarily stopped and the degree of deterioration of the imaging environment is the stage β which is an intermediate stage, the effective transmission from the counting unit 29C is effective. If the number N of data is less than the threshold value Nth, the preceding vehicle follow-up control is temporarily stopped, but if the number N is equal to or greater than the threshold value Nth, the preceding vehicle follow-up control for the host vehicle is canceled.

  Image recognition in a state where mismatching has occurred and the imaging environment has deteriorated, but the degree of deterioration is not a serious situation, that is, in a state where the degree of deterioration of the imaging environment is an intermediate stage β If the preceding vehicle Vah can be detected sufficiently accurately without degrading the performance, the information can be used effectively, the preceding vehicle Vah can be detected, and the preceding vehicle following control can be performed effectively. However, it is better to temporarily stop the preceding vehicle following control if the image recognition performance is deteriorated and the preceding vehicle Vah cannot be accurately detected even in a state where the degree of deterioration of the imaging environment is not serious.

  Therefore, in the state where the mismatching has occurred and the imaging environment has deteriorated as described above but the degree of the deterioration is not serious (stage β), the image recognition performance is limited as in this embodiment. It is desirable to continue the preceding vehicle follow-up control without pausing until the number N of effective data used for detection of the preceding vehicle Vah in the preceding vehicle detection means 29B of the detection device 2 is not sufficiently obtained. Further, if the image recognition performance recovers in a state where the preceding vehicle following control is paused, it is desirable to cancel the temporary stopping of the preceding vehicle following control.

[effect]
As described above, according to the driving support system 1 according to the present embodiment, effective data used for detection of a specific target (preceding vehicle Vah) by the detecting unit 29 (preceding vehicle detecting unit 29B) of the detecting device 2. Whether or not the imaging environment has deteriorated is determined based on the number N and the first parallax dp1 and the second parallax dp2 obtained from the reference image T _O and the comparison image T _C captured by the imaging unit 21. Based on the information of the stages α to γ that evaluate the degree of deterioration of the imaging environment in three stages, the driving support control (preceding vehicle tracking control) for the host vehicle is temporarily performed by the driving support device (preceding vehicle tracking control device 3). It was configured to stop or release the pause.

  Therefore, when the imaging environment is clearly deteriorated (stage γ), the driving support control is temporarily stopped, and when the imaging environment is clearly improved (stage α), the driving support control is temporarily stopped. Can be released.

  At the same time, even if the imaging environment deteriorates, the degree of the deterioration is not a serious situation, and the degree of the deterioration of the imaging environment is the intermediate stage β between the worst stage γ and the most improved stage α. In a certain state, the number N of effective data used for detection of a specific object by the detection unit 29 of the detection device 2 is monitored, and the image recognition performance reaches a limit, and the number N of effective data is sufficiently obtained. The driving support control can be continued without being temporarily stopped until it is no longer possible. In addition, when the image recognition performance is recovered while the driving support control is paused, the suspension of the driving support control can be canceled.

  Thus, according to the driving support system 1 according to the present embodiment, when the image recognition performance reaches the limit according to the degree of deterioration of the imaging environment, the driving support control is appropriately paused and the driving support control is performed. In the paused state, it is possible to accurately cancel the suspension of the driving support control when the image recognition performance is recovered.

[Modification]
In the above-described embodiment, the case where the control unit 31 that temporarily stops the driving support control or releases the suspension of the driving support control is provided on the driving support device 3 side is described. For example, the control unit 31 is detected. It is also possible to configure in the computer of the detection means 29 of the device 2 so that the control means 31 transmits a signal instructing the driving support control 3 to stop or cancel the driving support control.

In the above embodiment, the imaging environment determination unit 29D is configured in the detection device 2, and it is determined whether the imaging environment has deteriorated based on the images T _O and T _C captured by the imaging unit 21. The case where the processing and the processing for evaluating the degree of deterioration of the imaging environment in three stages are described in the detection unit 29 of the detection device 2 has been described. For example, the former processing is performed in the detection device 2. When it is determined that the imaging environment is deteriorated, a signal is transmitted to the driving support device 3 side, and the signal is counted by the control means 31 of the driving support device 3, for example, and the count number C is increased or decreased. The degree of deterioration of the imaging environment can be divided into three stages and evaluated by the control means 31 or the like.

  Further, in the above-described embodiment, the specific object detected by the detection device 2 is the preceding vehicle Vah, and the preceding vehicle following control device 3 that is a driving support device performs the preceding vehicle following control on the own vehicle. However, in addition to this, for example, the present invention is also applied to a case where the specific target detected by the detection device 2 is the left and right lanes of the host vehicle and the driving support device performs lane tracking control on the host vehicle. Can do.

  In this case, the driving support system includes a lane tracking control device 3 as a driving support device instead of the preceding vehicle tracking control device 3 in FIG. 1, and the control means 31 of the lane tracking control device 3 is detected by the detection means 2. Based on the lane information, for example, the response unit 32 having a control system having a steering servo function is instructed to give steering torque to the steering wheel, and the vehicle is controlled to follow the lane. Is done.

Moreover, the detection apparatus 2 is comprised so that a lane detection means for detecting a lane may be provided. Then, the lane detection means reads out the data of the reference image T _O shown in FIG. 3 from the image data memory 24 (see FIG. 2), and uses the reference image T _O as shown in FIG. For example, a horizontal line j having a width of 1 pixel is searched from the center of the reference image T _{O in} the left-right direction, and pixels whose luminance value changes greatly from a luminance value of an adjacent pixel to a threshold value or more are set as lane candidate points cl and cr. Detect as.

Then, while shifting the horizontal line j on the reference image T _O upward by one pixel at a time, the lane candidate points cl and cr are detected on each horizontal line j in the same manner. At this time, the lane detection unit reads out information on the first parallax dp1 from the first distance data memory 27a, for example, and based on the detected first parallax dp1 at the lane candidate points cl and cr, the lane candidate points cl and cr Is not on the road surface, the lane candidate points cl and cr are excluded from the lane candidate points. Of the remaining lane candidate points cl and cr, the lane is approximated by a straight line by Hough transformation or the like based on the lane candidate points cl and cr on the side closer to the own vehicle, and detected on the left and right sides of the own vehicle, respectively.

  At this time, a plurality of lanes are detected on one side (for example, the right side) of the host vehicle while improving the reliability of detection by excluding lane candidate points cl and cr that cannot be matched with a straight line. In this case, select the lane that is consistent with the lane detected on the other side of the vehicle (for example, the left side) or the lane that is consistent with the lane detected in the previous sampling cycle. Sort straight lines.

  In this way, when the lane is detected in a straight line on the side closer to the host vehicle, the lane candidate points cl and cr are selected and connected based on the positional relationship with the straight line based on the straight line on the far side. As shown in FIG. 19, lanes Ll and Lr are detected on the left and right sides of the host vehicle, respectively. The processing configuration of the above lane detection means is described in detail as a lane recognition device in Japanese Patent Application Laid-Open No. 2006-331389 previously filed by the applicant of the present application, and refer to those publications for detailed description. I want to be.

  In the driving support system in this case, the processing in the imaging environment determination unit that determines whether the imaging environment has deteriorated and evaluates the degree of deterioration of the imaging environment in three stages is the imaging in the above embodiment. This can be performed in the same manner as the environment determination unit 29D.

  In addition, in this case, for example, the right and left used for specifying the lanes LL and LR are the targets for counting the number of valid data used for detecting the specific target (lanes LL and LR) in the counting unit. It can be defined as the number of lane candidate points cl and cr. In that case, as a threshold value set in advance with respect to the number of valid data, for example, a number that guarantees that information such as each position of the detected lanes LL and LR is reliable enough is set. If the number of data is more than that number, it can be judged that the image recognition performance has not yet reached the limit, and if the number of valid data falls below that number, the number can be judged that the image recognition performance has exceeded the limit. .

  The target of counting the number of valid data can be defined as, for example, the length in real space from the position of the own vehicle in the lanes LL and LR that can be detected. In that case, as a threshold value set in advance with respect to the number of valid data, for example, a length such as 50 m is set as the length from the position of the host vehicle, and the length of the detected lanes LL and LR is 50 m or more. If so, it is determined that the image recognition performance has not yet reached the limit, and if the detected lanes LL, LR do not reach 50 m or the like, it is determined that the image recognition performance has exceeded the limit.

On the other hand, in the above-described embodiment, the first distance image T _Z1 and the second distance image T are differently pre-processed (including the case where the pre-process is not performed) with respect to the reference image T _O and the comparison image T _C. forming a _Z 2, compared first distance image T _Z 1 and the first parallax dp1 of the second distance image T _Z 2 of the corresponding pixel block PB _O and a second parallax dp2, the absolute value of their difference there has been described a case where determining that the pixel block PB _O is above a predetermined threshold Δdpth is if there more than a predetermined number, the imaging environment is getting worse. In this case, whether the cause of the deterioration of the imaging environment is the influence of flare Fl and smear Sm (see FIG. 24) generated in the image, or the balance of the brightness of the pair of images is lost (FIG. 25 (A), It does not matter whether or not (see (B)).

  However, for example, it is also possible to simply detect that smear Sm is generated in the image and to determine that the imaging environment has deteriorated based on the smear Sm. This may be performed in place of the above-described embodiment, or may be configured to be performed in parallel with the above-described embodiment.

In the configuration for detecting the smear Sm and determining the deterioration of the imaging environment, it is sufficient if there is a reference image T _O , a comparison image T _C, and a distance image formed therefrom, as described below. Therefore, when performing in parallel with the above embodiment, for example, first distance first parallax dp1 image T _Z 1 formed by the first image processing unit 25a is configured to be used.

  In the following description, a case where a detection apparatus having only one system of image processing means is used will be described. In addition, about the apparatus, means, etc. which have the function similar to said embodiment, the same code | symbol is attached | subjected and demonstrated. In the following description, the preceding vehicle Vah is detected as a specific target by the preceding vehicle detecting means 29B of the detecting means 29 of the detecting device 2, and the preceding vehicle is compared with the own vehicle based on the information on the preceding vehicle Vah as a driving support device. The case where the preceding vehicle follow-up control device 3 that performs the preceding vehicle follow-up control for controlling the vehicle to follow Vah will be described. The present invention is also applied to the case where lane tracking control is performed to control to follow.

  The overall configuration of the driving support system in this case is the same as that of the driving support system 1 according to the above-described embodiment shown in FIG.

  As shown in FIG. 20, the detection device 2 includes an imaging unit 21, a conversion unit 22, a detection unit 29, and the like. The configuration from the imaging unit 21 to the counting unit 29C of the detection unit 29 is also shown in FIG. It is the same as that of the detection apparatus 2 which concerns on said embodiment. However, in this case, the preprocessing means 28 shown in FIG. 2 is unnecessary.

In this case, the imaging environment determination unit 29E divides each distance image T _Z (same as the first distance image T _Z 1 in the above embodiment) in the object detection unit 29A of the detection unit 29 illustrated in FIG. The voting status of the parallax dp (same as the first parallax dp1 in the above embodiment) of each pixel block PB _O belonging to Dn to the histogram Hn is monitored and corresponds to a far distance including infinity in the histogram Hn. It is configured to monitor the number of votes of the parallax dp.

This is because the smear Sm (see FIG. 24) penetrates the image T _O from the top to the bottom, and the smear Sm reflects the backlight of the sun Su. The smear Sm portion is based on the knowledge that the parallax dp corresponding to the distance from the vehicle to the sun Su, that is, the far distance including virtually infinite distance is calculated.

Further, for example, a columnar object that shines white by reflecting sunlight or the like may be imaged so as to penetrate the image T _O from the top to the bottom in the same manner as the smear Sm. since the parallax dp corresponding to infinity finite distance not far range containing corresponding to the actual position on the object is calculated, also from above in the same manner on the image T _O appeared to penetrate to the bottom, the parallax By monitoring dp, the object and the smear Sm can be clearly distinguished.

Further, the value of the parallax dp corresponding to the far distance including infinity from the vehicle, the above equation (4) by substituting infinity ∞ on the left side of the distance Z ₁ of the deformed first parallax dp1 as the parallax dp ,
∞ = CD / (PW × (dp−DP))
dp-DP = CD / (PW × ∞) = 0
That is,
dp = DP
As can be seen from this, it is the vanishing point parallax DP or a value close thereto.

Therefore, in this embodiment, a disparity parallax threshold value corresponding to a far distance including infinity is provided in the value of the parallax dp calculated by the stereo matching process, and the far parallax threshold value is set to, for example, a parallax DP corresponding to infinity. A pixel block PB _O that is set to a close value DP + 1 and the parallax dp is equal to or smaller than DP + 1 that is a disparity parallax threshold is detected as a pixel block PB _O for which the value of the disparity dp corresponding to the far distance including infinity It is supposed to be.

As described above, when the information of each distance Z in each pixel block PB _O of the reference image T _O in association to form a distance image T _Z is the distance Z defines the far distance including infinity set the far distance threshold as a threshold, the distance Z is a pixel block PB _O is more distant distance threshold, so as to detect a pixel block PB _O value of the distance Z corresponding to the far distance is calculated including infinity Configure. In this case, the distance value corresponding to the disparity DP + 1 set as the above-mentioned disparity parallax threshold is set as the far distance threshold, for example.

Specifically, the imaging environment determination unit 29E monitors the voting status in each histogram Hn where the parallax dp is voted as described above, and the value of the parallax dp in the histogram Hn is DP + 1 or less as shown in FIG. range by monitoring the number of votes of the parallax dp of, if the total value of the number of votes of the parallax dp of the range exceeds a predetermined threshold, the section Dn on distance image T _Z that corresponds to the histogram Hn Smear candidate.

  It should be noted that what value is set for the far parallax threshold and the far distance threshold, or what value is set for the predetermined threshold related to the total number of votes, is determined by the imaging means 21 of the detection device 2. Appropriate values are set as appropriate because they depend on the performance of the CCD image sensor and the like.

Subsequently, the imaging environment determination unit 29E sets a pixel row in the same region on the reference image T _O as the section Dn of the distance image T _Z that satisfies the above condition and is regarded as a smear candidate. If the section Dn on the distance image T _Z is a pixel row having a width of 4 pixels extending in the vertical direction, the pixel row Dn having a width of 4 pixels is applied to the same position of the reference image T _O as shown in FIG. Set. In FIG. 22, one pixel width is expressed larger than actual.

  Here, it is possible to determine whether smear Sm has occurred or not for the entire pixel row Dn having a width of 4 pixels. However, there may be a case where smear Sm occurs only in one column of pixel columns having a width of 1 pixel that forms a pixel column Dn having a width of 4 pixels. Therefore, the imaging environment determination unit 29E determines whether smear Sm has occurred in each of the pixel columns Dn1 to Dn4 (see FIG. 22) having a pixel width included in the pixel column Dn having a pixel width of 4 pixels.

Specifically, the imaging environment determination unit 29E searches for the luminance value p1i, j of each pixel in the pixel column for each pixel column Dn1 to Dn4 having one pixel width, and determines a luminance value equal to or greater than a predetermined luminance value. It is detected whether or not the number of high-luminance pixels is greater than or equal to a predetermined ratio of the total number of pixels belonging to the pixel row having the one-pixel width. Here, when the luminance value p1i, j of each pixel of the reference image T _O is set with a gray scale of 0 to 255, the predetermined luminance value is as high as 240 or the like close to the saturation state of the luminance value, for example. Set to a value. The predetermined ratio is set to 75%, for example.

  The imaging environment determination unit 29E determines that the smear Sm is present when the above condition is satisfied in at least one pixel column among the pixel columns Dn1 to Dn4 having one pixel width in the pixel column Dn having four pixel widths. It is determined that it has occurred.

Note that the upper portion of the reference image T _O, since the objects in the higher position than the front of the vehicle such as a vehicle is imaged, in the reference pixel block PB _O of the upper portion of the reference image T _O, smear Sm is Even when no occurrence occurs, the parallax corresponding to the far distance including infinity may be calculated as the parallax dp by the stereo matching process.

Therefore, such a reference image T _O In order to avoid adverse effects of the upper portion, of the area of each pixel row Dn1~Dn4 the reference image T _O pixel set by applying the column Dn and one pixel wide, their It is also possible to perform a search for a pixel having a high luminance value for a predetermined number of pixels (a number corresponding to, for example, 80% of the total number of pixels in the vertical direction) above the lowermost pixel in the reference image T _O of the pixel column. Is possible. In this case, it is determined that smear Sm has occurred when the number of pixels having a luminance value equal to or higher than the luminance value set to a value such as 240, for example, is 90% or more.

  When determining whether or not smear Sm has occurred, the imaging environment determination unit 29E counts a frame in which it is determined that smear Sm has occurred and the imaging environment has deteriorated, as in the case of the above-described embodiment (see FIG. 16). In a frame where it is determined that smear Sm has not occurred and the imaging environment has not deteriorated, the count number C of the counter is decreased to monitor the history of deterioration of the imaging environment. . Then, according to the count number C of the counter, the degree of deterioration of the imaging environment is further evaluated in three stages α to γ.

  If comprised in this way, the control means 31 of the preceding vehicle follow-up control apparatus 3 is effective used for the detection of the specific target (preceding vehicle Vah) by the detection means 29 (preceding vehicle detection means 29B) of the detection apparatus 2. Based on the number N of data and information on the degree of deterioration of the imaging environment (stages α to γ) transmitted from the imaging environment determination unit 29E, the driving support device (preceding vehicle following control device 3) drives the vehicle. The support control (preceding vehicle follow-up control) can be accurately temporarily stopped, or the temporary stop of the preceding vehicle follow-up control can be accurately canceled.

  Further, at the stage β in which the degree of deterioration of the imaging environment is intermediate, when the image recognition performance reaches the limit, the driving support control (preceding vehicle follow-up control) is stopped accurately and the driving support control (leading vehicle follow-up control) When the vehicle is temporarily stopped, it is possible to accurately cancel the suspension of the driving support control (preceding vehicle follow-up control) when the image recognition performance is recovered.

  Furthermore, there is disclosed a technique for detecting poor visibility due to fog, snowstorm, etc., and deterioration of the imaging environment due to fogging or dirt on the lens of the camera or the windshield of the host vehicle. It can also be configured to be applied. The technique is described in detail in, for example, Japanese Patent Application Laid-Open No. 2001-28746, which has been previously submitted by the applicant of the present application, and the detailed description is left to those publications. A brief description is given below.

  In addition, the following method applying the technology may be performed instead of the above-described embodiment or the above-described modified example in which the occurrence of smear Sm in the image is detected, or may be performed in parallel therewith. It is also possible to configure.

  The overall configuration of the driving support system in this case is the same as that of the driving support system 1 according to the above-described embodiment shown in FIG. Further, the detection device 2 can be configured in the same manner as the configuration shown in FIG.

Imaging environment determination means 29E, in this case, reads out the data of the reference image T _O from the image data memory 24, as shown in FIG. 23, extending in the horizontal direction of the reference image T _O at the top center of the reference image T _O A rectangular monitoring area R to be set is set. Then, an average value of luminance values p1i, j of the pixels belonging to the monitoring region R, that is, a luminance average value is calculated, and it is determined whether or not the luminance average value is larger than a predetermined first determination value.

This and poor visibility due to fog or snowstorm, etc., in a situation where image blur in the reference image T _O etc. imaged due to fogging and dirt of the windshield of the lens of the camera and the vehicle has occurred, usually This is because the overall brightness of the image tends to increase. Therefore, as the first determination value, a relatively high luminance value that can appropriately determine whether or not such image blur has occurred is appropriately set.

In addition, the imaging environment determination unit 29E calculates a luminance difference p1i, j of each pixel pair in the vertical direction of the reference image T _O , that is, a luminance difference in each pixel belonging to the monitoring region R, and a luminance difference equal to or greater than a predetermined value. Is calculated as the number of edges, and it is determined whether or not the number of edges is smaller than a predetermined second determination value.

This is because when the monitoring region R is set at the upper center of the reference image T _O , the preceding vehicle Vah and other various objects are usually imaged in the monitoring region R, and if the image is not blurred, it is adjacent. This is because there is a portion (edge portion) where the luminance value p1i, j of the pixel to be abruptly changed, but when the image blur occurs, the change in the luminance value p1i, j tends to be gradual. Accordingly, as the predetermined value, a relatively large value that can appropriately determine the luminance difference between when the image blur occurs and when the image blur does not occur is appropriately set.

  Further, when there is no image blur, a large number of such edge portions are found in the monitoring region R, but when the image blur occurs, the number of edge portions decreases. Therefore, as the second determination value, the number of edge portions that can appropriately determine whether or not such image blur has occurred is set as appropriate.

  The imaging environment determination unit 29E determines that the imaging environment has deteriorated when the average brightness value is larger than the first determination value and the number of edges is smaller than the second determination value. .

  The subsequent processing is the same as in the above case, and the imaging environment determination unit 29E increases the count number C of the counter in the frame determined that the imaging environment has deteriorated, and determines that the imaging environment has not deteriorated. In such a frame, the count number C of the counter is decreased, and the history of the imaging environment deterioration status is monitored. Then, according to the count number C of the counter, the degree of deterioration of the imaging environment is further evaluated in three stages α to γ.

  If comprised in this way, the control means 31 of the preceding vehicle follow-up control apparatus 3 is effective used for the detection of the specific target (preceding vehicle Vah) by the detection means 29 (preceding vehicle detection means 29B) of the detection apparatus 2. Based on the number N of data and information on the degree of deterioration of the imaging environment (stages α to γ) transmitted from the imaging environment determination unit 29E, the driving support device (preceding vehicle following control device 3) drives the vehicle. The support control (preceding vehicle follow-up control) can be accurately temporarily stopped, or the temporary stop of the preceding vehicle follow-up control can be accurately canceled.

  Further, at the stage β in which the degree of deterioration of the imaging environment is intermediate, when the image recognition performance reaches the limit, the driving support control (preceding vehicle follow-up control) is stopped accurately and the driving support control (preceding vehicle follow-up control) When the vehicle is temporarily stopped, it is possible to accurately cancel the suspension of the driving support control (preceding vehicle follow-up control) when the image recognition performance is recovered.

It is a figure showing the whole driving support system composition concerning this embodiment. It is a block diagram which shows the structure of the detection apparatus which concerns on this embodiment. It is a figure which shows an example of a reference | standard image. It is a figure explaining the method of a stereo matching process. It is a figure which shows the 1st distance image formed based on the reference | standard image etc. of FIG. (A) A photograph showing a reference edge image obtained from the reference image of FIG. 25 (A), and (B) a photograph showing a comparative edge image obtained from the comparison image of FIG. 25 (B). It is a figure explaining the strip-shaped division which divides | segments a distance image. It is a figure explaining the histogram produced for every division of FIG. It is a figure showing each point which plotted the distance for every division on real space. It is a figure showing each group formed by grouping each point of FIG. It is a figure showing the example of the object detected by carrying out the straight line approximation of each point of each group of FIG. It is a figure showing each object detected by being enclosed by the frame line on the reference | standard image. It is a figure which shows the driving | running | working locus | trajectory, the advancing path, and preceding vehicle of the own vehicle in real space. It is a figure explaining the state which applied the frame line set on the reference | standard image to the same pixel position of the 1st distance image. It is a figure explaining the method of the comparison with the 1st parallax in a mismatching determination means, and a 2nd parallax. It is a graph showing the increase / decrease for every frame of the count number of a counter, and a threshold value. It is a figure which represents notionally the reference | standard of the start and cancellation | release of the temporary stop of preceding vehicle follow-up control in a control means. It is a figure explaining the method of the detection of the lane candidate point on a reference | standard image. It is a figure which shows the left and right lane detected on the reference | standard image. It is a block diagram which shows the structure of the detection apparatus in a modification. It is a figure showing an example of the vote situation to a histogram in case smear has occurred. It is a figure explaining the pixel row | line | column applied and set to the reference | standard image. It is a figure explaining the surveillance field set up in the center upper part in a standard picture. It is a figure which shows an example of the image which the flare and the smear have generate | occur | produced. (A) It is a photograph which shows an example of a reference | standard image, (B) It is a photograph which shows the comparative image imaged darker than the reference | standard image of (A).

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Driving assistance system 2 Detection apparatus 3 Leading vehicle following control apparatus (driving assistance apparatus)
21 Imaging means 21a, 21b A pair of cameras (main camera, sub camera)
26a First stereo matching means 26b Second stereo matching means 29 Detection means 29B Preceding vehicle detection means 29C Counting means 29D, 29E Imaging environment determination means 31 Control means A Own vehicle C Count number Cth1 First threshold Cth2 Second threshold Dn, Dn1 ~Dn4 pixel column dp parallax dp1 first parallax dp2 second parallax LL, LR lane N number of valid data Nth threshold PB _O pixel block p1i, j luminance value R monitoring region T _C comparative image T _O reference image (image)
TE _C comparison image preprocessing image is performed (Comparative edge image)
TE _O reference image preprocessing image is performed (reference edge image)
T _Z distance image T _Z 1 first distance image T _Z 2 second distance image Vah preceding vehicle (specific target)
Z distance α The stage where the degree of deterioration of the imaging environment is most improved β The stage where the degree of deterioration of the imaging environment is intermediate γ The stage where the degree of deterioration of the imaging environment is worst Δdpth threshold

Claims (7)

A detection apparatus comprising: an imaging unit that captures an image of the surroundings of the host vehicle; and a detection unit that detects a specific target from an image captured by the imaging unit;
A driving support device that performs driving support control on the host vehicle based on the information on the specific target detected by the detection unit;
In a driving support system comprising:
Counting means for counting the number of valid data used for detection of the specific object by the detection means;
An imaging environment determination unit that determines whether the imaging environment is deteriorated based on an image captured by the imaging unit, and evaluates the degree of deterioration of the imaging environment in three stages;
When the imaging environment determination means evaluates that the degree of deterioration of the imaging environment is the worst stage, or when the degree of deterioration of the imaging environment is evaluated as an intermediate stage, and Control means for temporarily stopping the driving support control for the host vehicle by the driving support device when the number of the valid data counted by the counting means is less than a preset threshold;
Equipped with a,
The imaging environment determination unit includes a counter that increases a count number in a frame that is determined that the imaging environment is deteriorated, and that decreases a count number in a frame that is not determined that the imaging environment is deteriorated, A first threshold value and a second threshold value set to a value larger than the first threshold value are provided for the count number of the counter, and the degree of deterioration of the imaging environment when the count number of the counter is less than the first threshold value Is evaluated as the most improved stage, and when the count number of the counter is greater than or equal to the first threshold and less than the second threshold, the degree of deterioration of the imaging environment is evaluated as an intermediate stage, When the count number of the counter is greater than or equal to the second threshold value, the degree of deterioration of the imaging environment is evaluated to be the worst stage,
When the driving support control for the host vehicle is temporarily stopped, the driving support device notifies the driver of the host vehicle that the driving support control has been stopped,
The control unit evaluates that the imaging environment determination unit evaluates that the degree of deterioration of the imaging environment is most improved, or that the degree of deterioration of the imaging environment is an intermediate stage. If the number of the valid data that is counted by and said counting means in a case where is is preset threshold or more, characterized that you unpause the driving support control for the vehicle Driving support system. The specific object is a preceding vehicle,
The driving support device according to claim 1, characterized in that the preceding vehicle follow-up control to control so as to follow the preceding vehicle relative to the host vehicle based on the information of the preceding vehicle detected by said detecting means operation support system according to. The driving support system according to claim 2 , wherein the threshold value relating to the number of valid data is set to be smaller as the distance from the host vehicle to the preceding vehicle becomes longer. The specific target is the left and right lanes of the host vehicle,
The said driving assistance apparatus performs lane tracking control which controls so that it may track the said lane with respect to the own vehicle based on the information of the said lane detected by the said detection means. 3 driving support system according to any one of. The imaging means is a stereo camera that images the environment around the host vehicle with a pair of cameras and outputs a reference image and a comparison image,
The detection unit divides the reference image into pixel blocks having a predetermined number of pixels, performs a stereo matching process with the comparison image for each pixel block, and calculates a parallax calculated for each pixel block of the reference image or Based on distance images formed by associating distances with each other, a specific target is detected from the images captured by the imaging unit,
The imaging environment determining unit is configured to detect a distance of the reference image to which the pixel block to which a disparity equal to or greater than a distant disparity threshold corresponding to the distant disparity threshold corresponding to the distant disparity threshold is calculated as the disparity including the distant distance threshold including infinity as the distance. A frame in which the luminance value of each pixel in the pixel column extending in the vertical direction is searched, and the number of pixels having a luminance value equal to or higher than a predetermined luminance value is greater than or equal to a predetermined ratio of the number of pixels belonging to the pixel column On the other hand , it determines with the said imaging environment deteriorating, The driving assistance system as described in any one of Claims 1-4 characterized by the above-mentioned. The imaging means is a stereo camera that images the environment around the host vehicle with a pair of cameras and outputs a reference image and a comparison image,
The detection means includes
The reference image or an image formed by performing predetermined preprocessing on the reference image is divided into pixel blocks having a predetermined number of pixels, and the predetermined image is divided into the comparison image or the comparison image for each pixel block. A first stereo matching unit that performs a stereo matching process with the image formed by the processing and associates the calculated first parallax with each pixel block to form a first distance image;
An image formed by subjecting the reference image to another preprocessing different from the preprocessing is divided into pixel blocks having the predetermined number of pixels, and the separate preprocessing is performed on the comparison image for each pixel block. A second stereo matching unit that performs a stereo matching process with the image formed by applying the second parallax calculated to each of the pixel blocks, and forms a second distance image.
The imaging environment determination unit compares the first parallax and the second parallax of corresponding pixel blocks of the first distance image and the second distance image, and an absolute value of the difference is equal to or greater than a predetermined threshold value. The driving support system according to any one of claims 1 to 5 , wherein the imaging environment is determined to be deteriorated with respect to a frame in which a predetermined number or more of pixel blocks exist. The imaging means is a stereo camera that images the environment around the host vehicle with a pair of cameras and outputs a reference image and a comparison image,
The detection unit divides the reference image into pixel blocks having a predetermined number of pixels, performs a stereo matching process with the comparison image for each pixel block, and calculates a parallax calculated for each pixel block of the reference image or Based on distance images formed by associating distances with each other, a specific target is detected from the images captured by the imaging unit,
The imaging environment determination unit sets a rectangular monitoring area extending in the horizontal direction of the image at the upper center of the image, calculates a luminance average value of the monitoring area, and in the monitoring area in the vertical direction of the image The luminance difference between each pixel pair is calculated, the number of pixel pairs having a luminance difference equal to or greater than a predetermined value is calculated as the number of edges, the average luminance value is greater than a predetermined first determination value, and the number of edges is The driving support system according to any one of claims 1 to 6 , wherein the imaging environment is determined to be deteriorated with respect to a frame smaller than a predetermined second determination value.