JP4863951B2 - Traffic light recognition device - Google Patents

Description

  The present invention relates to a traffic signal recognition device that captures and recognizes a traffic signal with an imaging means mounted on a vehicle.

The stereo camera installed in the vehicle recognizes the color (red, yellow or blue) of the traffic light ahead of the vehicle, the traffic light is red, and the distance between the vehicle and the traffic signal is within a certain value. Further, Japanese Patent Application Laid-Open Publication No. 2004-228561 is known that issues a warning to the driver so that the own vehicle can surely stop at a position in front of the traffic signal when the vehicle speed of the own vehicle is equal to or higher than a certain value.
JP 2004-199148 A

  By the way, there is a case where an arrow signal is provided in addition to the red, yellow or blue light emitters in the traffic light, and even if the color of the light emitter of the traffic light is red, the vehicle can travel in the direction in which the arrow signal is lit. . However, in the conventional traffic signal recognition device that cannot recognize the arrow signal, there is a possibility that an unnecessary warning is issued to the driver when the arrow signal is lit and the vehicle can travel, and the driver may feel uncomfortable.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to reliably identify and extract an arrow signal of a traffic light based on an image captured by an imaging unit.

In order to achieve the above object, according to the first aspect of the present invention, in a traffic signal recognition device that captures and recognizes a traffic signal by an imaging means mounted on a vehicle, According to the red / yellow signal detecting means for detecting the yellow signal and the red signal detected by the red / yellow signal detecting means below the red signal or the yellow signal detected by the red / yellow signal detecting means a search area setting means for setting a search region Te, the search area setting traffic signal recognition device, characterized in that it comprises a arrow signal detecting means for detecting the arrow signals within the search area set by means are proposed.

According to the second aspect of the present invention, in addition to the configuration of the first aspect, the arrow-shaped template is set according to the size of the red signal or the yellow signal detected by the red / yellow signal detecting means. There is proposed a traffic light recognition apparatus including a template setting unit, wherein the arrow signal detection unit detects the arrow signal by comparison with the template.

According to a third aspect of the present invention, in a traffic signal recognition device for capturing and recognizing a traffic light by an imaging means mounted on a vehicle, a red / yellow signal for detecting a red signal or a yellow signal among the light emitters of the traffic light. Yellow signal detection means, search area setting means for setting a search area below the red signal or yellow signal detected by the red / yellow signal detection means, and an arrow signal within the search area set by the search area setting means And an arrow signal detecting means, and a template setting means for setting an arrow-shaped template according to the magnitude of the red signal or the yellow signal detected by the red / yellow signal detecting means. A traffic signal recognizing device is proposed that detects the arrow signal by comparison with the above .

  The television camera 11 of the embodiment corresponds to the image pickup means of the present invention, and the traffic light determination means M4 of the embodiment corresponds to the red / yellow signal detection means of the present invention.

According to the first or third aspect of the invention, the red / yellow signal detection means detects a red signal or a yellow signal in the light emitter of the traffic light, and the search area setting means is below the red signal or the yellow signal . Since the search area is set and the arrow signal detection means detects the arrow signal in the search area, it becomes possible to set the search area at a position where the presence of the arrow signal is high, and the search of the minimum area is possible. The arrow signal can be reliably detected while setting the area and suppressing the calculation load.

In particular, according to the invention of claim 1. The search area setting means sets the search area according to the size of the red or yellow signal detected by the red / yellow signal detection means. The arrow signal can be reliably detected while limiting to the limit.

In particular, according to the configuration of claim 3, when the template setting means sets an arrow-shaped template according to the size of the red signal or yellow signal detected by the red / yellow signal detection means, the arrow signal detection means Therefore, the arrow signal can be detected with high accuracy by appropriately setting the size of the template.

According to the second aspect of the present invention, when the template setting means sets an arrow-shaped template according to the size of the red or yellow signal detected by the red / yellow signal detecting means, the arrow signal detecting means Therefore, the arrow signal can be detected with high accuracy by appropriately setting the size of the template.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

  1 to 10 show a first embodiment of the present invention. FIG. 1 is a block diagram of an electronic control unit of a traffic signal recognition device, and FIG. 2 is a diagram showing an image in front of the own vehicle taken by a TV camera. 3 is a diagram showing the light emitters of the traffic lights extracted at different thresholds, FIG. 4 is a flowchart of the main routine, FIG. 5 is a flowchart of the subroutine of step S1 in FIG. 4, and FIG. 6 is a flowchart of the subroutine of step S2 in FIG. 7 is a flowchart of the subroutine of step S3 in FIG. 4, FIG. 8 is a map for determining the light emitter of the traffic light from the position and radius of the second object, FIG. 9 is an enlarged view of part 9 in FIG. 2, and FIG. It is explanatory drawing of this template.

  As shown in FIG. 1, the electronic control unit U of the traffic signal recognition apparatus of the present embodiment includes a first extraction unit M1, a re-extraction area setting unit M2, a second extraction unit M3, and a traffic signal determination unit M4 (red). -Yellow signal detection means), second threshold value setting means M5, search area setting means M6, template setting means M7, and arrow signal detection means M8.

  The electronic control unit U stores a first threshold value of luminance in advance, and the first extraction means M1 has the first luminance value from the front image (see FIG. 2) in the traveling direction of the host vehicle captured by the TV camera 11. A first object having a threshold value or more is extracted. As shown in FIG. 2, in the image captured by the television camera 11, signboards 13, 14 and the like are shown in addition to the traffic light 12, and the signboards 13, 14 may be erroneously recognized as the traffic light 12. It is necessary to be surely excluded from the object. There is a large difference in luminance between the traffic light 12 and the signboards 13 and 14. The red signal 15, the yellow signal 16 and the blue signal 17 of the traffic light 12 that emit light by lamps and LEDs themselves have high brightness, and the signs 13 and 14 that do not emit light themselves. The brightness and the brightness of the arrow signals 18 and 19 of the traffic light 12 are lowered. Therefore, by setting the first threshold value to be higher than the luminance of the signboards 13 and 14 and lower than the luminance of the light emitter of the traffic light 12, there is a possibility of the light emitter of the traffic light 12 excluding the signboards 13 and 14. A certain first object can be extracted.

  FIG. 3A shows a case where the threshold value is too low, and an area that protrudes beyond the contour of the light emitter of the traffic light 12 is extracted. With such a too low threshold, not only the size and shape of the illuminant cannot be recognized correctly, but also the signs 13 and 14 are extracted, so that the traffic lights 12 and the signs 13 and 14 cannot be identified.

  FIG. 3B shows the case where the first threshold value is adopted as the threshold value, and not the entire light emitter of the traffic light 12 but only the high luminance portion at the center is extracted so that the size and shape of the light emitter are correct. Although it cannot be recognized, since the signboards 13 and 14 are not extracted, the traffic light 12 and the signboards 13 and 14 can be identified.

  The re-extraction area setting means M3 sets a predetermined range (for example, 40 pixels × 40 pixels) as the re-extraction area 22 (see FIG. 9) around the portion where the maximum luminance of the light emitter is detected, and the second extraction means M3 performs re-extraction in the re-extraction area 22 using a new second threshold value. The second threshold value setting means M5 for setting the second threshold value used at this time extracts the second object with a second threshold value lower than the first threshold value. Since the region where re-extraction is performed is a limited range in the vicinity of the first object (for example, the light emitter of the traffic light 12), even if extraction is performed with a low second threshold, the low-intensity signboards 13, 14 and the like are There is no risk of being extracted as two objects.

  When extraction is performed using the second threshold value, as shown in FIG. 3B, the first object (for example, the light emitter of the traffic light 12) extracted with the first threshold value having a high value is larger than the actual size. In contrast, as shown in FIG. 3C, the light emitter of the traffic light 12 can be extracted with a size close to the actual size. And the traffic light determination means M4 determines whether the 2nd object extracted by the 2nd extraction means M3 is a light-emitting body of the traffic light 12, based on the dimension, a position, a color, etc.

  When the red signal 15 or the yellow signal 16 is lit, the search area setting means M6 searches for a predetermined range (for example, 40 pixels × 80 pixels) based on the brightest portion of the illuminating body that is lit. Region 23 (see FIG. 9) is set. The predetermined range of the search area 23 is determined according to the diameter of the detected red signal 15 or yellow signal 16, and is set to a necessary minimum size including the arrow signals 18 and 19 therein.

  In the electronic control unit U, templates 20, 21 (see FIG. 10) having clear arrow shapes for right turn, straight advance, and left turn are stored in advance, and template setting means is determined from the diameter of the detected red signal 15 or yellow signal 16. M7 selects the templates 20 and 21 having dimensions corresponding to the diameter of the red signal 15 or the yellow signal 16 and performs a process of blurring the peripheral portions of the templates 20 and 21.

  Next, the above operation will be described in more detail based on the flowcharts of FIGS.

  In step S1 of the main routine of FIG. 4, a lighting area including a light emitter area of the traffic light 12 (high brightness area) is extracted from the image of the television camera 11, and the traffic light 12 is detected from the lighting area in a subsequent step S2. In step S3, the arrow signals 18 and 19 are detected from the position of the traffic light 12.

  In step S11 of the flowchart of FIG. 5 showing the lighting region extraction routine, an image in front of the vehicle is picked up by the TV camera 11, and each pixel of the image is represented by RGB (three primary colors of red, green and blue light) in step S12. Color representation method) to HSV [color representation method by chromaticity (hue), saturation, brightness (brightness)]. In subsequent step S13, a pixel having a first threshold value or more is detected as a first object from each pixel of the image. The first threshold value is set for each of chromaticity, saturation, and luminance. For example, the chromaticity is 0 to 150, 331 to 360 (in terms of 0 to 360) (or 0 to 100%). 0 to 42%, 55 to 100%), saturation is 270 to 360 (75 to 100%), and luminance is 180 to 360 (50 to 100%).

  In the subsequent step S14, the maximum values of the luminance and saturation of the pixels equal to or higher than the first threshold value are calculated, and in step S15, the second threshold value including the luminance and saturation is set to 50% of the maximum value. That is, the brightness of the second threshold is set to 50% of the maximum brightness, and the saturation of the second threshold is set to 50% of the maximum saturation.

  In the subsequent step S16, a re-extraction region 22 of 40 pixels × 40 pixels is set centering on the pixel having the maximum luminance, and in the re-extraction region 22, pixels having the second threshold value or more are extracted as the second object in step S17.

  The area formed by the pixels having the second threshold value or more in step S21 of the flowchart of FIG. 6 showing the traffic light detection routine is a circle that is the shape of the light emitter of the traffic light 12, and the pixels having the second threshold value or more are formed in step S22. If the height and diameter of the center of the area are within a predetermined range, it is determined that this is the light emitter of the traffic light 12.

  FIG. 8 is a map showing the criteria for judgment in step S22. The vertical axis represents the position of the pixel with the maximum luminance on the screen, and the horizontal axis represents the radius of the region formed by the pixel having the second threshold value or higher. If the traffic light 12 as the second object is close to the own vehicle, the position is on the upper side of the screen and the radius is larger. Conversely, if the traffic light 12 is far from the own vehicle, the position is on the lower side of the screen. In addition, since the radius is small, the second object existing in the shaded area in FIG. 8 can be determined as the light emitter of the traffic light 12.

  When the second object is thus determined to be the light emitter of the traffic light 12, the process proceeds to step S23, where the chromaticity is 0 to 30, 331 to 360 (0 to 8%, 92 to 100%). If it is within the range, it is determined in step S24 that the red signal 15 is lit. If the chromaticity is in the range of 31 to 90 (9 to 25%) in step S25, it is determined that the yellow signal 16 is lit in step S26. If the chromaticity is in the range of 91 to 150 (26 to 42%) in step S27, it is determined that the blue (green) signal is lit in step S28.

  If the red signal 15 or the yellow signal 16 is detected in step S31 of the flowchart of FIG. 7 showing the arrow signal detection routine, the red signal 15 or yellow signal 16 detected in step S32 is used as a reference and the red signal 15 is displayed below the red signal 15 or yellow signal 16. A search area 23 (see FIG. 9) having a predetermined area corresponding to the diameter of the signal 15 or the yellow signal 16 is set. The reason why the red signal 15 or the yellow signal 16 is used as a reference is that the arrow signals 17 and 18 are lit when the red signal 15 or the yellow signal 16 is lit. Since the positions of the arrow signals 17 and 18 are in a fixed positional relationship with respect to the position of the red signal 15 or the yellow signal 16, when the red signal 15 is turned on, a search for a predetermined area is performed on the basis of the red signal 15. When the area 23 is set and the yellow signal 16 is lit, the search area 23 having a predetermined area is set based on the yellow signal 16.

  In subsequent step S33, the sizes of the templates 20 and 21 for detecting the arrow signals 17 and 18 are set based on the diameter of the red signal 15 or the yellow signal 16 that is currently lit. The reason is that the diameter of the red signal 15 or the yellow signal 16 and the magnitude of the arrow signals 17 and 18 are in a fixed relationship, so it is necessary to set the templates 20 and 21 having the same dimensions as the actual arrow signals 17 and 18. Because there is.

  In subsequent step S34, the templates 20 and 21 are subjected to blurring processing by filtering. The reason is that the arrow signals 17 and 18 that are actually detected by the television camera 11 are not clear but blurred, but when compared with the clear templates 20 and 21, there is a possibility that it is erroneously determined that they do not match. Because. In the blurring process, an average value of 9 pixels of 3 × 3 centering on each pixel of the clear templates 20 and 21 is calculated, and the peripheral value of the arrow is blurred by using the average value as the luminance of the pixel. be able to.

  In step S35, the actual arrow signals 17 and 18 that are lit are compared with the templates 20 and 21, and if they match the template 20 that indicates the right turn arrow, it is determined in step S36 that the right turn arrow signal is lit. If it matches with the template (not shown) indicating the left turn arrow in step S37, it is determined in step S38 that the left turn arrow signal is lit, and if it matches with the template 21 indicating the straight arrow in step S39. In step S30, it is determined that the straight arrow signal is lit, and in any case, it is determined that the arrow signal is not output in step S41.

  If neither the red signal 15 nor the yellow signal 16 is detected in step S31, it is determined in step S42 that no arrow signal is output.

  As described above, by extracting the first object having a luminance equal to or higher than the first threshold value from the image in front of the vehicle imaged by the television camera 11, objects such as signs 14 and 14 having low luminance are excluded and the traffic light 12 Only the first object with high luminance including the illuminant is extracted, but the traffic light 12 lacks the low luminance part of the illuminant and only the high luminance part remains, so it may not be correctly recognized as the illuminant of the traffic light 12 There is sex. Therefore, a re-extraction area 22 having a predetermined size is set around the first object, and the luminance is not less than the second threshold value less than the first threshold value in the re-extraction area 22 where no low-luminance object such as the signboard 13 or 14 exists. Thus, the entire light emitter of the traffic light 12 can be extracted by excluding objects such as the signboards 13 and 14 while keeping the calculation load low. Thereby, it can be accurately determined whether or not the second object is the light emitter of the traffic light 12. In addition, since the second threshold value for extracting the first object is set as a half value of the maximum luminance of the first object, the light emitter of the traffic light 12 can be extracted with the optimum second threshold value.

  Further, the red signal 15 or the yellow signal 16 of the traffic light 12 is detected, and the arrow signals 18 and 19 are detected in the search area 23 having a predetermined positional relationship with the red signal 15 or the yellow signal 16. , 19 can be set at a position where the probability is high, and the arrow signals 18 and 19 can be reliably detected while setting the search area 23 with the minimum area and suppressing the calculation load. Can do. At this time, the search area 23 is set according to the detected red signal 15 or yellow signal 16, so that the size of the search area 23 can be appropriately set without excess or deficiency.

  In addition, since the arrow signals 18 and 19 are detected by comparison with the templates 20 and 21 set according to the size of the red signal 15 or the yellow signal 16, the size of the templates 20 and 21 is set appropriately and the arrow signal 18 is detected. , 19 can be detected with high accuracy.

  Further, since the templates 20 and 21 are blurred by filtering so as to be similar to the sharpness of the actually extracted arrow signals 18 and 19, the extracted arrow signals 18 and 19 are compared with the templates 20 and 21. When determining the similarity, the determination accuracy can be increased.

  Next, a second embodiment of the present invention will be described with reference to FIG.

  The first embodiment described with reference to FIG. 7 assumes a case in which the right turn arrow signal, the left turn arrow signal, and the straight arrow signal do not turn on at the same time. However, in the second embodiment, the right turn It is assumed that two or three of the left arrow signal, the left turn arrow signal, and the straight arrow signal are lit simultaneously. 7 and 11, since the contents of the same step numbers are the same, the following description will focus on the differences.

  The actual arrow signals 17 and 18 lit in step S35 are compared with the templates 20 and 21, and if they match the template 20 indicating the right turn arrow, it is determined that the right turn arrow signal is lit in step S36A. If it does not match, it is determined in step S36B that the right turn arrow signal is not lit, and if it matches the template (not shown) indicating the left turn arrow in step S37, the left turn arrow signal is lit in step S38A. If it does not match, it is determined in step S38B that the left turn arrow signal is not lit, and if it matches the template 21 indicating the straight arrow in step S39, the straight arrow signal is lit in step S40A. If they do not match, it is determined in step S40B that the straight arrow signal is not lit. If neither the red signal 15 nor the yellow signal 16 is detected in step S31, it is determined in step S42 'that no arrow signal is output.

  Although the embodiments of the present invention have been described above, various design changes can be made without departing from the scope of the present invention.

  For example, in the embodiment, the horizontal traffic signal 12 is exemplified, but the present invention can also be applied to a vertical traffic signal.

The block diagram of the electronic control unit of the traffic signal recognition apparatus which concerns on 1st Embodiment The figure which shows the picture ahead of the own car which is photographed with the television camera Diagram showing light emitters of traffic lights extracted with different thresholds Main routine flowchart Flowchart of the subroutine of step S1 in FIG. Flowchart of the subroutine of step S2 in FIG. Flowchart of the subroutine of step S3 in FIG. Map for determining light emitter of traffic light from position and radius of second object 9 enlarged view of FIG. Illustration of arrow signal template The figure corresponding to the said FIG. 7 based on 2nd Embodiment.

11 TV camera (imaging means)
12 traffic light 15 red signal 16 yellow signal 18 arrow signal 19 arrow signal 20 template 21 template 23 search area M4 traffic light determination means (red / yellow signal detection means)
M6 Search area setting means M7 Template setting means M8 Arrow signal detection means

Claims (3)

In a traffic signal recognition device that captures and recognizes a traffic signal (12) by an imaging means (11) mounted on a vehicle,
A red / yellow signal detecting means (M4) for detecting a red signal (15) or a yellow signal (16) of the light emitters of the traffic light (12);
The red signal (15) or yellow signal (16) detected by the red / yellow signal detection means (M4) below the red signal (15) or yellow signal (16) detected by the red / yellow signal detection means (M4). Search area setting means (M6) for setting the search area (23) according to the size of
Arrow signal detection means (M8) for detecting an arrow signal (18, 19) within the search area (23) set by the search area setting means (M6);
A traffic light recognizing device comprising: Before Kiaka, yellow signal detecting means (M4) at the detected red light (15) or the template setting means for setting a template (20, 21) of arrow-shaped according to the size of the yellow light (16) to (M7) The traffic light recognition device according to claim 1, wherein the arrow signal detection means (M8) detects the arrow signal (18, 19) by comparison with the template (20, 21). In recognizing traffic signal recognition device signals machine (12) and picked up by the image pickup means mounted on vehicles (11),
A red / yellow signal detecting means (M4) for detecting a red signal (15) or a yellow signal (16) of the light emitters of the traffic light (12);
Search area setting means (M6) for setting the search area (23) below the red signal (15) or yellow signal (16) detected by the red / yellow signal detection means (M4);
Arrow signal detection means (M8) for detecting an arrow signal (18, 19) within the search area (23) set by the search area setting means (M6);
A template setting means (M7) for setting an arrow-shaped template (20, 21) according to the size of the red signal (15) or the yellow signal (16) detected by the red / yellow signal detection means (M4); Prepared,
The traffic light recognition device, wherein the arrow signal detection means (M8) detects the arrow signal (18, 19) by comparison with the template (20, 21) .

Images