KR101237374B1 - Image processing method for detecting traffic light - Google Patents

Abstract

PURPOSE: An image processing method for traffic light discrimination is provided to prevent the degradation of a high beam control auxiliary device by continuously irradiating a high beam according to the recognition of a light source of a traffic light. CONSTITUTION: A light source object group is designated by using a set foreground pixels(S702). A highest illumination point(P) is designated in the center of the light source object group(S703). A traffic light estimation area is extracted from the light source object group(S705). A peripheral part of the traffic light estimation area is designated(S706). If U, V color gamut coordinate value in the peripheral part is greater than an estimation value of the peripheral part, a traffic light is determined(S709,S710). [Reference numerals] (AA) No; (BB) Yes; (S701) Setting a landscape pixel area; (S702) Designating a light source object group; (S703) Designating a highest illumination point(P); (S704) Inverting an image brightness value; (S705) Extracting a traffic light estimation area; (S706) Designating a bright area, Rb, in a peripheral part of the traffic light estimation area; (S707) Calculating average R, G, B brightness values in Rb; (S708) Converting to a Yuv color space; (S710) Determining as a traffic light; (S711) Not a traffic light

Description

Image processing method for traffic light discrimination {IMAGE PROCESSING METHOD FOR DETECTING TRAFFIC LIGHT}

The present invention relates to an image processing method for determining a traffic light, and more particularly, to an image processing method for identifying a traffic light for discriminating a traffic light from image data of a digital camera attached to a vehicle.

With the recent increase in the penetration rate of the video accident recording device for a car, the intelligent camera device that helps the user's safe driving and convenience such as the lane departure warning device, the preceding vehicle collision warning device, and the high beam control judgment assistance device using the camera attached to the vehicle Demand is increasing.

Among them, High Beam Assist is mainly used to help the driver secure vision at night. High beam irradiation is required when there is a low light environment where there is no ambient light while driving the vehicle and glare does not occur in the preceding vehicle and the opposite vehicle. Is not required.

The image processing method used in the conventional high beam determination assistance apparatus recognizes a low light source such as a street light and another light source, for example, a traffic light, as a light source such as a traffic light. Although there is a limit to acting as a light for securing the field of view, it is recognized as a street lamp, there is a problem that the performance degradation due to the error of illuminance judgment.

An object of the present invention is to provide an image processing method for determining a traffic light that can prevent the performance degradation of the high beam control by distinguishing the street light source and the traffic light source.

In the image processing method for determining the traffic light according to the first invention of the present application, the image processing method for determining the traffic light from the image received from the image sensor of the camera disposed in the vehicle, the light source data received from the image sensor in the foreground region Dividing into a background area; Designating the foreground area as a light source object group and designating a peripheral area of the light source object group as a peripheral part of the light source object group; And determining the color of the periphery of the light source object group.

In the image processing method for determining the traffic light according to the second invention of the present application, in the image processing method for determining the traffic light from the image received from the image sensor of the camera disposed in the vehicle, a predetermined area in the image received from the image sensor Setting a region of interest; Designating a light source object group using a foreground pixel in the ROI; Setting an area having a brightness greater than an estimated peripheral value of a peripheral area of the light source object group as a peripheral part of the light source object group; And determining the color of the periphery of the light source object group.

Preferably, comparing the illuminance value of the light source detected in the region of interest with a predetermined threshold illuminance value according to a position on the image; And removing the light source from the ROI when the illuminance value of the light source is greater than the threshold illuminance value.

Preferably, the step of designating a light source object group may include: dividing an illumination intensity of an individual pixel in the ROI into a foreground pixel and a background pixel by comparing the illumination intensity value with an estimated illumination intensity; And determining a region where the plurality of foreground pixels neighbor each other as a light source object group.

Preferably, the determining of the color of the periphery of the light source object group comprises: detecting a region in which the brightness of the red or green is greater than or equal to the periphery estimated illuminance value; Calculating a ratio of a red or green component to a region in which the brightness of the red or green is greater than or equal to the peripheral estimated illumination value; And determining whether the light source object group is a traffic light.

In the image processing method for determining the traffic light according to the third aspect of the present invention, in the image processing method for determining the traffic light from the image received from the image sensor of the camera disposed in the vehicle, setting a portion of the image as the region of interest ; Extracting a foreground pixel from the ROI; Designating a highest illumination point of the foreground pixels; Inverting an image brightness value of each of the foreground pixels; Extracting a traffic light estimation region from the image of which brightness value is inverted using gradient information about the highest illumination point; Designating a periphery of the traffic light estimation region; And determining the color of the periphery.

Preferably, the step of designating the periphery of the traffic light estimation region designates a bright portion of the periphery of the traffic light estimation region as the periphery of the traffic light estimation region.

Preferably, the determining of the color of the peripheral part may include calculating an average brightness value of red, green, and blue in the peripheral part; Converting the average brightness value into a color space coordinate value; And determining a color by comparing the color space coordinate value and an estimated estimated value of a peripheral portion of a traffic light source determined as a peripheral portion of the traffic light source.

Preferably, the step of determining the color, if the color space coordinate value is larger than the estimated value of the periphery of the traffic light source is determined as a traffic light.

According to the present invention, when the street light source and the traffic light source are distinguished and recognized as the traffic light source, the high beam irradiation is maintained and the performance of the high beam control auxiliary device is prevented.

1 is a schematic diagram showing a method of setting a region of interest for detecting a street light source and a traffic light source according to an embodiment of the present invention;
2 is a schematic diagram for explaining the reason for removing a large light source near the vanishing point;
3A is a schematic diagram showing illuminance of a light source on the image sensor and its surroundings,
3B is a schematic diagram showing a detection method of a light source object group when the threshold value is 90;
4 is a schematic diagram showing a detected light source object group and its surroundings;
5 is a flow chart for determining the color of the light source according to an embodiment of the present invention;
6 is a flowchart illustrating a color judging method according to an embodiment of the present invention;
7 is a flow chart for determining the color of the light source according to another embodiment of the present invention;
8 is a schematic diagram showing a preprocessing process for extracting a peripheral portion of a light source object group;
FIG. 9 is a schematic diagram illustrating conversion of average brightness values of R, G, and B in R _b into YUV color space coordinates. FIG.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

For example, without departing from the scope of the present invention, the second component may be referred to as a first component, and similarly, the first component may also be referred to as a second component.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Now, an image processing method for determining a traffic light according to an exemplary embodiment of the present invention will be described in detail with reference to the drawings. It may be omitted.

1 is a schematic diagram illustrating a method of setting a region of interest for detecting a street light source and a traffic light source according to an embodiment of the present invention, and FIG. 2 is a schematic diagram for explaining a reason for removing a large light source near a vanishing point.

Referring to FIG. 1, the upper portions 110 and 120 of the horizon 130 on the image generated by the image sensor are set as a region of interest for detecting a street light source and a traffic light source. This is because the streetlight layout on the road is somewhat different depending on the number of lanes and the installation location on the road, but is usually determined to be installed at least 6m above the ground. In addition, since the point close to the horizon 130 on the image is located far away, such a point cannot be considered, so the region of interest 110 and 120 is set by having a margin 140 from the horizon 130 on the image. .

In addition, all light sources 100, 121, 122 existing in the ROI 110, 120 are detected as the light source.

Referring to FIGS. 1 and 2, the ROIs 110 and 120 are set with a margin 140 from the horizon 130 on the image. The ROIs 110 and 120 are set to the horizon 130 on the image within the ROIs 110 and 120. Among the light sources detected in the adjacent region 120, the light source 122 having an illuminance larger than a predetermined threshold illuminance value is removed according to a position on the image. As shown in FIG. 2, since the region 120 near the horizon 130 on the image is a very far point, only a low light source may exist. Therefore, when the light source 122 of illuminance larger than the predetermined threshold illuminance value is detected according to the position on the image, the light source is removed because it cannot be a street light source or a traffic light source.

3 is a schematic diagram illustrating a detection method of a light source according to an exemplary embodiment of the present invention, which will be described below using any one light source in the ROIs 110 and 120 as an example. In order to detect the light source, the illuminance of the individual pixels of the image is regarded as the foreground pixel 310 when it is higher than the preset light source estimated illuminance value and the light source estimated illuminance value, and when it is lower than the light source estimated illuminance value, it is regarded as the background pixel 320. . The pixel-by-pixel comparison may proceed from left to right or right to left while processing the regions of interest 110 and 120 in a column unit, and may proceed from top to bottom or top to bottom while processing in a row unit.

An area where the plurality of foreground pixels 310 are adjacent to each other is regarded as the light source object group 300. That is, after separating the beginning of the foreground and the beginning of the background by making a line, the divided areas are grouped by objects, and the area where the foreground is objectized among the objectized areas is called the light source object group 300.

4 is a schematic diagram showing the detected light source object group and its surroundings.

Referring to FIG. 4, the center of the light source has a very high illuminance and is saturated on the image sensor to be expressed in white. However, the periphery of the light source is not saturated on the image sensor to distinguish colors. That is, the light source object group 300 has a very high illuminance and is saturated on the image sensor to be expressed in white. However, since the peripheral portion 400 of the light source object group 300 is not saturated on the image sensor, color can be distinguished. In order to distinguish colors, the sum of the R, G, and B brightnesses estimated by bilinear interpolation in the peripheral area 410 of the light source object group 300 is greater than the preset peripheral estimated illuminance value. The periphery 400 of the group 300 is divided. Here, the preset peripheral estimation illuminance value means a predetermined illuminance value that can be estimated by the peripheral unit 400 of the light source object group 300.

For example, an area R _R having a brightness value of R estimated by the bilinear interpolation method is greater than or equal to zero in the peripheral part 400, and an area R _G having a brightness value of _G or more estimated by the bilinear interpolation method is greater than zero. R _R And R _G The ratio of the number of pixels of red and green components in the area as below formula Ratio _R , Ratio _G .

Ratio _R = r / N _R * 100 (%), N _R : R _R The total number of pixels in the area

r: R _R The number of pixels with the red component found in the area

Ratio _G = g / N _G * 100 (%), N _G : R _G The total number of pixels in the area

g: R _G The number of pixels with the green component found in the area

Ratio _R , Ratio _G obtained by the above formula It is possible to distinguish whether the light source object group 300 is a traffic light source having a green or red component by using the.

That is, when Ratio _MAX = max (Ratio _R , Ratio _G ), if the ratio _MAX is greater than the traffic light source estimated value, the light source object group 300 is determined to be a traffic light source.

5 is a flow chart for determining the color of the light source according to an embodiment of the present invention.

Referring to FIG. 5, in operation S501 of setting regions of interest 110 and 120 for detecting a street light source and a traffic light source in an image on an image sensor, all light sources existing on the regions of interest 110 and 120 are detected. (S502), removing the nonsense light source among the light sources detected in the ROI (110, 120) (S503), extracting the foreground pixel 310 from the detected light source (S504), extracted foreground pixel 310 Designating the light source object group 300 using the step S505, setting the periphery 400 of the light source object group to determine the color in the light source object group 300, and setting the set light source object group. Determining the color of the peripheral portion 400 of (S507).

6 is a flowchart illustrating a color judging method in a peripheral area according to an embodiment of the present invention.

Obtaining a region R _{R in} which the brightness value of _R is not 0 at the periphery and a region R G in which the brightness value of _G is not 0 (S601), R _R , R _G In step S602, the ratio of the number of red and green pixels is calculated according to the following equation in the area, and when Ratio _MAX = max (Ratio _R , Ratio _G ), determining that the ratio _MAX is the traffic light source if the ratio _MAX is greater than the estimated traffic light source. (S603).

7 is a schematic view showing the pre-processing for extracting the periphery of the flow chart, and Figure 8 is a light source group of objects to determine the color of the light source according to another embodiment of the present invention, Figure 9 is R _b within the R, G, It is a schematic diagram which shows that the average brightness value of B was converted into the YUV color space coordinate.

Referring to FIG. 7, a pixel estimated as a light source in the ROI is set as a foreground pixel 310 (S701), and a light source object group 300 is designated using the foreground pixel 310 (S702). The highest illumination point P is specified at the center of the object group 300 (S703), and the image brightness value is inverted for each foreground pixel 310 (S704).

In the light source object group 300 in which the brightness value is inverted, the traffic light estimation region is extracted by the image segmentation method using the gradient information around the highest illumination point P (S705), and the bright part of the peripheral area of the traffic light estimation region is extracted. The peripheral portion R _b of the traffic light estimation region is designated (S706).

Calculate the R, G, B average roughness values in the periphery R _b of the traffic light estimation region (S707), convert the R, G, B average roughness values into YUV color space coordinates (S708), and convert the converted U, V When the value and the light source light peripheral edge estimate determined by the periphery of the traffic light source is compared (S709), if the U, V value is larger than the traffic light source peripheral estimate is judged as a traffic light (S710), if the U, V value is smaller than the traffic light source reference value It is determined that the traffic light is not (S711).

Referring to FIG. 8, a pixel having a value larger than a light source estimated illuminance value in a region of interest is designated as a foreground pixel. When a plurality of foreground pixels are adjacent to a light source object group, a hole 801 may be present inside the light source object group as a foreground pixel but not a foreground area due to noise components in the image (FIG. 8A). . Since the hole 801 cannot exist inside the light source object group, the hole 801 is regarded as a foreground pixel and filled with the foreground pixel (FIG. 8B).

The highest illuminance point P is designated within the center of the light source object group 300 to invert the image brightness value of each foreground pixel (FIG. 8C).

In the image in which the brightness value is inverted, the bright part of the peripheral area of the light source object group is designated as the peripheral part R _b of the light source area by the image segmentation method using the gradient information around the highest illumination point P (Fig. 8D). .

Referring to FIG. 9, in the method of determining color, first, an average illuminance value of a peripheral part is obtained from R, G, and B images, and the average illuminance value is converted into a YUV color space coordinate. If the U and V color values are in the red or green range, the light source object group is determined as the traffic light source.

One embodiment of the present invention described above should not be construed as limiting the technical spirit of the present invention. The scope of protection of the present invention is limited only by the matters described in the claims, and those skilled in the art will be able to modify the technical idea of the present invention in various forms. Therefore, such improvements and modifications will fall within the protection scope of the present invention as long as it will be apparent to those skilled in the art.

100: light source 110: region of interest
120: region of interest 121: light source
122: meaningless light source
130: horizon on the image 131: vanishing point
300: light source object group 310: foreground pixel
320: background pixels
400: periphery 410: periphery
801: Hole

Claims (10)

delete delete An image processing method for discriminating a traffic light from an image received from an image sensor of a camera disposed in a vehicle,
Setting a predetermined region as a region of interest in the light source data received from the image sensor;
Designating a light source object group using a foreground pixel in the ROI;
Designating a peripheral area of the light source object group as a peripheral part of the light source object group;
Determining a color of a periphery of the light source object group;
Comparing the illuminance value of the light source detected in the region of interest with a predetermined threshold illuminance value according to a position on the image; And
Removing the light source from the ROI when the illuminance value of the light source is greater than the threshold illuminance value.
Image processing method for determining the traffic light comprising a.
The method of claim 3, wherein the specifying of the light source object group comprises:
Dividing the illuminance of the individual pixel in the ROI into a foreground pixel and a background pixel by comparing with an estimated light source illuminance value; And
Determining a region where a plurality of foreground pixels neighbor each other as a light source object group
Image processing method for determining the traffic light comprising a.
The method of claim 3, wherein the determining of the color of the periphery of the light source object group comprises:
Detecting a color discrimination target area having a red or green brightness greater than a predetermined peripheral estimated illumination value and smaller than a white illumination value;
Calculating a ratio of red or green components with respect to the color determination target region; And
Determining whether the light source object group is a traffic light
Image processing method for determining the traffic light comprising a.
An image processing method for discriminating a traffic light from an image received from an image sensor of a camera disposed in a vehicle,
Setting a portion of the image as a region of interest;
Extracting a foreground pixel from the ROI;
Designating a highest illumination point of the foreground pixels;
Inverting an image brightness value of each of the foreground pixels;
Extracting a traffic light estimation region from the image of which brightness value is inverted using gradient information about the highest illumination point;
Designating a periphery of the traffic light estimation region; And
Determining the color of the periphery
Image processing method for determining the traffic light comprising a.
The method according to claim 6,
Comparing the illuminance value of the light source detected in the region of interest with a predetermined threshold illuminance value according to a position on the image; And
Removing the light source from the ROI when the illuminance value of the light source is greater than the threshold illuminance value.
Image processing method for determining the traffic light further comprising.
delete The method according to claim 6,
Determining the color of the peripheral portion,
Calculating average brightness values of red, green, and blue in the periphery;
Converting the average brightness value into a color space coordinate value; And
Determining a color by comparing the color space coordinate value with an estimated value of a peripheral part of a traffic light source determined as a peripheral part of a traffic light source;
Image processing method for determining the traffic light comprising a. delete

Images