JP7344776B2 - Traffic light recognition method and traffic light recognition device - Google Patents

Description

The present invention relates to a traffic light recognition method and a traffic light recognition device.

2. Description of the Related Art Conventionally, traffic light recognition devices are known that recognize traffic lights from images captured in the direction in which a vehicle is traveling. Patent Document 1 discloses a traffic light recognition device that sets a traffic light detection area in an image captured by a camera based on map information and the position and orientation of the own vehicle detected by a vehicle position detection device. This traffic light recognition device performs image processing on traffic lights existing within a detection area to detect the lighting state of the traffic lights.

Japanese Patent Application Publication No. 2007-241469

In the traffic light recognition device described in Patent Document 1, in places where the distance between traffic lights is short, such as in an urban area, traffic lights farther away than the nearest traffic light may enter the detection area at the same time. Therefore, there is a possibility that information about a distant traffic light may be mistakenly recognized as information about a nearby traffic light. If such erroneous recognition occurs, it may cause the vehicle to start or stop incorrectly.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a traffic light recognition method and a traffic light recognition device that prevent the occurrence of false recognition of traffic lights and prevent false starts and stops of vehicles.

According to one aspect of the present invention, a traffic light recognition method for detecting a lighting state of a traffic light is provided. In this traffic light recognition method, a first imaging section mounted on the vehicle is used to capture an image of the front of the vehicle, and a second imaging section mounted on the vehicle and having a narrower field of view than the first imaging section is used to capture an image of the front of the vehicle. Take an image. Then, the traffic light in the first image captured by the first imaging unit is identified, the traffic light in the second image captured by the second imaging unit is identified, and the traffic light in the first image and the traffic light in the second image are identified. Determine whether the traffic light is the same as the traffic light. When there are multiple traffic lights determined to be the same, and when one of the multiple traffic lights disappears from the second image, the lighting state of the traffic light that disappeared from the second image is detected.

According to the traffic light recognition method of the present invention, the traffic light in the first image captured using the first imaging unit and the traffic light in the second image captured using the second imaging unit having a narrower field of view than the first imaging unit Determine whether the traffic light in the image is the same traffic light. If there are a plurality of traffic lights determined to be the same, and one of the traffic lights disappears from the second image, the lighting state of the traffic light that disappeared from the second image is detected. When the vehicle approaches the nearest traffic light, the nearest traffic light disappears from the second image captured using the second imaging unit, which has a narrower field of view than the first imaging unit. On the other hand, a traffic light located further away than the nearest traffic light does not disappear from the second image before the nearest traffic light. Therefore, by detecting the lighting state of the traffic light that disappeared from the second image, it is possible to detect the lighting state of the most recent traffic light from among the plurality of traffic lights. Thereby, it is possible to prevent the occurrence of erroneous recognition of traffic lights, and it is possible to prevent erroneous starting and stopping of the vehicle.

FIG. 1 is a block diagram showing the configuration of a traffic light recognition device according to a first embodiment. FIG. 2 is a diagram illustrating a method for detecting the nearest traffic light when the imaging range of the imaging unit and the distance between the traffic lights are short. FIG. 3 is a flowchart illustrating the traffic light recognition method of the traffic light recognition device according to the first embodiment. FIG. 4 is a flowchart illustrating the traffic light recognition method of the traffic light recognition device according to the second embodiment. FIG. 5 is a flowchart illustrating a traffic light recognition method of a traffic light recognition device according to the third embodiment. FIG. 6 is a flowchart illustrating the traffic light recognition method of the traffic light recognition device according to the fourth embodiment.

Embodiments of the present invention will be described below with reference to the drawings and the like.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of a traffic light recognition device 100 according to the first embodiment.

As shown in FIG. 1, the traffic light recognition device 100 according to the first embodiment includes a first imaging section 10, a first traffic light identification section 11, a second imaging section 20, a second traffic light identification section 21, and a map information acquisition section 30. , a self-position acquisition section 40, a same traffic light determination section 50, an appearance detection section 60, and a controller 70. The traffic light recognition device 100 is installed, for example, in a vehicle (not shown) having an automatic driving function, and detects the lighting state (current state) such as the lighting color of a traffic light in front of the vehicle. The signs detected by the traffic light recognition device 100 are used for automatic driving control and the like.

The first imaging unit 10 is, for example, a digital camera equipped with a solid-state imaging device such as a CCD or CMOS, and is mounted on the own vehicle (vehicle). The first image capturing unit 10 captures an image of the area in front of the travel path of the own vehicle, and obtains a digital image (first image) of the surrounding area in front of the vehicle. The digital image (first image) acquired by the first imaging section 10 is output to the first traffic light identification section 11 . Information regarding the installation position of the first imaging unit 10 (particularly the distance from the installation position of the second imaging unit 20 described later in the longitudinal, vehicle width, and height directions of the vehicle) is obtained by the first traffic light identification unit 11, the same traffic light determination unit 50, the present state detection section 60 and the controller 70. The information regarding the installation position of the first imaging unit 10 is used for determination of the same signal by the same traffic light determination unit 50, which will be described later.

The first imaging unit 10 has an angle of view A1, and has a digital image (digital image) in a wide-angle range that is relatively short in front distance from the own vehicle compared to the second imaging unit 20 described later. 1 image). Note that details regarding the imaging range of the first imaging unit 10 will be described later.

The second imaging unit 20 is, for example, a digital camera equipped with a solid-state imaging device such as a CCD or CMOS, and is mounted on a vehicle (own vehicle) like the first imaging unit 10. The second imaging unit 20 images the area in front of the travel path of the own vehicle, and obtains a digital image (second image) of the surrounding area in front of the vehicle. The digital image (second image) acquired by the second imaging section 20 is output to the second traffic light identification section 21 . Information regarding the installation position of the second imaging unit 20 (especially the distance from the installation position of the first imaging unit 10 in the longitudinal, vehicle width, and height directions of the vehicle) is obtained by the second traffic light identification unit 21, the same traffic light determination unit 50, It is stored in the appearance detection section 60 and the controller 70. The information regarding the installation position of the second imaging section 20 is used for the determination of the same signal by the same traffic light determination section 50, which will be described later.

The second imaging unit 20 has an angle of view A2 narrower than the angle of view A1 of the first imaging unit 10, and is capable of imaging a distance farther than the imaging distance of the first imaging unit 10. Note that details regarding the imaging range of the second imaging unit 20 will be described later.

The map information acquisition unit 30 acquires map information around the travel route on which the vehicle is traveling from a map database (not shown). This map information includes position information of traffic lights existing near the driving route, etc. The map information acquisition unit 30 outputs the acquired map information to the self-position acquisition unit 40, the first traffic light identification unit 11, and the second traffic light identification unit 21, respectively.

The self-position acquisition unit 40 includes a sensor 41 and a position detection unit 42. The sensor 41 is a detection device that detects surrounding information of the own vehicle, and is, for example, an on-vehicle camera, radar, or the like. The surrounding information detected by the sensor 41 includes, for example, white lines and stop lines on the road around the host vehicle, letters, signs around the road, and the like. The position detection unit 42 determines the current position of the own vehicle on the map and the attitude (yaw, yaw, pitch, roll) is detected as position information of the own vehicle. That is, the position detection unit 42 detects the position information of the own vehicle by comparing the surrounding information obtained by the sensor 41 and the map information input from the map information acquisition unit 30. The position information of the own vehicle is output to the first traffic light identification unit 11 and the second traffic light identification unit 21.

The first traffic light identification unit 11 and the second traffic light identification unit 21 set a traffic light detection region (ROI) in the first image and second image input from the first imaging unit 10 and the second imaging unit 20, respectively, and Identify traffic lights within the detection area. The traffic light detection area includes map information including the position information of the traffic light inputted from the map information acquisition unit 30 and position information of the own vehicle inputted from the own position acquisition unit 40 (the current position and posture of the own vehicle on the map). It is set based on. That is, the first traffic light identification unit 11 and the second traffic light identification unit 21 calculate areas where a traffic light may exist in the first image and the second image, respectively, from the map information and the position information of the own vehicle, The calculated area is set as a traffic light detection area. Further, the first traffic light identification unit 11 and the second traffic light identification unit 21 identify a traffic light from the set traffic light detection area. Traffic light identification methods include, for example, a method that detects when the signal light of a traffic light flashes in synchronization with the AC frequency of a commercial power supply, and a method that uses machine learning to recognize objects in images based on color information and shape characteristics such as round shapes. You can use a method to identify traffic lights by doing this.

The first traffic light identification unit 11 outputs the first image acquired from the first imaging unit 10 and the coordinates on the image of the traffic light identified from the first image to the same traffic light determination unit 50 . Similarly, the second traffic light identification unit 21 outputs the second image acquired from the second imaging unit 20 and the coordinates on the image of the traffic light identified from the second image to the same traffic light determination unit 50 .

The same traffic light determination unit 50 identifies the same traffic light among the traffic lights identified in the first image and the second image. As described above, the same traffic light determination unit 50 has the coordinates of the traffic light in the first image identified by the first traffic light identification unit 11 and the coordinates of the traffic light in the second image identified by the second traffic light identification unit 21. is input. The same traffic light determining unit 50 determines whether the first image and the second image are the same based on the coordinates of the traffic lights in the first image and the second image and the pre-stored information regarding the installation positions of the first imaging unit 10 and the second imaging unit 20. The same traffic lights are identified among the traffic lights identified in the process. Information on the same traffic light (same traffic light information) identified by the same traffic light determination unit 50 is output to the current status detection unit 60 . Note that details of the method for identifying the same traffic signal will be described later.

The display detection unit 60 detects the nearest traffic light in front of the host vehicle (hereinafter referred to as the nearest traffic light), and detects the lighting state (display) of the nearest traffic light, such as the lighting color. Note that the details of the method for identifying the most recent traffic light will be described later.

The controller 70 is composed of a general-purpose electronic circuit including a microcomputer, a microprocessor, and a CPU, and peripheral devices, and executes processing for controlling the traffic light recognition device 100 by executing a specific program. The controller 70 realizes processing in, for example, the traffic light identification units 11 and 21, the self-position acquisition unit 40, the same traffic light determination unit 50, and the appearance detection unit 60, and also controls the entire traffic light recognition device 100, such as traffic light recognition control described later. I do. Note that the controller 70 may also be used as an ECU used for other controls related to the vehicle (for example, automatic driving control, etc.).

FIG. 2 is a diagram illustrating the imaging range of the imaging units 10 and 20 and a method of detecting the nearest traffic light.

Figure 2 shows the distance between the own vehicle and the first traffic light when the distance l between the traffic lights is short between the nearest traffic light and the next closest traffic signal (next intersection traffic light) from the own vehicle. It shows the relationship between the inclination angle (pitch angle to the traffic light) and the traffic light. The vertical axis in FIG. 2 represents the pitch angle to the traffic light, and the horizontal axis represents the distance from the own vehicle to the nearest traffic light ahead. Note that the distance between the nearest traffic light and the next intersection traffic light is l, so if the distance between your vehicle and the nearest traffic light is L in Figure 2, the distance between your vehicle and the next intersection traffic light is becomes L+l.

As described above, the first imaging section 10 has a wider angle of view A1 than the second imaging section 20, and is relatively shorter in front distance from the own vehicle than the second imaging section 20. A first image of the range (L1 to L3) is acquired. The first image is determined by the first traffic light identification unit 11 to identify the traffic light up to the inclination angle (pitch angle to the traffic light) d1 between the vehicle and the traffic light in the range of the forward distance (predetermined distance) L1 to L3 from the vehicle. It is an identifiable image. That is, the first traffic light identification unit 11 can identify the traffic lights within the range indicated by the rectangle R1 surrounded by points A, B, C, and D in FIG. Preferably, the angle of view A1 is such that even if the own vehicle passes the stop line and enters the intersection, the nearest traffic light is a predetermined distance L1 to L3, and the traffic light has a general height H. This is an angle of view that falls within the range (R1) of the pitch angle d1. Further, the predetermined distance L3 is the nearest traffic light among the positions where the first imaging section 10 can obtain an image having a resolution or a size of the traffic light on the image that allows the first traffic light identification section 11 to identify the nearest traffic light. corresponds to the furthest distance from

Further, the next intersection traffic light is farther from the host vehicle by a distance l than the nearest traffic light. Therefore, the first imaging unit 10 is configured to move from L1', where the distance from the own vehicle to the nearest traffic light is shorter than L1 by a distance l, to L3', where the distance from the own vehicle to the nearest traffic light is shorter than L3 by a distance l. It is possible to image the next intersection traffic light within the range of . That is, the first traffic light identification unit 11 can identify the next intersection traffic light within the range indicated by the rectangle R1' surrounded by points A', B', C', and D' in FIG.

The second imaging unit 20 has an angle of view A2 narrower than the angle of view A1 of the first imaging unit 10, and has a second imaging unit that images a range (L1 to L4) farther than the imaging possible distance of the first imaging unit 10. Get the image. The second image is determined by the second traffic light identifying unit 21 to identify the traffic lights up to the inclination angle (pitch angle to the traffic light) d2 between the vehicle and the traffic light in the range of the forward distance (predetermined distance) L1 to L4 from the vehicle. It is an identifiable image. That is, the second traffic light identifying unit 21 can identify the traffic lights within the range indicated by the rectangle R2 surrounded by points A, F, G, and E in FIG. As shown in FIG. 2, the second traffic light identification unit 21 can identify far away traffic lights compared to the first traffic light identification unit 11; narrow in comparison. Note that the predetermined distance L4 is the nearest traffic light among the positions where the second imaging section 20 can obtain an image having a resolution or a size of the traffic light on the image that allows the second traffic light identification section 21 to identify the nearest traffic light. corresponds to the furthest distance from

Further, the next intersection traffic light is farther from the host vehicle by a distance l than the nearest traffic light. Therefore, the second image capturing unit 20 is configured to capture images from L1', where the distance from the own vehicle to the nearest traffic light is shorter than L1 by a distance l, to L4', where the distance from the own vehicle to the nearest traffic light is shorter than L4 by a distance l. It is possible to image the next intersection traffic light within the range of . That is, the second traffic light identification unit 21 can identify the next intersection traffic light within the range indicated by the rectangle R2' surrounded by points A', F', G', and E' in FIG.

If the distance from the own vehicle to the nearest traffic light is within the range of L2 or more and L3' or less in FIG. imaged by both. That is, the nearest traffic light and the next intersection traffic light exist in both the first image and the second image. At this time, if the distance l between the most recent traffic signal and the next intersection signal is short, both the most recent traffic signal and the next intersection signal will fall into the traffic light detection area, and the first traffic light identification unit 11 and the second traffic light identification unit Two traffic lights (the nearest traffic light and the next intersection traffic light) are identified by the section 21.

Next, when the own vehicle approaches the nearest traffic light to a position where the distance to the nearest traffic light becomes L2, the pitch angle to the nearest traffic light becomes larger than d2, and the nearest traffic light is imaged by the second imaging unit 20. Not done. On the other hand, at L2, since the pitch angle to the next intersection traffic light is smaller than d2, the next intersection traffic light is imaged by the second imaging unit 20. Furthermore, in L2, since the pitch angles to the nearest traffic light and the next intersection traffic light are both smaller than d1, both the nearest traffic light and the next intersection traffic light are imaged by the first imaging unit 10. In other words, when the own vehicle approaches the nearest traffic light to a distance L2 from the nearest traffic light, there are two traffic lights (the nearest traffic light and the next intersection traffic light) in the first image, but the most recent traffic light exists in the second image. The traffic light disappears, and only the next intersection traffic light exists in the second image. Therefore, when the host vehicle is located at a distance L2 to the nearest traffic light, and the distance l between the nearest traffic light and the next intersection traffic signal is short, the first traffic light identification unit 11 determines whether two traffic lights (the nearest traffic light and the next intersection traffic light) are identified, and only the next intersection traffic light is identified by the second traffic light identification unit 21.

FIG. 3 is a flowchart illustrating a traffic light recognition method (control) by the traffic light recognition device 100 in the first embodiment. In a traffic light recognition device, if there are multiple traffic lights in the captured image in front of the vehicle, and the distance between the traffic lights is short as in an urban area, the traffic lights further away than the nearest traffic light are simultaneously detected in the traffic light detection area (ROI). ). In such a case, there is a possibility that information from a distant traffic light may be mistakenly recognized as information from a nearby traffic light, causing the vehicle to start or stop incorrectly. On the other hand, in this embodiment, the traffic light recognition method (control) described below is used to avoid problems such as the distance between the nearest traffic light and other traffic lights being short and multiple traffic lights entering the traffic light detection area. Even in such cases, it is possible to detect the appearance of the traffic light immediately ahead of the own vehicle. Therefore, erroneous recognition of traffic lights can be prevented. Note that the following controls are all executed by the controller 70 (including the traffic light identification units 11 and 21, the self-position acquisition unit 40, the same traffic light determination unit 50, and the current status detection unit 60) at regular intervals.

In step S<b>101 , the first traffic light identification unit 11 and the second traffic light identification unit 21 detect a traffic light detection area ( ROI) is set, and a traffic light is identified from the set traffic light detection area. As described above, the traffic light detection area is set by calculating the area where the traffic light may exist in the first image and the second image, respectively, from the map information and the position information of the own vehicle.

In step S102, the controller 70 determines whether there is a plurality of traffic lights identified by the second traffic light identification unit 21. If there is no traffic light identified by the second traffic light identification unit 21 or if there is only one traffic light identified by the second traffic light identification unit 21, traffic lights further away than the nearest traffic light may enter the traffic light detection area at the same time. This does not apply to situations where it is difficult to store. Therefore, in this case, the controller 70 ends the traffic light recognition control. On the other hand, in step S102, if there are a plurality of traffic lights identified by the second traffic light identification unit 21, the controller 70 executes the process of step S103.

In step S103, the controller 70 determines whether there is a plurality of traffic lights identified by the first traffic light identification unit 11. If there is no traffic light identified by the first traffic light identification unit 11 or if there is only one traffic light identified by the first traffic light identification unit 11, a traffic light further away than the nearest traffic light is simultaneously detected as a traffic light in the first image. This does not apply to situations in which it would enter the territory. Therefore, in this case, the controller 70 ends the traffic light recognition control. On the other hand, in step S103, if there are multiple traffic lights identified by the first traffic light identification unit 11, a situation where a traffic light further away than the nearest traffic light enters the traffic light detection area of the first image at the same time, that is, the own vehicle is This corresponds to the case where it is located in the range of L2 to L3' in 2. In this case, the controller 70 executes the process of step S104.

In step S104, the same traffic light determining unit 50 (controller 70) identifies the same traffic light among the multiple traffic lights identified by the first traffic light identifying unit 11 and the multiple traffic lights identified by the second traffic light identifying unit 21. do. Identification of the same traffic signal can be performed using, for example, the following equations (1) to (3). In equation (1), x1 _image and y1 _image are the coordinates of the traffic light in the first image, k1 _x and k1 _y are the effective pixel sizes in the horizontal and vertical directions of the first imaging unit 10, and c1 _x and c1 _y are The image center of the first image, f1 is the focal length of the first imaging section 10, and X1, Y1, and Z1 are the distances from the first imaging section 10 to the traffic light in the left-right, up-down, and front-back directions in real space. Furthermore, in equation (2), x2 _image and y2 _image are the coordinates of the traffic light in the second image, k2 _x and k2 _y are the effective pixel sizes in the horizontal and vertical directions of the second imaging unit 20, and c2 _x and c2 _y is the image center of the second image, f2 is the focal length of the second imaging section 20, and X2, Y2, and Z2 are the distances from the second imaging section 20 to the traffic light in the left-right, up-down, and front-back directions in real space. L in equation (3) is the distance between the first imaging section 10 and the second imaging section 20 in real space. Note that in this embodiment, the installation positions of the first imaging section 10 and the second imaging section 20 differ only in the X direction (vehicle width direction), and are the same in the Y (front and back) and Z (height) directions; The installation positions of the first imaging section 10 and the second imaging section 20 are not limited to this. For example, they may be set at different positions in the Z direction.

When the first imaging unit 10 and the second imaging unit 20 face the traffic light, the traffic light identified by the first traffic light identification unit 11 and the traffic light identified by the second traffic light identification unit 21 are the same. In this case, equation (3) holds true. That is, the same traffic light determination unit 50 can identify the same traffic light based on equations (1) to (3).

In step S104, after identifying the correspondence between the traffic light identified by the first traffic light identification unit 11 and the traffic light identified by the second traffic light identification unit 21 (same traffic light), the controller 70 executes the process of step S105.

In step S105, the appearance detection unit 60 (controller 70) determines whether there is a traffic light that has disappeared from the second image, among the same traffic lights associated (identified) in step S104. If there is no traffic light that has disappeared from the second image, the host vehicle is located at a distance from the nearest traffic light that is greater than L2. In this case, the controller 70 repeats the process of step S105 until one traffic light disappears from the second image, that is, until the host vehicle reaches a position where the distance from the nearest traffic light is L2. On the other hand, if there is a traffic light that has disappeared from the second image, that is, if the host vehicle reaches a position where the distance from the nearest traffic light is L2, the controller 70 executes the process of step S106.

In step S106, the appearance detection unit 60 (controller 70) detects the appearance of the traffic light that has disappeared from the second image. As described above, when the distance from the own vehicle to the nearest traffic light becomes L2, the pitch angle to the nearest traffic light becomes larger than d2 and smaller than d1 (see FIG. 2). Therefore, when the host vehicle reaches a position where the distance to the nearest traffic light is L2, the nearest traffic light exists in the first image but disappears from the second image. On the other hand, in L2 of FIG. 2, a traffic light located further away than the nearest traffic light (next intersection traffic light) has a pitch angle smaller than d2, so it does not disappear from the second image. Therefore, the appearance detection unit 60 detects the appearance of the most recent traffic light by detecting the appearance of the same traffic light that disappeared from the second image among the traffic lights identified in the first image. can do. In other words, even if the distance l between the most recent traffic light and the next intersection signal is short and multiple traffic lights enter the traffic light detection area of the first image at the same time, the traffic light recognition device 100 detects the appearance of the most recent traffic light. can do. Upon detecting the disappearance of the traffic light in the second image, the controller 70 ends the traffic light recognition control.

With the traffic light recognition control described above, the traffic light recognition device 100 of this embodiment can be used even when the distance l between the nearest traffic light and a traffic light further away is short and multiple traffic lights enter the traffic light detection area at the same time. , it is possible to identify the nearest traffic light and detect the appearance of the nearest traffic light.

According to the traffic light recognition device 100 of the first embodiment described above, the following effects can be obtained.

In the traffic light recognition device 100, the traffic light in the first image captured using the first imaging unit 10 and the second traffic light captured using the second imaging unit 20 having a narrower angle of view than the first imaging unit 10 are used. Determine whether the traffic light in the image is the same traffic light. If there are a plurality of traffic lights determined to be the same, when one of the traffic lights disappears from the second image, the lighting state (presentation) of the traffic light that disappears from the second image is detected. When the own vehicle approaches the nearest traffic light, the pitch angle to the nearest traffic light increases, and the nearest traffic light disappears from the second image captured using the second imaging unit 20. On the other hand, since the pitch angle to a traffic light further away than the nearest traffic light is smaller than the pitch angle to the nearest traffic light, the farthest traffic light does not disappear from the second image before the nearest traffic light. Therefore, by detecting the lighting state (presentation) of the traffic light that has disappeared from the second image, it is possible to detect the lighting state (presentation) of the most recent traffic light from among the plurality of traffic lights. Thereby, it is possible to prevent the occurrence of erroneous recognition of traffic lights, and it is possible to prevent erroneous starting and stopping of the vehicle.

In this embodiment, the information regarding the installation positions of the first imaging section 10 and the second imaging section 20 is obtained from the first traffic light identification section 11, the second traffic light identification section 21, the same traffic light determination section 50, and the appearance detection section 60. and the configuration is stored in the controller 70, but the configuration is not necessarily limited to this. For example, a calibration mark or the like may be installed at a known position with respect to the host vehicle, and the installation position may be calculated from the position on the images captured by the first and second imaging units 10 and 20. In this case, the information regarding the calculated installation position is output to the first and second traffic light identification units 11 and 21, the same traffic light determination unit 50, the appearance detection unit 60, and the controller 70 together with the first and second images. .

(Second embodiment)
A traffic light recognition device 100 according to a second embodiment will be described with reference to FIG. 4. Note that the same elements as in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

FIG. 4 is a flowchart illustrating a traffic light recognition method (control) by the traffic light recognition device 100 in the second embodiment. In the second embodiment, while one of the plurality of traffic lights has not disappeared from the second image, the nearest traffic light is identified based on the parallax between the first imaging unit 10 and the second imaging unit 20. This differs from the first embodiment in this point. Note that in the second embodiment, the first imaging section 10 and the second imaging section 20 are mounted at different positions in the host vehicle.

Since steps S101 to S105 are the same as those in the first embodiment, their explanation will be omitted.

If there is a traffic light that has disappeared from the second image in step S105, the appearance detection unit 60 (controller 70) detects the appearance of the traffic light that has disappeared from the second image in step S106. Thereby, the appearance detection unit 60 (controller 70) can detect the latest appearance of the traffic light. Upon detecting the disappearance of the traffic light in the second image, the controller 70 ends the traffic light recognition control.

On the other hand, in step S105, if there is no traffic light that has disappeared from the second image, that is, if the own vehicle has not reached the position where the distance from the nearest traffic light is L2, the controller 70 executes the process of step S207. do. In step S207, the appearance detection unit 60 (controller 70) calculates the distance from the own vehicle to each traffic light based on the parallax between the first imaging unit 10 (first image) and the second imaging unit 20 (second image). calculate. Since the appearance detection unit 60 stores information regarding the installation positions of the first imaging unit 10 and the second imaging unit 20, the installation positions of the first imaging unit 10 and the second imaging unit 20, and the first image Parallax can be determined based on the second image and the second image. Note that the parallax may be determined by any known method. Based on this parallax, the appearance detection unit 60 calculates the distance from the own vehicle to each traffic light associated (specified) in step S104. After calculating the distance from the host vehicle to each traffic light, the controller 70 executes the process of step S208.

In step S208, the appearance detection unit 60 (controller 70) detects the appearance of the traffic light whose distance from the own vehicle calculated in step S207 is the shortest among the traffic lights specified in the first image. As a result, when the distance between the nearest traffic light and other traffic lights is short and multiple traffic lights enter the traffic light detection area at the same time, the vehicle reaches a position where the distance from the nearest traffic light is L2. It is possible to detect the latest traffic lights even when the vehicle is not in use.

According to the traffic light recognition device 100 of the second embodiment described above, the following effects can be obtained.

In the traffic light recognition device 100, the first imaging unit 10 and the second imaging unit 20, which has a narrower field of view than the first imaging unit 10, are mounted at different positions within the own vehicle (vehicle) 80. Further, if there are a plurality of traffic lights determined to be the same in the first image captured by the first imaging unit 10 and the second image captured by the second imaging unit 20, the first image capturing unit 10 and the second image capturing unit 20 The distance of each traffic light from the own vehicle (vehicle) 80 is calculated based on the parallax with the section 20. Then, while one of the plurality of traffic lights determined to be the same does not disappear from the second image, the lighting state (current state) of the traffic light with the shortest calculated distance is detected. As a result, when the distance between the nearest traffic light and other traffic lights is short and multiple traffic lights enter the traffic light detection area at the same time, the position where the nearest traffic light disappears from the second image (L2) can be adjusted. It is possible to detect the appearance of the latest traffic light even before the vehicle has arrived. Therefore, it is possible to more reliably prevent the occurrence of erroneous recognition of traffic lights, and it is possible to prevent erroneous starting and stopping of the vehicle.

(Third embodiment)
A traffic light recognition device 100 according to a third embodiment will be described with reference to FIG. 5. Note that the same elements as in other embodiments are denoted by the same reference numerals, and the explanation thereof will be omitted.

FIG. 5 is a flowchart illustrating a traffic light recognition method (control) by the traffic light recognition device 100 in the third embodiment. In the third embodiment, while one of the plurality of traffic lights does not disappear from the second image, the point of detecting the appearance of the traffic light identified at the highest position in the first image is the first point. This is different from the first and second embodiments.

Steps S101 to S105 are the same as those in the first and second embodiments, so their description will be omitted.

If there is a traffic light that has disappeared from the second image in step S105, the appearance detection unit 60 (controller 70) detects the appearance of the traffic light that has disappeared from the second image in step S106. Thereby, the appearance detection unit 60 (controller 70) can detect the latest appearance of the traffic light. Upon detecting the disappearance of the traffic light in the second image, the controller 70 ends the traffic light recognition control.

On the other hand, in step S105, if there is no traffic light that has disappeared from the second image, that is, if the own vehicle has not reached the position where the distance from the nearest traffic light is L2, the controller 70 executes the process of step S307. do. In step S307, the appearance detection unit 60 (controller 70) detects the appearance of the traffic light located at the highest position in the image among the traffic lights specified in the first image. The closer the host vehicle gets to the traffic light, the larger the pitch angle to the traffic light becomes, and the traffic light is specified at a higher position in the image. Therefore, the closer the distance to the host vehicle is, the higher the position in the image is specified. Therefore, by detecting the appearance of the traffic light specified at the highest position in the first image, can be detected. In other words, if the distance between the nearest traffic light and other traffic lights is short and multiple traffic lights enter the traffic light detection area at the same time, the vehicle may arrive at a position where the distance from the nearest traffic light is L2. It is possible to detect the latest traffic light display even when the vehicle is not in use.

Note that in this embodiment, in step S307, the appearance of the traffic light located at the highest position in the image among the traffic lights identified in the first image is detected; Among the traffic lights, the display of the traffic light having the highest position in the image may be detected.

According to the traffic light recognition device 100 of the third embodiment described above, the following effects can be obtained.

In the traffic light recognition device 100, the first image capturing unit 10 captures a first image, and the second image capturing unit 20, which has a narrower angle of view than the first image capturing unit 10, captures a second image. If there are multiple traffic lights determined to be the same in the first image and the second image, as long as one of the multiple traffic lights determined to be the same does not disappear from the second image, the first Detects the lighting status (presentation) of the traffic light located at the highest position in the image. As a result, when the distance between the nearest traffic light and other traffic lights is short and multiple traffic lights enter the traffic light detection area at the same time, the position where the nearest traffic light disappears from the second image (L2) can be adjusted. It is possible to detect the appearance of the latest traffic light even before the vehicle has arrived. Therefore, it is possible to more reliably prevent the occurrence of erroneous recognition of traffic lights, and it is possible to prevent erroneous starting and stopping of the vehicle.

Further, since the appearance of the latest traffic light is detected by a simple method without using parallax, complicated calculations are not required, and the device can be simplified.

(Fourth embodiment)
A traffic light recognition device 100 according to a fourth embodiment will be described with reference to FIG. 6. Note that the same elements as in other embodiments are denoted by the same reference numerals, and the explanation thereof will be omitted.

FIG. 6 is a flowchart illustrating a traffic light recognition method (control) by the traffic light recognition device 100 in the fourth embodiment. The fourth embodiment differs from the other embodiments in that while there is no traffic light in the first image, the appearance of a traffic light specified at the highest position in the second image is detected.

Steps S101 to S103 are the same as in other embodiments, so their explanation will be omitted.

In step S103, if a plurality of traffic lights are not identified by the first traffic light identification unit 11 (there is no identified traffic light or only one traffic light is identified), the controller 70 executes the process of step S407. Execute. On the other hand, if there are multiple traffic lights identified by the first traffic light identifying unit 11, the controller 70 executes the process of step S104.

In step S407, the controller 70 determines whether there is a traffic light identified by the first traffic light identification unit 11. When there is a traffic light identified by the first traffic light identification unit 11, that is, when one traffic light is identified by the first traffic light identification unit 11, the controller 70 executes the process of step S408.

In step S407, if the first traffic light identification unit 11 determines that one traffic light has been identified, the host vehicle is located within a distance of L3 to L3' from the nearest traffic light (see FIG. 2). . In the range from L3 to L3', only the most recent traffic light exists in the first image, so the traffic light identified by the first traffic light identification unit 11 is the most recent traffic light. Therefore, the appearance detection unit 60 (controller 70) detects the appearance of the traffic light specified in the first image in step S408. This makes it possible to detect the latest traffic light display. In step S408, upon detecting the appearance of the traffic light specified in the first image, the controller 70 ends the traffic light recognition control.

If it is determined in step S407 that there is no traffic light identified by the first traffic light identification unit 11, the controller 70 executes the process of step S409.

In step S409, if it is determined by the first traffic light identification unit 11 that there is no traffic light identified, the host vehicle is located within a distance of L3 to L4' from the nearest traffic light (see FIG. 2). In the range from L3 to L4', the first imaging unit 10 cannot image the nearest traffic light, and there is no traffic light in the first image. Therefore, in step S409, the appearance detection unit 60 (controller 70) detects the appearance of the traffic light located at the highest position in the image among the traffic lights specified in the second image. As described above, the closer the host vehicle gets to the traffic light, the larger the pitch angle to the traffic light becomes, and the traffic light is specified at a higher position in the image. Therefore, the closer the distance to the host vehicle is, the higher the position in the image is specified. Therefore, by detecting the appearance of the traffic light specified at the highest position in the second image, can be detected. In other words, when the distance between the nearest traffic light and other traffic lights is short, even if the traffic light is far away (at a distance where the first imaging unit 10 cannot image the nearest traffic light), the display of the nearest traffic light is not displayed. can be detected.

In step S409, when the controller 70 detects the appearance of the traffic light with the highest position in the image among the traffic lights specified in the second image, the controller 70 ends the traffic light recognition control.

If it is determined in step S103 that there are a plurality of traffic lights identified by the first traffic light identification unit 11, the controller 70 executes the processes of steps S104 to S106, and ends the traffic light recognition control. Note that the processing in steps S104 to S106 is the same as that in the first embodiment, so a description thereof will be omitted.

According to the traffic light recognition device 100 of the fourth embodiment described above, the following effects can be obtained.

In the traffic light recognition device 100, the first image is captured by the first image capturing unit 10, and the second image capturing unit has a narrower angle of view than the first image capturing unit 10 and is capable of capturing an image farther than the imaging distance of the first image capturing unit 10. 20 captures a second image. If there is no specified traffic light in the first image, the lighting state of the traffic light at the highest position in the second image is detected until the traffic light appears in the first image. As a result, if the distance between the nearest traffic light and other traffic lights is short and multiple traffic lights enter the traffic light detection area (in the second image) at the same time, there is no traffic light identified in the first image. In the meantime, it is possible to detect the latest traffic light appearance. Therefore, it is possible to more reliably prevent the occurrence of erroneous recognition of traffic lights, and it is possible to prevent erroneous starting and stopping of the vehicle.

Although the embodiments of the present invention have been described above, the above embodiments merely show a part of the application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments. do not have.

Furthermore, although each of the embodiments described above has been described as a single embodiment, they may be combined as appropriate.

10 First imaging section 11 First traffic light identification section 20 Second imaging section 21 Second traffic light identification section 30 Map information acquisition section 40 Self-position acquisition section 50 Same traffic light determination section 60 Appearance detection section 70 Controller 100 Traffic light recognition device

Claims (5)

A traffic light recognition method for detecting a lighting state of a traffic light, the method comprising:
capturing an image in front of the vehicle using a first imaging unit mounted on the vehicle;
Imaging the front of the vehicle using a second imaging unit that is mounted on the vehicle and has a narrower field of view than the first imaging unit;
identifying a traffic light in a first image captured by the first imaging unit;
identifying a traffic light in a second image captured by the second imaging unit;
determining whether the traffic light in the first image and the traffic light in the second image are the same traffic light;
When there are multiple traffic lights determined to be the same, when one of the multiple traffic lights disappears from the second image, detecting the lighting state of only the traffic light that disappeared from the second image.
A traffic light recognition method characterized by: The first imaging unit and the second imaging unit are mounted at different positions within the vehicle,
If there are multiple traffic lights that are determined to be the same, calculate the distance of each traffic light from the vehicle based on the parallax between the first imaging unit and the second imaging unit,
While one traffic light among the plurality of traffic lights determined to be the same does not disappear from the second image, detecting the lighting state of the traffic light with the shortest calculated distance;
The traffic light recognition method according to claim 1, characterized in that: If there are multiple traffic lights determined to be the same, one of the traffic lights determined to be the same remains at the highest position in the first image until it disappears from the second image. Detects the lighting status of traffic lights,
The traffic light recognition method according to claim 1, characterized in that: If there is no traffic light specified in the first image, until the traffic light appears in the first image, detecting the lighting state of the traffic light at the highest position in the second image.
The traffic light recognition method according to any one of claims 1 to 3, characterized in that: a first imaging unit that is mounted on a vehicle and captures an image in front of the vehicle;
a second imaging unit that is mounted on the vehicle, has a narrower field of view than the first imaging unit, and is capable of capturing an image in front of the vehicle and farther than the imaging distance of the first imaging unit;
a first traffic light identification unit that identifies a traffic light in a first image captured by the first imaging unit;
a second traffic light identification unit that identifies a traffic light in a second image captured by the second imaging unit;
a same traffic light determination unit that determines whether the traffic light identified by the first traffic light identification unit and the traffic light identified by the second traffic light identification unit are the same traffic light;
an indication detection unit that detects a lighting state of a traffic light closest to the vehicle ;
A traffic light recognition device comprising:
When there are a plurality of traffic lights determined to be the same by the same traffic light determination unit, when one traffic light among the plurality of traffic lights disappears from the second image, the appearance detection unit detecting the lighting state of the disappeared traffic light as the nearest traffic light ;
Traffic light recognition device.

Images