JP5056679B2 - Night view system - Google Patents

Description

  The present invention relates to a night view system that detects a pedestrian from a captured image and displays the pedestrian in an emphasized manner on a display device or the like.

  A night view system that alerts the driver to the presence of a pedestrian walking in front of the vehicle is known. The night view system is a system for notifying a driver of the presence of a pedestrian by detecting a pedestrian from a dense state of brightness values and an aspect ratio of the captured infrared image and highlighting the pedestrian on the infrared image.

  In the night view system, it has been conventionally known that an artificial object may be erroneously detected as a pedestrian, and a technique for suppressing erroneous detection has been proposed (for example, see Patent Documents 1 to 3). Patent Document 1 discloses a pedestrian detection device that determines the similarity of a plurality of pedestrian candidates and determines that a pedestrian candidate with a similarity equal to or greater than a predetermined value is not a pedestrian, thereby suppressing false detection of an artifact. It is disclosed. Since artifacts such as guardrails and utility poles have substantially the same shape, a series of pedestrian candidates can be determined not to be pedestrians at the same time based on the degree of similarity.

  Further, in Patent Document 2, a vehicle is identified by detecting another lamp of a lamp having the same distance as the lamp detected from the infrared image and comparing the amount of infrared of the other lamp with a threshold value. A night view system is disclosed. The detection accuracy of a pedestrian can be improved by detecting a tail lamp (lamp) of a vehicle that is easy to mix with the pedestrian from the height and shape.

Patent Document 3 discloses a periphery monitoring device that calculates a movement vector of an object from an infrared image and identifies and excludes features and other vehicles based on the movement vector.
JP 2008-123113 A JP 2006-176020 A JP 2006-153776 A

  However, calculating the similarity of a plurality of pedestrian candidates as in the pedestrian detection device described in Patent Document 1 is not necessarily suitable for a night view system that requires a large processing load and requires real-time performance. In addition, in the method of detecting another lamp having the same distance as in the night view system described in Patent Document 2, when a pedestrian is present at the same distance as the lamp, the pedestrian is excluded as a lamp. There is a fear. In addition, the periphery monitoring device described in Patent Document 3 does not describe how to specifically identify a feature or another vehicle using a movement vector of an object.

  In view of the above problems, the present invention has an object to provide a night view system that is hardly affected by the distance between a pedestrian and an artifact and that can suppress the increase in processing load and can identify the artifact and the pedestrian. .

In view of the above problems, the present invention detects a pedestrian from captured image data and alerts the driver of the presence of the pedestrian in the night view system. Pedestrian candidate detection means for detecting a candidate and the same pedestrian candidate in the image data for each cycle time are monitored, and when the amount of change in the position and size of the pedestrian candidate is within a threshold, the pedestrian candidate is And selecting means for determining that it is not.

It is possible to provide a night view system that is hardly affected by the distance between a pedestrian and an artificial object, and that can suppress an increase in processing load and can identify the artificial object and the pedestrian.

The best mode for carrying out the present invention will be described below with reference to the drawings.
FIG. 1 is an example of a diagram schematically illustrating detection of a pedestrian 13 by the night view system 100 of the present embodiment. As shown to Fig.1 (a), the front vehicle 11 and the pedestrian 13 exist in front of the own vehicle, and the front vehicle 11 and the pedestrian 13 are image | photographed by the infrared image. The night view system (hereinafter referred to as the “NV system”) 100 detects pedestrian candidates by image processing of infrared images. The tail lamp 12 of the front vehicle 11, particularly the vertical tail lamp 12, is similar in appearance to the pedestrian 13. Therefore, pedestrian candidates a1 to a3 are detected. Here, the pedestrian candidates a1 to a3 are indicated by rectangular areas.

  Since the NV system 100 captures an infrared image every cycle time, it captures infrared images including the same pedestrian candidates a1 to a3 one after another as shown in FIG. However, while the preceding vehicle 11 is traveling at the same vehicle speed as the own vehicle, the moving speed of the pedestrian 13 is slower than the own vehicle, so that the pedestrian candidates a1 and a2 are at the same position and size. In contrast, the position of the pedestrian candidate a3 approaches the host vehicle and its size gradually increases.

  The NV system 100 of the present embodiment uses such a relationship in the pedestrian candidates a1 to a3 to eliminate the tail lamp 12 of the forward vehicle 11 and reduce false detection. Since it is only necessary to monitor the size and position of the pedestrian candidates a1 to a3, an increase in processing load can be suppressed, and even if the pedestrian 13 and the preceding vehicle 11 are at the same level, the own vehicle moves so that the tail lamp The pedestrian 13 can be detected without erroneously detecting 12.

  FIG. 2 shows an example of a block diagram of the NV system 100 of the present embodiment. The NV system 100 is controlled by an image processing ECU (electronic control unit) 22, and a camera 21 and a display unit 23 are connected to the image processing ECU 22 via a CAN (Controller Area Network) or a dedicated line.

  The camera 21 mounts a photoelectric conversion element having sensitivity to near-infrared light (for example, 800 to 2000 [μm]), and captures an infrared image every predetermined cycle time (for example, 15 to 30 frames / second). The infrared image has a luminance value (about 0 to 255 levels) for each pixel, and the whole is displayed with the shading. Therefore, an object that reflects near-infrared light, such as a pedestrian 13, can be suitably photographed. The detection range of the pedestrian 13 is about 30 to 100 [m], and the farther the pedestrian 13 is detected as the vehicle speed of the host vehicle increases. The camera 21 is attached to the back side of the rearview mirror, for example, with the optical axis slightly downward in front of the vehicle.

  For example, a near-infrared projector is arranged in the headlight, and when the NV system 100 is turned on, the high beam and the near-infrared projector are switched, and the visible light cut filter is activated, so that the light of the headlamp and the near-infrared projector are activated. Near-infrared light is preferentially irradiated in front of the host vehicle. Thereby, detection of the pedestrian 13 at night becomes easy.

  As the display unit 23, for example, an FPD (flat panel display) such as a liquid crystal display or an organic EL display, or a HUD (Head Up Display) is used. In the case of FPD, for example, the meter display portion of the liquid crystal display changes on the entire surface, and an infrared image is displayed at the center of the meters. Further, it may also be used as the display unit 23 of an AV system such as a navigation system or a television. On the display unit 23, the detected pedestrian 13 is displayed with emphasis by a rectangular frame or the like.

  The image processing ECU 22 is a computer including a CPU, a RAM, a ROM, an input / output interface, an ASIC (Application Specific Integrated Circuit), and the like. The CPU executes a program stored in the ROM, or the pedestrian candidate detection unit 24, the selection unit 25, and the pedestrian enhancement unit 26 are realized by ASIC.

  The pedestrian candidate detection unit 24 detects a pedestrian candidate that is estimated to include the pedestrian 13 according to a procedure described later from an infrared image captured at each cycle time. The selection unit 25 monitors changes in the position and size of pedestrian candidates, selects pedestrian candidates that do not include the pedestrian 13, and detects the pedestrian 13. The pedestrian emphasizing unit 26 surrounds the detected pedestrian 13 with a rectangular frame, for example, and emphasizes and displays it on the display unit 23.

  FIG. 3 is an example of a flowchart illustrating a procedure in which the pedestrian candidate detection unit 24 detects the pedestrian 13. The pedestrian candidate detection unit 24 repeats the process of FIG. 3 for the infrared image captured every cycle time. The flowchart of FIG. 3 shows that the startup switch of the NV system 100 is ON, the illuminance sensor detects that it is nighttime, the headlight is on, and the vehicle speed is below a predetermined value (for example, 60 [km / h ])

  The pedestrian candidate detection unit 24 detects a pedestrian candidate that may be the pedestrian 13 from the infrared image (S10). Since the pedestrian 13 reflects near-infrared light, the pixel on which the pedestrian 13 is photographed has high luminance. Further, since the pedestrian 13 stands in the vertical direction with a certain degree of extension, it is considered that the high-luminance pixels are continuous in a vertical direction at a predetermined value or more. Based on such a concept, the pedestrian candidate detection unit 24 detects a series of pixels having a luminance equal to or higher than a predetermined value and continuous for a predetermined value or more in the vertical direction. A series of pixels or a rectangular area of a predetermined size surrounding the series of pixels is a pedestrian candidate.

  FIG. 4 is an example of a diagram schematically showing detected pedestrian candidates. Since the pedestrian candidate detected at this time is a rough detection, for example, a tail lamp 12, a signboard, a utility pole, a guardrail, a headlamp, a white line, and the like are included in addition to the pedestrian. Among these, since the position of the white line is detected by the white line recognition logic, it is excluded from pedestrian candidates in advance.

  Next, the pedestrian candidate detection unit 24 excludes non-pedestrians from the pedestrian candidates. First, an obvious rectangular object is excluded from pedestrian candidates. An obvious rectangular object is a signboard, a utility pole, or the like. When a pixel value having a high luminance value occupies the four corner areas of a circumscribed rectangle of a series of pixels or its periphery, it is determined to be a rectangular object. This eliminates signboards and utility poles.

  The pedestrian candidate detection unit 24 excludes continuous objects such as guardrails and curbs from pedestrian candidates. When such a continuous object is photographed, since a series of pixels are continuous from the lower side to the upper side, the left side to the upper side, or the right side to the upper side of the infrared image, the pedestrian candidate detection unit 24 includes a series of pixels. Is excluded from pedestrian candidates. This eliminates guardrails and utility poles.

  In addition, the pedestrian candidate detection unit 24 excludes a series of pixels with saturated pixel values from the pedestrian candidates. The saturation of the pixel value means that the luminance value becomes the maximum value (255 or its vicinity) over a certain area because the intensity of light incident on the camera 21 exceeds the resolution of the luminance value. Such a phenomenon occurs when a light source or its reflector is photographed, and the pedestrian 13 rarely shows such a high luminance value. Therefore, the pedestrian candidate detection unit 24 excludes pedestrian candidates whose pixel values having the highest luminance value are a predetermined ratio or more from the pedestrian candidates. Thereby, it can be expected that the headlamp is eliminated and the tail lamp 12 is also eliminated.

  Next, the pedestrian candidate detection unit 24 detects the pedestrian 13 by pattern matching, for example, from the remaining pedestrian candidates (S30). First, the pedestrian candidate detection unit 24 determines whether or not the aspect ratio of the circumscribed rectangle of a series of pixels falls within a predetermined ratio. Moreover, the pedestrian candidate detection part 24 pattern-matches a series of pixels, for example using a standard template. The standard template is a figure that characterizes the human head, shoulders, torso, and legs, for example (the head and legs are smaller than the shoulders, the legs are smaller than the heads, etc.) Is a binary image. The pedestrian candidate detection unit 24 binarizes the circumscribed rectangle of the series of pixels and calculates the degree of coincidence with the standard template. If the degree of coincidence is equal to or greater than a predetermined value, the pedestrian candidate is left as it is. Note that the predetermined value for determining the degree of coincidence is set to a low value in order to prevent erroneous detection of the pedestrian 13 as not being a pedestrian. For this reason, it may occur that an artificial object is erroneously detected as a pedestrian.

  By the way, although pattern matching is performed first in FIG. 3, you may perform selection of the pedestrian candidate of step S40 prior to pattern matching. After selecting pedestrian candidates, the processing load of pattern matching can be reduced.

  Even if non-pedestrians are excluded as described above, in the case of the tail lamp 12, the illuminance is lower than that of the headlamp, and depending on the angle of light incident on the camera 21, it is not determined to be saturated, and pedestrian candidates May remain as. In particular, when the tail lamps 12 are arranged in the vertical direction, the circumscribed rectangle is similar to the pedestrian 13 and the circular lamp is similar to the head of the pedestrian. There is also a risk of erroneous detection as a pedestrian 13. Therefore, in the NV system 100 of the present embodiment, the position and size of pedestrian candidates are monitored to further select pedestrian candidates (S40).

  FIG. 5 is an example of a flowchart showing in detail the procedure of step S40 of FIG. The selection unit 25 detects the position and size of the pedestrian candidate (S110). When there are a plurality of pedestrian candidates, an identification number is assigned to each pedestrian candidate.

  FIG. 6 shows an example of a pedestrian candidate table managed by the selection unit 25. The position and the size of the circumscribed rectangle are recorded in association with the identification number of the pedestrian candidate. Among these, the positions are the four corner coordinates (upper left, upper right, upper right, lower left) of the circumscribed rectangle of the pedestrian candidate in the infrared image. The size is indicated by the number of pixels (number of pixels) in the vertical and horizontal dimensions of the circumscribed rectangle. If the number of vertical pixels is known, the approximate size (height) of the pedestrian 13 can be known. Since the focal length of the lens of the camera 21, the pixel pitch, and the distance from the photoelectric conversion element to the lens are known, the distance between the lens and the pedestrian candidate can be calculated from the ratio relationship generated therebetween.

  And the pedestrian candidate detection part 24 performs the procedure to step S30 of FIG. 3 with respect to the following infrared image. Thereby, a pedestrian candidate is detected also in the next infrared image. The selecting unit 25 determines whether any of the four corner coordinates of the circumscribed rectangle of the pedestrian candidate is included within a predetermined distance from the four corner coordinates of the circumscribed rectangle registered in the pedestrian candidate table. Identify the same pedestrian candidate. Similarly, the position of the pedestrian candidate and the size of the circumscribed rectangle are detected, and the position and size of each pedestrian candidate are recorded in time series in the pedestrian candidate table.

  The selection unit 25 compares the position and size of the same pedestrian candidate detected in time series (S120). In this embodiment, it is only necessary to detect a pedestrian candidate that moves at a speed similar to that of the host vehicle, such as the tail lamp 12 of the preceding vehicle 11. For example, the movement amount V of one point of a pedestrian candidate, such as one of the four corner coordinates or the diagonal intersection of the circumscribed rectangle, the average of the movement amount of the four corner coordinates, or the movement amount of the four corner coordinates and the circumscription The average amount of movement at the intersection of the rectangles is calculated.

  Further, the sorting unit 25 compares the vertical w and the horizontal d for the same pedestrian candidate detected in time series, and detects the vertical change amount Wd and the horizontal change amount Dd.

  Although the movement amount V, the vertical change amount Wd, and the horizontal change amount Dd can be detected every time an infrared image is taken, a desired change can be expected when the host vehicle moves. In particular, when traveling at a low speed, a significant difference cannot be obtained even if the movement amount V, the longitudinal change amount Wd, and the lateral change amount Dd for each cycle time are detected.

  For this reason, selection of a pedestrian candidate compares the infrared images image | photographed with the space | interval of 0.5 second-1 second, for example. Further, since the position and size detected from one infrared image are likely to contain errors, the average value of 5 to 10 frames of infrared images is used for the position and size. From the above, the selecting unit 25 detects the position and size from, for example, five frames of infrared images while monitoring the same pedestrian candidate, and moves the moving amount V, the vertical change amount Wd, and the horizontal amount every 0.5 seconds. A change amount Dd is detected. If the vehicle speed of the host vehicle is zero, the pedestrian 13 may be excluded, so the procedure of FIG. 5 is executed only when the vehicle speed is equal to or higher than a predetermined value.

Next, the selection unit 25 compares the determination threshold stored in advance with the movement amount V, the vertical change amount Wd, and the horizontal change amount Dd (S120). When the determination threshold values are Th1, Th2, and Th3, it is determined whether or not all of the following expressions are satisfied. Note that it may be determined whether one or more of the movement amount V, the vertical change amount Wd, and the horizontal change amount Dd satisfy the condition.
Travel distance V <Th1
Longitudinal change amount Wd <Th2
Lateral change amount Dd <Th3
Assuming that the preceding vehicle 11 is traveling at the same level as the host vehicle, the determination threshold values Th1 to Th3 need not be determined according to the vehicle speed, but the determination threshold values Th1 to Th3 are determined according to the vehicle speed and acceleration. It may be. For example, by determining the acceleration according to the acceleration of the host vehicle, whether the movement amount V, the vertical change amount Wd, and the lateral change amount Dd have changed in consideration of the acceleration / deceleration even if the host vehicle suddenly accelerated / decelerated. Can be determined. The acceleration / deceleration may be detected from a G sensor, or may be detected from a change in throttle opening, a change in master cylinder pressure, or the like.

  When the movement amount V <Th1, the vertical change amount Wd <Th2, and the horizontal change amount Dd <Th3 (Yes in S120), the selection unit 25 excludes the pedestrian candidate as a non-pedestrian (S130). In addition, when any of the movement amount V <Th1, the longitudinal change amount Wd <Th2, and the lateral change amount Dd <Th3 is not satisfied (No in S120), the pedestrian candidate is detected as the pedestrian 13.

  Next, the pedestrian emphasizing unit 26 detects the position of the pedestrian 13 that has not been excluded from the pedestrian candidate table or the infrared image, and highlights the pedestrian 13 by blinking, for example, a rectangular frame surrounding the pedestrian 13. Thereby, the driver | operator of the own vehicle can grasp | ascertain that the pedestrian 13 exists ahead. In addition, you may be alerted further by a buzzer sound.

  As described above, the NV system 100 according to the present embodiment can eliminate the tail lamp 12 of the preceding vehicle 11 that is likely to be erroneously detected as a pedestrian 13 by moving at the same speed as the own vehicle. it can. Since it is only necessary to compare the position and size of the circumscribed rectangle of the pedestrian candidate, an overload does not add processing load to the image processing ECU 22.

It is an example of the figure which explains typically detection of a pedestrian by a night view system. It is an example of the block diagram of a NV system. It is an example of the flowchart figure which shows the procedure in which a pedestrian candidate detection part detects a pedestrian. It is an example of the figure which shows the detected pedestrian candidate typically. FIG. 5 is an example of a flowchart showing in detail the procedure of step S30 of FIG. It is a figure which shows an example of the pedestrian candidate table which a selection part manages.

Explanation of symbols

11 Front vehicle 12 Tail lamp 13 Pedestrian 21 Camera 22 Image processing ECU
23 Display Unit 24 Pedestrian Candidate Detection Unit 25 Sorting Unit 26 Pedestrian Emphasis Unit 100 Night View System

Claims (3)

In the night view system that detects pedestrians from the captured image data and alerts the driver of the presence of pedestrians,
Pedestrian candidate detection means for detecting pedestrian candidates from image data photographed every cycle time;
The same pedestrian candidate in the image data for each cycle time is monitored, and when the amount of change in the position and size of the pedestrian candidate is within a threshold, the selection means for determining that the pedestrian candidate is not a pedestrian
The night view system characterized by having. The pedestrian candidate detecting means is
A rectangular area of a predetermined size surrounding a series of pixels that are continuous from a predetermined value in the vertical direction with a luminance of a predetermined value or more from image data is specified as a pedestrian candidate,
From the pedestrian area, a rectangular area occupied by pixel values of luminance values that can be regarded as four corner areas as signs and utility poles,
A series of continuous pixels from the lower side to the upper side, the left side to the upper side, or the right side to the upper side of the image data, and
After excluding the saturation region where the luminance value is greater than or equal to the predetermined value,
The selection means eliminates the tail lamp of the front vehicle by determining that the pedestrian candidate's circumscribed rectangle does not include a pedestrian when the amount of change in the position and size of the pedestrian candidate is within a threshold,
The night view system according to claim 1.   The night view system according to claim 1, wherein the threshold value is changed according to acceleration and deceleration of the host vehicle.

Images