JP2012142832A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus which captures with a single camera an image of an obstacle candidate and a landscape image and causes the landscape image and the obstacle candidate image overlap without displacement.SOLUTION: An infrared camera 10 comprises a far-infrared imaging means 11 and a near-infrared imaging means 12 and switches between a far-infrared image and a near-infrared image by imaging means switch means 18. Then, the obstacle candidate area is identified from the far-infrared image by a feature amount extraction part 152, and the obstacle candidate area is caused to overlap on the near-infrared image and displayed on a display monitor 50.

Description

  The present invention relates to an imaging device that is mounted on a vehicle and images an object in front of the vehicle, such as a pedestrian.

  In recent years, devices that assist the recognition of pedestrians when traveling at night are starting to be put into practical use. The following Patent Document 1 also discloses such a pedestrian recognition device. Such a system is called a night view system or a night vision system, and in many cases, an image of an infrared camera is superimposed on an image of a visible light camera to recognize a pedestrian.

JP 2006-338594 A

However, since the imaging device used in such a conventional night view system uses both a visible light camera and an infrared camera in combination, the system is often complicated and expensive, and the two cameras are driven. Therefore, there was a problem that power consumption was large.
Accordingly, an object of the present invention is to provide an imaging apparatus that is inexpensive and consumes low power by using a single camera.

(1) The imaging apparatus of the present invention
An imaging device that images the front of the vehicle, a near-infrared light source that irradiates near-infrared light in front of the vehicle, an image imaging unit that switches a captured image by the imaging device to a far-infrared image and a near-infrared image using a wavelength band filter, and Based on the far-infrared image, the obstacle candidate region specifying means for specifying the obstacle candidate region, the image superimposing processing unit for superimposing the obstacle candidate region on the near-infrared image, and the image superimposing processing unit are generated. And an image display unit for displaying a superimposed image.
According to the above configuration, it is possible to switch between the near-infrared image and the far-infrared image by using a single imaging unit.

(2) The imaging device of the present invention is characterized in that, in addition to the configuration of (1), the wavelength band filter is a wavelength tunable filter.
According to the above configuration, the wavelength tunable filter can be disposed at a fixed position with respect to the image sensor, so that the near-infrared image and the far-infrared image can have the same optical axis, and there is no image shift.

(3) The imaging device of the present invention
In addition to the configuration of at least (1), an alert symbol conversion unit that converts the obstacle candidate region into an alert symbol, and superimposing the alert symbol as the obstacle candidate region on the near-infrared image, To do.
According to the above configuration, since an obstacle is converted into an alert symbol, an image with high visibility can be generated.

(4) The imaging device of the present invention
In addition to at least the configuration (1), the near-infrared light source is driven based on the near-infrared image.
According to the said structure, a near-infrared light source can be light-emitted optimally without waste.

(5) The imaging device of the present invention
In addition to at least the configuration (1), the image sensor is a pyroelectric image sensor.
According to the said structure, it can image widely from near infrared rays to far infrared rays by using a pyroelectric image sensor.

(6) The imaging apparatus of the present invention
In addition to the configuration of (3), the obstacle candidate specifying unit includes a determining unit that determines a dynamic pedestrian and a static pedestrian, and converts only the dynamic pedestrian into the alert symbol. And
According to the said structure, the image which can alert a dangerous dynamic pedestrian can be produced | generated.

1 is a block diagram illustrating a configuration of an imaging apparatus according to the present invention. The figure of the near-infrared image acquired with the infrared camera of the present invention. The figure of the far-infrared image acquired with the infrared camera of the present invention. FIG. 3 is a flowchart showing the operation principle of the imaging apparatus of the present invention. The figure of the conversion image converted in the imaging device of this invention. The figure of the display image displayed on the display monitor of this invention. FIG. 3 is a flowchart of image processing performed in the imaging apparatus of the present invention.

  Embodiments of the present invention will be described below with reference to the drawings. A configuration diagram showing a configuration for notifying a driver of information using the imaging apparatus of the present invention is shown in FIG.

  The imaging apparatus 1 is mounted on a vehicle and includes an infrared camera 10 that captures the front of the vehicle, a near-infrared light source 20, a central processing unit (CPU) 30, and a display monitor 50. The infrared camera 10 is provided with at least a wavelength variable filter 101 and an image sensor 102. By changing the transmission wavelength band of the wavelength tunable filter 101, it is possible to appropriately transmit only the far infrared rays or transmit only the near infrared rays. In other words, the infrared camera 10 is configured to be able to switch between a far infrared image and a near infrared image. The infrared camera 10 is connected to a central processing unit (CPU) 30 that performs arithmetic processing related to pedestrian recognition. The CPU 30 controls the operation of the near infrared light source 20.

  The infrared camera 10 includes a thermal reaction type imaging device 102 that reflects heat distribution, and has the capability of detecting humans through an obstacle that is about the thickness of a planted bush. Moreover, the infrared camera 10 can obtain a near-infrared image close to a visible light image by acquiring the near-infrared light by the near-infrared light emitted from sunlight during the daytime. Moreover, a near-infrared image can be obtained by emitting the near-infrared light source 20 not only in the daytime but also at night.

  The CPU 30 has a function of performing image recognition processing on the detection result (image, video) of the infrared camera 10 and calculating the probability that the detected object is a pedestrian. The navigation system 40 is connected to the CPU 30, and the CPU 30 receives various information from the navigation system 40, and can operate the infrared camera 10 and the near infrared light source 20 optimally. The CPU 30 is also connected to a display monitor 50 for finally notifying a passenger (driver, driver 60) of a specific pedestrian (a difficult-to-view pedestrian).

  FIG. 2 is a near-infrared image 12 a at night obtained by the infrared camera 10. At night, the near-infrared light source 20 emits light toward the front of the vehicle and is photographed by the infrared camera 10. As can be seen from FIG. 2, although the near-infrared image 12a is an image close to a visible light image, it is difficult to distinguish the pedestrian from the background.

  FIG. 3 is a far-infrared image 11 a obtained by the infrared camera 10. Objects with higher temperatures are displayed lighter (whiter). As can be seen from FIG. 3, the far-infrared image 11a can clearly detect a pedestrian compared to the near-infrared image 12a. However, since the background cannot be detected, it is impossible to determine where the pedestrian is by using only the far infrared image 11a.

  Hereinafter, the operating principle of the present invention will be described in more detail. FIG. 4 is a flowchart of the imaging apparatus 1 according to the present embodiment. The target imaging apparatus 1 according to the present embodiment includes an infrared camera 10, a CPU 30, and a display monitor 50 as shown in FIG.

  First, a switching signal is transmitted to the imaging means switching unit 18 of the infrared camera 10 by the imaging switching signal output unit 155 generated by the CPU 30. The imaging means switching unit 18 has a function in which the infrared camera 10 switches between the far infrared imaging means 11 and the near infrared imaging means 12 based on the switching signal. The far-infrared imaging means 11 acquires an image that reacts to infrared light having a wavelength of 7 to 14 μm, and the near-infrared imaging means 12 acquires an image that reacts to infrared light having a wavelength of 0.8 to 3 μm.

Since the far-infrared imaging unit 11 and the near-infrared imaging unit 12 are composed of an optical system including the same imaging element 102, the same wavelength variable filter 101, and the same optical lens group (not shown), the viewing angle. Then, an image in which the focal length and the optical axis are completely matched is acquired.
The far-infrared image 11 a acquired by the far-infrared imaging means 11 is transmitted to the obstacle candidate selection unit 13 built in the infrared camera 10. After the far-infrared image unit 11 captures the far-infrared image 11a at a predetermined timing, the imaging unit switching unit 18 drives the near-infrared imaging unit 12. Thereby, the infrared camera 10 that has acquired the far-infrared image until then switches to the acquisition of the near-infrared image, and the near-infrared imaging unit 12 acquires the near-infrared image 12a.

  The CPU 30 includes the imaging switching signal output unit 155, the template candidate selection unit 151, the feature amount extraction unit 152, the template determination unit 153, the image superimposition processing unit 154, and the near infrared light source control unit 156 described above. .

  The obstacle candidate selection unit 13 selects a specific portion of the far-infrared image 11a (the portion displayed in white in FIG. 3) as an image candidate, and the CPU 30 performs image processing with the template candidate selection unit 151 and the feature amount extraction unit 152. As a result, the position of the object that is an obstacle candidate ahead of the vehicle is specified. In other words, it can be said that the obstacle candidate selection unit 13, the template candidate selection unit 151, and the feature amount extraction unit 152 are means for specifying an obstacle candidate area in front of the vehicle.

  When the vehicle is traveling at night, a near-infrared light source control unit 156 built in the CPU 30 transmits a light emission command signal to the near-infrared light source 20 to cause the near-infrared light source 20 to emit light toward the front of the vehicle. In order to determine whether the vehicle is traveling at night or during the day, information may be received from a clock function provided in the navigation system 40, or an illuminance meter outside the vehicle may be separately installed in the imaging device 1. It may be determined from the illuminance. Furthermore, even if such a special discrimination means is not used, a near-infrared image in front of the vehicle is acquired by the infrared camera 10 in a state where the near-infrared light source 20 is not emitted, and the brightness of the image is analyzed and discriminated. good. Specifically, when the brightness of the acquired near-infrared image is darker than a certain value, it can be determined that the night.

  The feature of this embodiment is that the infrared camera 10 is driven by switching between the far-infrared imaging means 11 and the near-infrared imaging means 12. First, as candidate regions from the far infrared image 11a (FIG. 3) which is a thermal image (having temperature information) captured by the far infrared imaging means 11, the positions of the high temperature region and the intermediate temperature region in the thermal image and these candidate regions The representative temperature is extracted. And the near-infrared image 12a which is a landscape thermal image is acquired by the near-infrared imaging means 12 by the imaging means switching part 18 switching to near-infrared specification.

  Based on the position of the candidate area extracted from the far-infrared imaging means 11, a detection area for searching for a detection target is set in a grayscale image or a color image (color image), and according to the temperature of the extracted candidate area Template matching. This template matching is processed by the template candidate selection unit 151 built in the CPU 30.

  A region that approximates the temperature of the human body is processed by a feature amount extraction unit 152 built in the CPU 30. In the image data, an object existing in a region that approximates the temperature of the human body is determined as a pedestrian and replaced with an alert symbol incorporated in a template used in advance. This process is processed by the template determination unit 153 built in the CPU 30. In other words, the template determination unit 153 can be said to be an alert symbol conversion unit that converts an obstacle candidate area ahead of the vehicle into an alert symbol.

FIG. 5 is a converted image 11b obtained by recognizing the white area at the center of the far-infrared image 11a of FIG. 3 as a pedestrian and converting it into an alert symbol. And the process which superimposes and synthesize | combines the conversion image 11b of FIG. 5 on the near-infrared image 12a of FIG. 2 is performed, and the display image 12b shown in FIG. 6 is produced | generated. Then, the display image 12 b is displayed on the display monitor 50.
That is, the near-infrared imaging unit 12 captures the entire landscape, and the far-infrared imaging unit 11 detects the far-infrared rays from the pedestrian and determines the position of the pedestrian.

  Since far-infrared rays use a temperature difference, a white display is further performed in order to reliably detect a pedestrian's body temperature and to highlight an area where the pedestrian exists. In contrast, near-infrared images have a natural image, but it is difficult to distinguish the presence of pedestrians. In the case of an in-vehicle imaging device, near-infrared rays projected in front of the vehicle are directed in a certain direction regardless of the behavior of the vehicle and the road structure. Yes, pedestrians are difficult to see.

  Also, near-infrared images show a decrease in contrast between the pedestrian and the background when the pedestrian becomes difficult to see due to halation due to the oncoming vehicle, or when the near-infrared reflectance of the pedestrian clothing and the background is comparable. It may be difficult to see pedestrians. For this reason, a signboard or the like may be erroneously recognized as a pedestrian.

Therefore, in this embodiment, the pedestrian area detected by the far-infrared method is highlighted in white, and is determined as a pedestrian by pattern matching with the template. And superimpose only the alert symbol on the near-red line image. In that case, the dynamic information of the pedestrian acquired is added to the alert symbol in order to increase alerting to the driver.
In other words, when the pedestrian is dynamic, the alert symbol is displayed in the same size as the pedestrian image picked up with the alert symbol reddish and the attention is further increased. On the other hand, if it is static, no alert symbol is displayed. In other words, the driver is not given more information than necessary, and the driver is not distracted.
The determination of dynamic or static is made based on the degree of pedestrian opening of the pedestrian and the open state of both arms in the acquired image.

  In this embodiment, the wavelength tunable filter 101 is used as a switching method between the far infrared imaging and the near infrared imaging.

Hereinafter, the image processing of the CPU 30 will be described with reference to the flowchart of FIG.
If the imaging apparatus 1 is started when the power is turned on, the infrared camera 10, the CPU 30, and the display monitor 50 are activated, and in step (S1), initial setting of parameters necessary for processing is performed.

Next, in step (S2), the front in the vehicle traveling direction is imaged, and a far infrared image 11a as shown in FIG. 3 is captured and sequentially stored in the obstacle candidate selection unit 13 frame by frame.
Here, when the far-infrared imaging means 11 includes a human body as a detection target, it is sufficient that the far-infrared imaging means 11 has sensitivity in a far-infrared wavelength region of 7 to 14 μm, and the far-infrared imaging means 11 has a longer distance (approximately 200 m away). In order to obtain sensitivity, a pyroelectric infrared sensor camera is better.

  In step (S3), the near-infrared imaging means 12 images the front in the vehicle traveling direction, and captures a near-infrared image 12a that is a forward roadway image as shown in FIG. One frame at a time is sequentially stored.

In the present embodiment, the near-infrared imaging unit 12 and the far-infrared imaging unit 11 can acquire color information, but a case where monochrome is used will be described.
Note that the order of step (S2) and step (S3) may be any one as long as the interval between steps is short.

  Next, in step (S4), candidate regions are extracted from the far-infrared image 11a that is the thermal image obtained in step (S2). That is, the temperature region representing the feature of the object is extracted as a lump. In the present embodiment, the detection target object is described as a preceding vehicle and a pedestrian (person). When a target other than this is set as a detection target, the temperature information of the target may be added as a feature amount to the processing target in this step.

  First, in the case of a preceding vehicle, using the fact that the muffler region becomes a high temperature of about 100 ° C., a region having temperature information of, for example, 80 ° C. or more is searched from the thermal image, and the position information of the region is obtained. Detect temperature information.

  On the other hand, in the case of a pedestrian, a region having temperature information of, for example, about 30 ° C. is searched from a thermal image by utilizing the fact that the surface temperature of the exposed face, neck, and hand is about 30 ° C. Then, position information and temperature information of the area are detected. Here, among the results detected in each region, the position information may be, for example, the center of gravity coordinate position of the lump, and the temperature information may be, for example, the average temperature of the lump. In addition to the center-of-gravity coordinate position, the existence of a part satisfying conditions such as the aspect ratio, the sufficiency rate, and the actual area is examined.

As a result, the position of the pedestrian is obtained as a ratio to the width and height of the entire screen on the two-dimensional coordinates on the screen. Here, in the actual display process, the far infrared image or the near infrared image is displayed on the display monitor 50 and is not processed on the display screen, but the CPU 30 actually does not display the image frame of the obstacle candidate selection unit 13. Run on the data.
In step (S5), a position and temperature list in which the position information and temperature information of each region obtained in step (S4) are associated with each other is created, and the subsequent process is performed.

Next, in step (S6), based on the list created in step (S5), it is determined whether or not temperature information indicating the feature amount of the preceding vehicle or pedestrian that is the detection target is extracted.
If no candidate object is extracted from the list, it is determined that there is no preceding vehicle or pedestrian as a detection target in the image captured by the infrared camera 10 this time. Return to step (S2) for processing. On the other hand, if one or more candidate objects are extracted from the list, the process proceeds to the next step (S7), and the subsequent processing is performed based on the temperature information of the list created in step (S5). Select a template candidate to use. Specifically, a template having a value within the range of the average temperature information ± ΔT of the candidate area is set as a template candidate used for detecting an object in one candidate area. As a template used for the preceding vehicle presence candidate region, the temperature information of the template is 100 ± 3 ° C. However, since the high temperature portion is special in the normal driving environment, an area of 80 ° C. or higher may be selected as a template in advance. On the other hand, the temperature information held by the template is 28 ± 3 ° C., which is selected as the template used for the pedestrian presence candidate region. Considering that the high temperature part is affected by the outside air temperature, it is necessary to consider that the temperature of the same object changes within a predetermined range. In this example, the object having medium temperature information is a pedestrian, but in order to distinguish it from, for example, a dynamic animal (such as a dog), an animal template is also selected as a candidate in this step.

An optimal template is determined for each detection region, and template matching processing is performed in the infrared image using the template.
The template matching process has a mechanism for detecting existence candidates based on temperature information and replacing the detection candidates with alert symbols. For example, in the case of a pedestrian image, it is converted into a symbol such as a pictogram. Moreover, it is also possible to display a warning sound or a pictogram with a color at the same time as displaying the pictogram.

  That is, template matching processing is performed in the infrared image to detect the final presence or absence of the object and its position within each detection region. In step (S8), a vehicle template is selected for the preceding vehicle presence candidate region, and a person template is selected for the pedestrian presence candidate region. In this step, when the preceding vehicle is detected from the preceding vehicle presence candidate area, the presence / absence of the preceding vehicle is determined by the correlation calculation of the density value (gray value) in the preceding vehicle existence candidate area and the vehicle template using the vehicle template. The detailed position is detected. Further, when detecting a pedestrian from the pedestrian presence candidate area, the presence / absence of the pedestrian and its detailed position are detected by correlation calculation between the pedestrian existence candidate area and the density value in the human template. Such a detection process is performed in step (S9). In the present embodiment, the template matching process is performed by the correlation calculation using the density value during the template matching process. However, the present invention is not limited to this. For example, an edge image is provided as a template and the correlation value is set. The final presence / absence of the object and its position may be detected.

  Next, in step (S10), it is determined whether or not the detection process has been performed for all the regions in the list created in step (S5). If all the areas in the list have not been processed, the process returns to step (S8) to set the detection area in order to switch the processing area. If the processing for all the areas in the list has been completed, the process proceeds to the next step in order to output the processing results up to that point.

  Finally, in step (S11), as shown in the display image 12b of FIG. 6, the entire near-infrared image is displayed on the externally connected display monitor 50, and the template matching is performed in detail with the far-infrared image. The alert symbol is displayed, and only the dynamic pedestrians who need attention in the future are displayed large.

As a result, the alerts are displayed according to their purpose, which is accurately displayed to the driver, which contributes to cost reduction by improving the visibility, shortening the processing time by displaying the alert symbols, and simplifying the processing method.
After the display on the display monitor 50, the process returns to step (S2) in order to perform the next and subsequent processes.

As described above, in this embodiment, when the surrounding landscape is a near-infrared image centered on an infrared image, only the danger caused by the roadway is detected by the far-infrared image and displayed as an alert warning. Visibility has been improved, making judgment easier. In addition, image processing speed has been increased by replacing far-infrared pedestrians with alert symbols.
Furthermore, the captured images are not shifted by performing far-infrared and near-infrared imaging using a single imaging device.

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 10 ... Infrared camera, 20 ... Near infrared light source, 30 ... CPU, 50 ... Display monitor, 101 ... Variable wavelength filter, 102 ... Imaging element, 11 ... Far infrared imaging means, 12 ... Near infrared imaging means, 18... Imaging means switching unit, 152... Feature amount extraction unit, 154.

Claims (6)

An image sensor for imaging the front of the vehicle;
A near-infrared light source that irradiates near-infrared light in front of the vehicle;
Image imaging means for switching the image captured by the image sensor to a far-infrared image and a near-infrared image using a wavelength band filter;
Obstacle candidate area specifying means for specifying an obstacle candidate area based on the far-infrared image;
An image superimposition processing unit for superimposing the obstacle candidate region on the near-infrared image;
An image display unit for displaying the superimposed image generated by the image superimposition processing unit;
An imaging device comprising:   The imaging apparatus according to claim 1, wherein the wavelength band filter is a wavelength variable filter.   The alert symbol conversion unit that converts the obstacle candidate region into an alert symbol, and superimposes the alert symbol as the obstacle candidate region on the near-infrared image. Imaging device.   The imaging apparatus according to any one of claims 1 to 3, wherein the near-infrared light source is driven based on the near-infrared image.   The imaging device according to claim 1, wherein the imaging device is a pyroelectric imaging device.   The imaging according to claim 3, wherein the obstacle candidate specifying unit includes a determining unit that determines a dynamic pedestrian and a static pedestrian, and converts only the dynamic pedestrian into the alert symbol. apparatus.