JP5026398B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging device mounted on the vehicle.

From a captured image obtained through an imaging device mounted on a vehicle, there are a first object region (high luminance region) and a plurality of relatively small second object regions (high luminance regions) below the first object region. When extracted, it is determined that the object corresponds to a quadruped animal such as a deer, and a technique for notifying the driver of the presence of the object is known (see Patent Document 1).
JP 2007-310705 A

  However, according to the above-described method, the first target region and the second target region below the first target region can be extracted from the imaging device, and an erroneous determination is made that the artificial structure such as a vehicle corresponds to a quadruped animal such as a deer. there is a possibility.

  Then, an object of this invention is to provide the apparatus which can aim at the improvement of the determination precision whether a target object corresponds to a quadruped animal.

The vehicle periphery monitoring device of the first invention is a vehicle environment monitoring apparatus for monitoring a periphery of a vehicle using infrared captured image obtained by the infrared imaging apparatus mounted on the vehicle, from the infrared captured image, the Object extraction means for extracting an area where a high-luminance object is present on the infrared captured image as an object area, and luminance for a plurality of pixels constituting the object area extracted by the object extraction means A feature amount extracting means for calculating a standard deviation of the object, and determining that the object corresponds to a part or all of a quadruped animal on the condition that the standard deviation calculated by the feature amount extracting means is equal to or greater than a threshold value And an object classification means.

According to the vehicle periphery monitoring device of the first invention, the requirement that the infrared captured image is extracted from the standard deviation or variation degree of the luminance of the object area of the high luminance on the infrared captured image is equal to or greater than the threshold, It is determined that this object corresponds to a quadruped animal. The infrared captured image is a concept including not only an original image captured by an infrared imaging device but also a processed image generated by processing the original image. In view of the fact that brightness varies in quadruped animals in infrared imaging images due to undulations on the surface of the head, neck or torso of the quadruped animal, or uneven hair, the object is Whether it falls under an animal can be determined with high accuracy. In addition, a quadruped animal does not mean all the quadruped animals, but means a quadruped animal that can be identified by the technique of the present invention.

The vehicle periphery monitoring device of the second invention is a vehicle environment monitoring apparatus for monitoring a periphery of a vehicle using infrared captured image obtained by the infrared imaging apparatus mounted on the vehicle, from the infrared captured image, the Only in a plurality of pixels constituting the object area extracted by the object extraction means, the object extraction means for extracting an area where a high-luminance object exists on the infrared imaged image as the object area. The feature quantity extraction means for calculating the difference between the brightness and the minimum brightness as the first brightness difference, and the target object has four requirements on the condition that the first brightness difference calculated by the feature quantity extraction means is not less than a first threshold value. And an object classification means for determining that it corresponds to a part or all of a paw animal.

According to the vehicle periphery monitoring device of the second invention, extracted from an infrared captured image, the maximum brightness and the first luminance difference is the first threshold value as the difference between the minimum luminance of the object area of the high luminance on the infrared imaging image Based on the above requirements, it is determined that this object corresponds to a quadruped animal. In view of the fact that brightness varies in quadruped animals in infrared imaging images due to undulations on the surface of the head, neck or torso of the quadruped animal, or uneven hair, the object is Whether it falls under an animal can be determined with high accuracy.

  In the vehicle periphery monitoring device according to a third aspect of the present invention, in the vehicle periphery monitoring device according to the second aspect of the invention, the feature amount extraction means calculates the difference between the average luminance of the object region and the average luminance of the background region of the object region. It is calculated as a luminance difference, and the object classification means determines that the object corresponds to a part or all of a quadruped animal as a further requirement that the second luminance difference is a second threshold value or more. And

According to the vehicle periphery monitoring device of the third aspect of the invention, the second luminance difference as the difference between the average luminance of the object area extracted from the infrared captured image and the luminance of the background area of the object area is greater than or equal to the second threshold value. As a requirement, it is determined that this object corresponds to a quadruped animal. In view of the high probability that a relatively large luminance difference will occur between the object region and its background image, it can be determined with higher accuracy whether or not the object corresponds to a quadruped animal by the determination method.
The vehicle periphery monitoring device according to a fourth aspect of the present invention is the vehicle periphery monitoring device according to the first or second aspect of the present invention, and further, regarding the object region extracted by the object extracting means, the object existing in the object region corresponds to a human being. Human feature determination means for determining whether or not to perform, wherein the feature amount extraction means is a target determined by the human determination means as not being a human being among the target region extracted by the target object extraction means. For the object area, the luminance standard deviation is calculated only for a plurality of pixels constituting the object area.

A vehicle periphery monitoring device according to a fifth aspect of the present invention is the vehicle periphery monitoring device according to any one of the first to fourth aspects of the present invention , wherein a group of pixels having a luminance belonging to a specified luminance range from the object region extracted by the object extraction means. A feature amount extracting unit configured to extract a specified region having a specified shape and having a specified shape; and the number of the specified regions extracted by the feature amount extracting unit is equal to or greater than a specified value. And an object classifying unit that determines that it corresponds to a part or all of a quadruped animal.

According to the vehicle periphery monitoring device of the fifth aspect of the present invention structure, extracted from an infrared captured image, in the object region of high intensity on the infrared imaging image, the pixel group of people grouped with luminance included in the designated luminance range In addition, it is determined that the target object corresponds to a quadruped animal on the condition that a specified region having a specified shape is present at a specified value or more. In view of the possibility of detecting uneven brightness portions having a unique shape in the quadruped animal in the captured image due to undulation of the surface of the head, neck or torso of the quadruped animal or uneven hair. The determination method can determine with high accuracy whether or not the object corresponds to a quadruped animal.

  An embodiment of a vehicle periphery monitoring device of the present invention will be described. First, the structure of the vehicle periphery monitoring apparatus of this embodiment is demonstrated. The vehicle periphery monitoring device includes an image processing unit 1 shown in FIGS. 1 and 2. The image processing unit 1 is connected to a pair of left and right infrared cameras (corresponding to the “imaging device” of the present invention) 2R and 2L that image the front of the vehicle 10 and sensors that detect the traveling state of the vehicle 10. The yaw rate sensor 3 for detecting the yaw rate of the vehicle 10, the vehicle speed sensor 4 for detecting the traveling speed (vehicle speed) of the vehicle 10, and the brake sensor 5 for detecting the presence or absence of a brake operation of the vehicle 10 are connected. Further, the image processing unit 1 has a speaker 6 for outputting auditory report information such as voice and the vicinity of the front window for displaying captured images taken by the infrared cameras 2R and 2L and visual report information. HUD (Head Up Display) 7 is connected. When the distance to the object is measured by a radar, only one infrared camera may be mounted on the vehicle 10. Instead of or in addition to the HUD, a display that displays a traveling state such as the vehicle speed of the vehicle 10 or a display that an in-vehicle navigation device has may be employed as the image display device.

  The image processing unit 1 includes an ECU (electronic control unit) that includes an A / D conversion circuit, a memory such as a CPU, a RAM, and a ROM, an I / O circuit, and a bus that connects these. The image processing unit 10 (specifically, a CPU that executes arithmetic processing in accordance with a program stored in a memory or a storage device) performs “object extraction means”, “feature amount extraction means”, and “object type determination” according to the present invention. Means ". Analog signals output from the infrared cameras 2R, 2L, the yaw rate sensor 3, the vehicle speed sensor 4, and the brake sensor 5 are converted into digital data by the A / D conversion circuit, and based on the data digitized by the CPU, humans (pedestrians) Or an object such as a cyclist or a two-wheeled vehicle) or a quadruped animal (such as a deer, a horse or a dog), and when the detected object satisfies a predetermined notification requirement, the object is detected through the speaker 6 or the HUD 7. The driver is notified of the presence or the possibility of contact between the vehicle 10 and the object.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion of the vehicle 10 (the front grill portion in the figure) in order to image the front of the vehicle 10. The right infrared camera 2R is disposed at a position to the right of the center of the vehicle 10 in the vehicle width direction, and the left infrared camera 2L is disposed at a position to the left of the center of the vehicle 10 in the vehicle width direction. Their positions are symmetrical with respect to the center of the vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L are fixed so that their optical axes extend in the front-rear direction of the vehicle 10 in parallel with each other, and the heights of the respective optical axes from the road surface are equal to each other. The infrared cameras 2R and 2L have sensitivity in the far-infrared region, and have a characteristic that the higher the temperature of an object to be imaged, the higher the level of the output video signal (the higher the luminance of the video signal). is doing.

  Next, functions of the vehicle periphery monitoring device having the above-described configuration will be described. Details of the image processing are disclosed in Japanese Patent Laid-Open Nos. 2001-006096 and 2007-310705, and will be briefly described. First, infrared image signals of the infrared cameras 2R and 2L are input to the image processing unit 1, the infrared image signal is A / D converted, and a gray scale image is generated based on the A / D converted infrared image signal. Then, the reference grayscale image (right image) is binarized. After that, an area where the object exists in the binarized image is extracted as the object area S (FIG. 3 / STEP002). Specifically, a pixel group constituting a high luminance area of the binarized image is converted into run length data, and a label (identifier) is attached to each of the line groups that overlap in the vertical direction of the reference image. Are extracted as the object region S. As a result, a high-luminance region composed of a group of high-luminance pixels (pixel value = 1 pixel) as shown by hatching in FIGS. 5A and 5B is extracted as the object region S. The

  Subsequently, the position of the center of gravity of each object (position on the reference image), the area, and the aspect ratio of the circumscribed rectangle are calculated. Further, tracking of the target object is performed, and the identity of the target object is determined for each calculation processing cycle of the image processing unit 1. Further, the outputs (vehicle speed detection value and yaw rate detection value) of the vehicle speed sensor 4 and the yaw rate sensor 5 are read. Further, in parallel with the calculation of the aspect ratio of the circumscribed rectangle and the tracking of the object during the time, the area corresponding to each object (for example, the area of the circumscribed rectangle of the object) is extracted as a search image from the reference image. Furthermore, by executing the correlation calculation, an image (corresponding image) corresponding to the search image is searched and extracted from the left image. Further, a real space distance (distance in the front-rear direction of the vehicle 10) z from the vehicle 10 to the object is calculated. Moreover, the real space position (relative position with respect to the vehicle 10) of each object is calculated. Further, the X component of the real space position (X, Y, Z) (see FIG. 2) of the object is corrected according to the time-series data of the turning angle. Further, the relative movement vector of the object with respect to the vehicle 10 is calculated (FIG. 3 / STEP004).

  Furthermore, the level of the possibility of contact between the vehicle 10 and the object is determined (FIG. 3 / STEP006). Since this determination method is also disclosed in Japanese Patent Laid-Open Nos. 2001-006096 and 2007-310705, it will be briefly described. First, it is determined whether or not the real space distance z of the object is equal to or less than the product of the relative speed Vs and the margin time T. When it is determined that the real space distance z is equal to or less than the value, it is determined whether or not the object exists in the approach determination region (first contact determination process). The approach determination area extends symmetrically in the left-right direction of the vehicle 10 in front of the vehicle 10 and has a width obtained by adding an offset value to the width of the vehicle 10. When the determination result in the first contact determination process is affirmative, it is determined that there is a high possibility that the object is in contact with the vehicle 10. On the other hand, when the determination result in the first contact determination process is negative, the object exists in the entry determination area outside the approach determination area, and the relative movement vector of the object moves toward the approach determination area. It is further determined whether or not there is (second contact determination process). When the determination result in the second contact determination process is affirmative, it is determined that there is a high possibility that the target object is in contact with the vehicle 10.

  When it is determined that the possibility of contact between the vehicle 10 and the object is high (when the determination result in the first or second contact determination process is affirmative) (FIG. 3 / STEP006... YES), the object is artificial. It is determined whether or not the object, such as a structure, has a low necessity for reporting its existence (FIG. 3 / STEP008). For example, when an external feature that does not correspond to a pedestrian or a quadruped is detected in this object, it is determined that the object corresponds to an artificial structure. When it is determined that the possibility of contact between the vehicle 10 and the object is low (when the determination results in the first and second contact determination processes are negative) (FIG. 3 / STEP006... NO), or the object Is determined not to fall under the notification target (FIG. 3 / STEP008... NO), the processing subsequent to the extraction of the object region S is repeated without executing the notification processing described later (see FIG. 3 / STEP002, etc.). .

  When it is determined that the object does not correspond to an artificial structure or the like (FIG. 3 / STEP008... YES), it is determined whether or not the object corresponds to a person such as a pedestrian or a bicycle occupant (FIG. 3 / (STEP010). When it is determined that the object does not correspond to a human (FIG. 3 / STEP010... NO), it is determined whether or not the object corresponds to a quadruped animal such as a deer, a horse or a dog (FIG. 3). / STEP012). Details of this determination method will be described later.

  When it is determined that the object corresponds to a human (FIG. 3 / STEP010... YES), or when it is determined that the object corresponds to a quadruped animal (FIG. 3 / STEP012... YES), “report processing” Is executed (FIG. 3 / STEP014). As a result, sound such as “beep” is output from the speaker 6. One or both of a rectangular frame surrounding the object and a mark indicating that the object is a human are displayed on the HUD 7. Note that only one of the image and the sound may be output instead of both.

  Next, a method for determining whether a subject is a quadruped animal will be described. First, it is determined whether or not the object region S satisfies the basic requirements (FIG. 4 / STEP 202). Specifically, (1) the height Y (see FIG. 5A) of the center of gravity of the object region S (or the center of the circumscribed rectangle) P in the real space position is the standard height of the quadruped animal body. And (2) the width W of the region R that includes the object region S and has a correlation degree equal to or greater than a set value in the captured images of the cameras 2R and 2L (see FIG. 5B). ) Is included in the standard width range of quadrupeds when viewed from the side. When it is determined that the object area S does not satisfy the basic requirements (FIG. 4 / STEP 202... NO), it is determined that the object area S does not correspond to a part or an anterior part of the quadruped animal (FIG. 4). / STEP 240). It should be noted that the subsequent processing may be executed after omitting the satisfaction determination of the basic requirements.

  When it is determined that the object area S satisfies the basic requirements (FIG. 4 / STEP 202... YES), a luminance histogram representing the luminance distribution of a plurality of pixels constituting the object area S is created (FIG. 4). / STEP 204). Subsequently, based on this luminance histogram, a luminance standard deviation σ representing the degree of variation in luminance of the plurality of pixels constituting the object region S is calculated (FIG. 4 / STEP 206), and this luminance standard deviation σ is greater than or equal to the threshold σ0. It is determined whether or not there is (FIG. 4 / STEP 208). If it is determined that the luminance standard deviation σ is equal to or greater than the threshold σ0 (FIG. 4 / STEP208... YES), the first determination flag aflag1 is set to “1” (FIG. 4 / STEP210). On the other hand, when it is determined that the luminance standard deviation σ is less than the threshold σ0 (FIG. 4 / STEP208... NO), the first determination flag aflag1 is set to “0” (FIG. 4 / STEP212).

  Further, based on the luminance histogram, the difference between the highest luminance and the lowest luminance of the plurality of pixels constituting the object region S is calculated as the first luminance difference ΔL1 (FIG. 4 / STEP 214), and the first luminance difference ΔL1 is the first threshold value. It is determined whether or not Δ1 or more (FIG. 4 / STEP 216). When it is determined that the first luminance difference ΔL1 is greater than or equal to the first threshold value Δ1 (FIG. 4 / STEP 216... YES), the average luminance of the object area S and the background image of the object area S (for example, the area R The average brightness of the area excluding the object area S) is calculated (see FIG. 4 / STEP 218). Then, the difference between the average luminance of the object region S and the luminance of the background region is calculated as the second luminance difference ΔL2 (FIG. 4 / STEP 220), and whether or not the second luminance difference ΔL2 is equal to or larger than the second threshold value Δ2. Is determined (FIG. 4 / STEP 222). When it is determined that the second luminance difference ΔL2 is greater than or equal to the second threshold value Δ2 (FIG. 4 / STEP222... YES), the second determination flag aflag2 is set to “1” (FIG. 4 / STEP224). On the other hand, when it is determined that the first luminance difference ΔL1 is less than the first threshold value Δ1 (FIG. 4 / STEP216... NO), or when the second luminance difference ΔL2 is determined to be less than the second threshold value Δ2. (FIG. 4 / STEP222... YES), the second determination flag aflag2 is set to “0” (FIG. 4 / STEP226). Note that the processing of STEP 218 to STEP 222 may be omitted, and the second determination flag aflag2 may be set to “1” only on the condition that the first luminance difference ΔL1 is equal to or greater than the first threshold value Δ1.

  Furthermore, the designated area is extracted from the object area S determined to satisfy the basic requirements as described above (FIG. 4 / STEP 228), and the number N of the extracted designated areas is equal to or greater than the designated value N0. It is determined whether or not (FIG. 4 / STEP 230). The designated area is composed of a group of pixels having a luminance included in a designated luminance range (for example, a luminance range determined based on the standard deviation σ or the average luminance), and has a designated shape (for example, four images captured through an infrared camera). A shape such as a horizontally long ellipse empirically extracted from a paw animal. When it is determined that the number N of extractions of the specified area is equal to or greater than the specified value N0 (FIG. 4 / STEP230... YES), the third determination flag aflag3 is set to “1” (FIG. 4 / STEP232). On the other hand, if it is determined that the number N of extractions of the specified area is less than the specified value N0 (FIG. 4 / STEP230... NO), the third determination flag aflag3 is set to “0” (FIG. 4 / STEP234).

  Subsequently, the sum Σaflagi (i = 1, 2, 3) of the three determination flags set as described above is calculated, and it is determined whether or not this is equal to or greater than the determination threshold Σth (FIG. 4 / STEP 236). The determination threshold Σth can be set to “1”, “2”, or “3”. When it is determined that the total determination flag Σaflagi is greater than or equal to the determination threshold Σth (FIG. 4 / STEP 236... YES), it is determined that the object corresponds to a part or all of a quadruped animal (FIG. 4 / STEP 238). On the other hand, when it is determined that the total determination flag Σaflagi is less than the determination threshold Σth (FIG. 4 / STEP 236... NO), it is determined that the object does not correspond to part or all of the quadruped animals (FIG. 4 / (STEP 240).

  According to the vehicle periphery monitoring device that exhibits the above function, it is necessary that the standard deviation σ or the variation degree of the luminance of the target region S extracted from the captured image is equal to or greater than the threshold σ0. (See STEP 208 and STEP 236 in FIG. 4). Moreover, it can be determined that the target object corresponds to a quadruped animal on the condition that the first luminance difference ΔL1 as the difference between the maximum luminance and the minimum luminance of the target region S is equal to or greater than the first threshold value Δ1 (FIG. 4). / STEP 216, STEP 236). In view of the fact that the variation in brightness specific to the quadruped animal in the captured image is caused by the undulation of the surface such as the head, neck or torso of the quadruped animal, or unevenness of the body hair, the above-mentioned determination method makes it possible to detect the four objects. It can be determined with high accuracy whether it corresponds to a paw animal.

  Furthermore, it is required that the second luminance difference ΔL2 as a difference between the average luminance of the target region S and the luminance of the background region of the target region is equal to or greater than the second threshold value Δ2, and this target is a quadruped animal. It can be determined that this is the case (see FIG. 4 / STEP 220, STEP 236). In view of the high probability that a relatively large luminance difference will occur between the object region S and its background image, it can be determined with higher accuracy whether or not the object corresponds to a quadruped animal by the determination method.

  Further, in the object area S, it is necessary that the object area is composed of a group of pixels having a luminance included in the specified luminance range and that the specified area having the specified shape is greater than or equal to the specified value. It can be determined that it corresponds to a paw animal (see FIG. 4 / STEP 208, STEP 230). In view of the possibility of detecting uneven brightness portions having a unique shape in the quadruped animal in the captured image due to undulation of the surface of the head, neck or torso of the quadruped animal or uneven hair. The determination method can determine with high accuracy whether or not the object corresponds to a quadruped animal.

  In the above embodiment, the three determination flags aflag1 to aflag3 are set, and whether or not the subject is a quadruped animal is determined depending on whether the total of the three determination flags is equal to or greater than the determination threshold Σth. (See FIG. 4 / STEP 210, 212, 224, 226, 232, 234, 236) As another embodiment, only one determination flag among these three determination flags is set, and the one determination flag is set to the determination threshold “ Depending on whether or not it is “1” or more, the suitability of the quadruped animal of the object may be determined. For example, when the first determination flag aflag1 can be set to 1, it may be determined immediately that the object corresponds to a body part of a quadruped animal. Further, only two determination flags among these three determination flags are set, and depending on whether the total of the two determination flags is equal to or greater than the determination threshold “1” or “2”, the subject quadruped animal Applicability may be determined.

Configuration explanatory diagram of the vehicle periphery monitoring device of the present invention Explanatory drawing about a vehicle equipped with a vehicle periphery monitoring device Flow chart showing basic functions of vehicle periphery monitoring device Flow chart regarding a method for determining whether a subject is a quadruped animal Explanatory drawing on the method for determining the target of a quadruped animal

Explanation of symbols

1. Image processing unit (object extraction means, feature amount extraction means, object classification means), 2R, 2L, infrared camera (imaging device)

Claims (5)

A vehicle periphery monitoring device that monitors the periphery of a vehicle using an infrared imaging image obtained by an infrared imaging device mounted on the vehicle,
From the infrared captured image, and the object extracting means for extracting a region where the object of a high luminance on the infrared imaging image exists as the object region,
Feature quantity extraction means for calculating a standard deviation of luminance only for a plurality of pixels constituting the object area extracted by the object extraction means;
An object classification unit that determines that the object corresponds to a part or all of a quadruped animal on the condition that the standard deviation calculated by the feature amount extraction unit is greater than or equal to a threshold value. A vehicle periphery monitoring device. A vehicle periphery monitoring device that monitors the periphery of a vehicle using an infrared imaging image obtained by an infrared imaging device mounted on the vehicle,
From the infrared captured image, and the object extracting means for extracting a region where the object of a high luminance on the infrared imaging image exists as the object region,
Feature quantity extraction means for calculating a difference between the highest luminance and the lowest luminance as a first luminance difference only in a plurality of pixels constituting the object region extracted by the object extraction means;
Object classification means for determining that the object corresponds to a part or all of a quadruped animal on the condition that the first luminance difference calculated by the feature quantity extraction means is not less than a first threshold value. A vehicle periphery monitoring device characterized by comprising: In the vehicle periphery monitoring device according to claim 2,
The feature amount extraction means calculates a difference between the average luminance of the object area and the average luminance of the background area of the object area as a second luminance difference,
The vehicle periphery monitoring device, wherein the object classification means determines that the object corresponds to a part or all of a quadruped animal on the further requirement that the second luminance difference is equal to or greater than a second threshold value. . The vehicle periphery monitoring device further includes a human determination unit that determines whether or not a target object existing in the target region corresponds to a human with respect to the target region extracted by the target object extraction unit.
The feature amount extraction unit includes a plurality of object regions that are determined by the human determination unit as not corresponding to a human among the target regions extracted by the target extraction unit. The vehicle periphery monitoring device according to claim 1, wherein the standard deviation of luminance is calculated only for the pixels. A feature amount extracting unit configured to extract a designated region having a designated shape, which is configured by a group of pixels having luminance belonging to a designated luminance range, from the target region extracted by the target object extracting unit;
Object classification means for determining that the object corresponds to a part or all of a quadruped animal on the condition that the number of the designated areas extracted by the feature quantity extraction means is equal to or greater than a designated value. The vehicle periphery monitoring device according to any one of claims 1 to 4, wherein

Images