JP4543861B2 - Person detection apparatus and method - Google Patents

Description

  The present invention relates to a person detection apparatus and method for distinguishing and detecting other vehicles and persons among objects existing around the host vehicle and presenting them to a driver or the like.

  Conventionally, for example, in order to detect a person as an obstacle present in the traveling direction of a vehicle, an infrared image obtained by changing the brightness of a result of measuring a value corresponding to a temperature is captured by an infrared camera. As a technique for specifying a person, a technique described in Patent Document 1 below is known.

In the technique described in Patent Document 1, a high temperature region is detected by binarization processing, and a stereo image processing is performed to measure the distance from the high temperature region to the host vehicle. It is determined whether a high temperature region of distance and size exists in pairs. When it is determined that there is a pair of high-temperature regions that are symmetrical and the same distance, there is another vehicle in front of the host vehicle, and when it is determined that there is no person, there is a person in front of the host vehicle. By discriminating what to do, the other vehicle and the person are distinguished.
JP 2003-230134 A

  However, since the technique described in Patent Document 1 described above is based on the assumption that the back surface of the vehicle has a bilaterally symmetric shape, the following vehicle captures the back surface of the preceding vehicle while traveling. Can prevent false detection that the preceding vehicle is a person, but if the preceding vehicle is traveling diagonally with respect to the following vehicle, such as when turning right or left, or crossing the front of the host vehicle When such a sideways vehicle exists, it cannot be determined that the high temperature region is another vehicle.

  Moreover, in the technique described in Patent Document 1 described above, there is a problem in that it is impossible to distinguish a vehicle and a person who are not symmetrical with each other, such as a two-wheeled vehicle.

  Therefore, the present invention has been proposed in view of the above-described circumstances, and a person who can reliably distinguish another vehicle from a person regardless of the direction of the other vehicle relative to the host vehicle and the shape of the other vehicle. It is an object to provide a detection apparatus and method.

  In the present invention, a thermal image composed of pixels having a luminance value corresponding to the temperature of the object is captured, and the thermal image has a luminance value within the range between the upper threshold value and the lower threshold value corresponding to the temperature of the person. A binarized image for human detection is created by distinguishing pixels and pixels outside the range of the upper and lower thresholds of the luminance value corresponding to the temperature of the person, and binarization for human detection Referring to the image, detect an area where the pixel group within the upper and lower thresholds of the luminance value corresponding to the person's temperature is greater than or equal to the predetermined area, and out of the detected areas One or a plurality of areas existing within a predetermined distance threshold are grouped as person candidate areas, and the object of the person candidate area is a person based on the area and shape of the grouped person candidate areas It is determined whether or not.

  In the present invention, pixels that have a luminance value that is clearly not a person are subject to human detection by excluding a high temperature region that is higher than the temperature of the person from the captured thermal image. Can be removed.

  According to the present invention, pixels that have a luminance value that is clearly not a person are subject to human detection by excluding a high-temperature region that is higher than the temperature of the person from the captured thermal image. Therefore, if a high-temperature region is detected, it is possible to reliably distinguish the other vehicle from the person, regardless of the direction of the other vehicle relative to the own vehicle and the shape of the other vehicle. Can be determined.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
For example, as shown in FIG. 1, the present invention detects a person using a thermal image captured by a far-infrared camera 10 (imaging means) in which the forward direction of the host vehicle 1 is an imaging range. This is applied to the detection device 20. 1A shows the optical axis of the far-infrared camera 10 when the host vehicle 1 is viewed from the side, and FIG. 1B shows the far-infrared camera when the host vehicle 1 is viewed from above. 10 optical axes and a reference coordinate system are shown. In this coordinate system, the horizontal axis is X, the vertical axis is Y, and the coordinates indicating the pixel position in the image can be obtained with the upper left pixel as the origin.

  The far-infrared camera 10 is provided, for example, in the vicinity of an instrument panel of a vehicle or an interior mirror, and has an imaging range in front of the vehicle. The far-infrared camera 10 generates thermal image data having a higher luminance value as the temperature value of the object is higher, and outputs the thermal image data to the person detection device 20. Therefore, the luminance value of the thermal image data is a value corresponding to the temperature of an object existing in front of the vehicle.

  The person detection device 20 has an input I / F for inputting thermal image data from the far-infrared camera 10, an image memory, and a configuration for performing processing as shown in steps S 1 to S 9 as shown in FIG. It consists of a program that describes the contents of various processes to be described later, a CPU that executes the program, and the like.

  That is, the human detection device 20 includes an input I / F for inputting thermal image data from the far-infrared camera 10 in step S1, an image memory for storing thermal image data in step S2, and a human detection and high-temperature area in step S3. A temperature threshold value setting unit for setting a temperature threshold value for detection, and a binarization processing unit for performing binarization processing using each temperature threshold value in step S4 (binarization processing means for person detection) , Binarization processing means for detecting a high temperature area), a labeling processing section for performing a labeling process using two respective binarized images in step S5 (area detecting means for detecting a person, area detection for detecting a high temperature area) Means), a grouping target area detecting unit (high temperature) for detecting a region (label) to be grouped by removing the vicinity of the high temperature area in step S6 Frequency removing means), a grouping processing unit for detecting a person candidate region by performing grouping (grouping means), the person determining section for performing person determination in Step S8 (the person determination means) in step S7.

  First, when the human image data from the far-infrared camera 10 is input by the input I / F (step S1), the person detection device 20 stores the thermal image data in the image memory (step S2).

  Next, the person detection device 20 performs temperature threshold setting processing (step S3A) and binarization processing (step S4A) as processing for detecting a label for the person in the thermal image stored in the image memory in step S2. ) And a labeling process (step S5A), and a temperature threshold setting process (step S3B) as a process for detecting a label for a high-temperature object in a thermal image having a temperature higher than that corresponding to a person. Binarization processing (step S4B) and labeling processing (step S5B) are performed.

  When detecting a label for a person, first, in step S3A, the temperature threshold value setting unit sets upper and lower threshold values suitable for detecting the person based on the weather and temperature of the day when the thermal image was taken. To do. The luminance value of the thermal image data captured by the far infrared camera 10 corresponds to a temperature value within the imaging range of the far infrared camera 10. Further, the skin temperature, which is the temperature of the person, is usually around 36 ° C., and in the thermal image data, when the distance between the person and the vehicle 1 is small, the skin portion of the person has a luminance value close to 36 ° C., As the distance between the person and the vehicle 1 increases, the infrared potential for temperature detection decreases, and the luminance value is affected by the temperature around the person.

  For this reason, when the temperature is high, about 25 ° C. to 36 ° C. is set as a temperature threshold for human detection, and when the temperature is low, about 20 ° C. to 28 ° C. is set for human detection. Temperature threshold. Thus, based on a preset distance range to the person to be detected and an environment such as the weather, a temperature threshold for human detection that appears in a thermal image when a person is imaged is set. Set. It should be noted that the temperature threshold for human detection is set so as not to include a temperature clearly higher than the body temperature, such as 45 ° C. or higher.

  On the other hand, in step S3B, the temperature threshold value setting unit sets a temperature threshold value that detects a high temperature region having a temperature higher than that of a person such as a muffler or a light unit of the vehicle. At this time, the temperature threshold setting unit sets a temperature threshold for detecting a high temperature region so as to detect a region of 40 ° C. or higher, for example.

  Here, when a forward vehicle and a person are present in the traveling direction of the host vehicle 1, a thermal image as illustrated in FIGS. 3A and 3B is captured by the far-infrared camera 10. This thermal image is taken on a day when the temperature is lower than the body temperature (for example, about 10 ° C. to 20 ° C.) and includes a person, a preceding vehicle, a road surface, a wall, and the like. The lower part is darker. Also, depending on the material, the wall and road surface are detected as a temperature that is almost the same as the air temperature, and because the human body attenuates infrared rays according to the distance, the human body temperature itself cannot be measured if the distance is long, but the skin The exposed face and limbs are detected as a temperature slightly lower than the body temperature of 36 ° C, and the torso covered with clothes is slightly lower than the limbs, but slightly higher than the road surface and walls. Detected. The front vehicle has a body part that is detected at a temperature slightly higher than the road surface and background, although it depends on the duration of sunshine and the temperature inside the vehicle. At very high temperatures. As shown in FIG. 3B, in the high temperature region such as the light or the muffler, the substantially central portion is detected as a temperature higher than the person, and the outer portion of the substantially central portion is detected as the temperature of the person. .

  Accordingly, in step S3A, a temperature threshold value for detecting a person for detecting at least the head and limbs of the person as shown in FIG. 4B described later is set. In step S3B, the temperature threshold shown in FIG. As shown in d), a temperature threshold value for detecting a high temperature region is set which does not overlap with the temperature threshold value for human detection and includes a high temperature region.

  Here, on a relatively low temperature day, the temperature varies depending on the material of the clothes of the person's torso, and in many cases, the temperature is close to the temperature. For this reason, in order to reliably extract only the person portion, a temperature range for detecting a temperature close to the skin temperature that can be determined to be a person is set as a temperature threshold for person detection. On the other hand, when the temperature threshold for detecting a person is set as the temperature range for detecting the entire person including the torso part, the torso part is at a temperature close to the road surface or wall as the background object. Not only a person but also a background object is detected by binarization. In this way, when a human part excluding the body part is detected, the grouping process described later enables the detection of a person even in a binarized image that does not include the body part. Only a part such as a part is detected, and a temperature threshold value for detecting a person from which a background object is not detected is used.

  Next, in step S4A, the human detection device 20 uses the binarization processing unit to set pixels between the upper threshold and lower threshold for human detection set by the temperature threshold setting unit, Binarization is performed with pixels other than those between the upper threshold and the lower threshold for person detection. As a result, the person detection device 20 creates a binary image in which the luminance value corresponding to the temperature corresponding to the person is set to “1” and the luminance value not corresponding to the temperature corresponding to the person is set to “0”. On the other hand, in step S4B, the human detection device 20 uses the binarization processing unit to detect pixels between the upper threshold value and the lower threshold value for detecting the high temperature region set by the temperature threshold value setting unit. And binarization is performed with pixels other than those between the upper threshold and the lower threshold for human detection.

  Specifically, a thermal image similar to that shown in FIG. 3A is taken as shown in FIG. 4A, and the temperature threshold for human detection shown in FIG. 4B is obtained in step S3A, and in step S3B. A temperature threshold value for detecting a high temperature region shown in FIG. When the thermal image of FIG. 4 (a) is binarized with the temperature threshold for human detection shown in FIG. 4 (b), the binarized image shown in FIG. 4 (c) is obtained. Not only the head and limbs of the person but also the part outside the high temperature area of the vehicle ahead is detected as the person temperature. On the other hand, when the thermal image of FIG. 4A is binarized with the temperature threshold for detecting the high temperature region shown in FIG. 4D, the binarized image shown in FIG. As a result, the approximate center position of the ramp portion and the muffler portion of the preceding vehicle is detected as a high temperature region.

  In the next step S5A and step S5B, the person detection device 20 uses the binarized image created by the binarization processing unit by the labeling processing unit, and is an area having a luminance value of “1”. A labeling process for detecting an area corresponding to a person is performed. At this time, the labeling processing unit searches for pixels having a luminance value of “1” in order from the upper left pixel of the binarized image, for example, and sets adjacent pixels having a luminance value of “1” or within a predetermined distance to each other. One area (label).

  In step S5A, the labeling processing unit obtains a binarized image (FIG. 4C, FIG. 5A) obtained by the temperature threshold for person detection, that is, the head, hand, and foot of the person. Labeling is performed on the binarized image that is separated and has a region outside the high temperature region. As a result, as shown in FIG. 5 (b), the label d of the head, the labels e and h of the hands, and the labels f and g of the feet are extracted, and the labels a and c around the ramp of the other vehicle, and the muffler The label b around the part is extracted.

  On the other hand, in step S5B, the labeling processing unit obtains a binarized image (FIGS. 4 (e) and 5 (c)) obtained from the temperature threshold value for detecting the high temperature region, that is, the approximate center position of the lamp unit. Then, a labeling process is performed on the binarized image in which an area corresponding to the substantially center position of the muffler portion exists. Thereby, as shown in FIG. 5D, the labels a and c of the lamp and the label c of the muffler part are extracted.

  Note that the images shown in FIGS. 5B and 5D are images after removing a small region of, for example, about 1 pixel or 2 pixels as noise in the grouping target temperature region detection process described later. .

  As described above, the person detection device 20 performs the processes in steps S3A to S5A for detecting a person's label and the processes in steps S3B to S5B for detecting a label in the high temperature region in parallel, thereby Labels and vehicle labels can be detected separately.

  In the next step S6, the grouping target area detecting unit excludes the label area smaller than the predetermined image area from the label areas detected by the labeling processing unit, and selects only the label area larger than the predetermined image area. (Step S6). Here, the predetermined image area is set so as to detect a person existing within a person detection target distance from the host vehicle 1, and a too small label area (for example, about 1 to 2 pixels) is excluded as noise. The label area is set so as to be away from the host vehicle 1 so that the driver of the host vehicle 1 need not be warned.

  In addition, the person detection device 20 removes the labels detected by the temperature around the high temperature region that is not the target of the person grouping from the labels obtained by the labeling processing unit in step S5A and step S5B, and performs grouping. Only labels that are subject to At this time, the grouping target area detection unit performs detection positions (for example, center positions) of labels a to h shown in FIG. 5B and detection positions (for example, center positions) of labels a to c shown in FIG. ) And the labels having the same detection position are removed from the labels shown in FIG. That is, when the label shown in FIG. 5D is included in the label shown in FIG. 5B, the label shown in FIG. 5B is removed. Here, since the detection positions of labels a, b, and c shown in FIG. 5B coincide with the detection positions of a, b, and c shown in FIG. Leaves the labels d, e, f, g, h obtained by removing the labels a, b, c from the labels a to h in FIG. 5B, and creates an image as shown in FIG. Here, the labels a, b, and c in FIG. 5B are labels detected by the lamp and the muffler in FIG. 4A, and are approximately centered by the temperature around the high temperature region higher than the person temperature. It is formed.

  In the next step S <b> 7, the person detection device 20 performs a grouping process for grouping the labels detected based on the temperature of the person in step S <b> 6 and detecting a person candidate area by the grouping processing unit. At this time, the grouping processing unit detects the distance between the labels selected by the grouping target region detection unit, and the distance is the same as the labels within a predetermined distance threshold as shown in FIG. Perform grouping to be set as a group. The predetermined distance threshold value includes a distance threshold value thy in the Y-axis direction (vertical direction) and a distance threshold value thx in the X-axis direction (horizontal direction).

  Here, in the state where the detection target of the person detection device 20 is a person and the person is walking, the person in the thermal image is detected vertically, or as described above, on a cold day when the temperature is low. In the acquired thermal image, the torso portion of the person is often not extracted by the labeling process, and therefore, a distance threshold value thy in the Y-axis direction is set to about the distance between the head and the leg. On the other hand, the distance threshold thx in the X-axis direction is approximately the lateral width of the person shorter than the distance threshold value thy in the Y-axis direction because the person is detected in a vertically long position. The distance is set to about the width of.

  Thereby, as shown in FIG. 7, when there are three labels a, b, and c, the grouping processing unit moves the labels a to c within the range of the distance threshold value thy. When within the range of the threshold value thx, the labels a to c are grouped as a group of labels detected from the same object, and a rectangular area including the maximum and minimum X coordinates and the maximum and minimum Y coordinates of the labels a to c In this case, candidate human regions having distances xLn, yLn (n = 1, 2, 3...) Are set.

  Specifically, when the labels d to h are detected in the thermal image after the labeling process as shown in FIG. 6A, first, the grouping processing unit pays attention to the label d and starts from the label d. When a label existing in the distance range of the distance threshold values thx, thy is searched, as shown in FIG. 6B, the labels e, f, g, h is detected. Then, the grouping processing unit detects that the labels e, f, g, and h are located within the range of the distance threshold value thy in the vertical direction and the distance threshold value thx in the horizontal direction, respectively. To group the labels d, e, f, g, h as the same object.

  As a result, the grouping processing unit, as shown in FIG. 6C, regardless of whether or not the torso portion of the person has been detected by the labeling process, is a rectangular person having a vertical distance yL1 and a horizontal distance xL1. Candidate areas can be detected as a grouping result.

  Next, in step S8, the person detection device 20 obtains the size (xL1 × yL1) and the aspect ratio (xL1: yL1) of the person candidate area detected by the grouping processing unit by the person determination unit, and It is determined whether or not the object in the candidate area is a person. At this time, the person determination unit determines whether or not the aspect ratio of the detected human candidate area, that is, yLn: xLn is about the person as shown in FIG. Here, when the aspect ratio of the person candidate area is within the range of 2: 1 to 3: 2 in the length: width direction, it is determined that the label in the person candidate area constitutes a person.

  In this way, the aspect ratio for determining that a label is a person does not always detect the label from the person's head to the feet, depending on the temperature, and the limbs can swing laterally when walking. Therefore, in order to make it slightly thicker than the aspect ratio of the actual person, the range of vertical: horizontal = 2: 1 to 3: 2 is set. Further, the aspect ratio for determining that the person is a person is not limited to the case where the aspect ratio is set to 2: 1 to 3: 2, but in a time zone where there are many children pedestrians, for example, 1.5: 1 As in the above, it may be set slightly longer than 1: 1.

  Furthermore, since the person walking speed is usually about 4 to 5 km / h, the person determination unit uses a plurality of person candidate areas that are detection results that are temporally continuous, and from the movement of the person, the number of continuous detections, and the like. Alternatively, an object that is detected continuously a plurality of times, does not move rapidly, and has an aspect ratio of the order of a person may be determined as a person.

  Next, in step S9, when the person determination unit determines that the object in the grouping region is a person, the person detection device 20 performs an alarm for presenting the presence of a person on a vehicle-mounted display or the like, vehicle control, or the like. And so on. At this time, the person detection device 20 may perform an alarm or vehicle control when a person is continuously detected a plurality of times (for example, three times) in front of the host vehicle 1 based on the results of steps S1 to S8. good.

  The person detection device 20 performs the above-described grouping process on the thermal image after the labeling process in step S5B as shown in FIG. 5D, and determines the vehicle from the aspect ratio and size of the grouped area. It may be determined whether or not exists. At this time, the person detection device 20 compares the preset horizontally long rectangle of the vehicle with the aspect ratio and size of the grouped area. Thereby, the person detection device 20 can also perform detection and type determination of other vehicles in front of the host vehicle 1.

  As described above in detail, according to the human detection device 20 according to the first embodiment to which the present invention is applied, a human temperature region obtained by binarizing a thermal image with a temperature threshold for human detection, A high temperature region binarized with a temperature threshold value for detecting a high temperature region is detected, and a human temperature region around the high temperature region is removed from the person determination processing target among the regions detected as the human temperature. Thus, the person determination can be performed by reliably distinguishing the area other than the person such as the preceding vehicle and the background object from the person area.

  Therefore, according to this person detection device 20, even if the approximate center position of the lamp part or the muffler part of the preceding vehicle is a high temperature region, and there is a region equivalent to the person temperature around the high temperature region. The person and the vehicle can be distinguished without using the symmetry of the shape peculiar to the vehicle, and the other vehicle and the person can be reliably connected regardless of the direction of the other vehicle with respect to the own vehicle 1 and the shape of the other vehicle. Can be distinguished.

[Second Embodiment]
Next, a human detection device 20 according to a second embodiment to which the present invention is applied will be described. In addition, about the same part as the content mentioned above, the detailed description is abbreviate | omitted by using the same code | symbol and the same name.

  As shown in FIG. 8, the person detection device 20 according to the second embodiment inputs the thermal image data from the far-infrared camera 10 in step S <b> 1 in the same manner as the person detection device 20 according to the first embodiment. / F, an image memory for storing thermal image data in step S2, a temperature threshold setting unit for setting a temperature threshold for person detection in step S3 ', and a temperature threshold for person detection in step S4 A binarization processing unit (binarization processing means) that performs binarization processing, a labeling processing unit (region detection means) that performs labeling processing using a binarized image in step S5, and grouping in step S7 A grouping processing section (grouping means) for detecting a person candidate area, and a person determination section (person determination means) for performing person determination in step S8.

  Here, unlike the above-described temperature threshold value setting unit, the person detection device 20 sets only a temperature threshold value for person detection, and binarizes using only the temperature threshold value for person detection. Process. That is, in the second embodiment, the binarization process and the labeling process for the high temperature region are not performed. In addition, in step S11 after step S5, the person detection device 20 detects a region (label) to be subjected to grouping by removing a high temperature region according to processing contents different from step S6 described above. An area detection unit is provided.

  In such a person detection device 20, when the thermal image shown in FIG. 3A is captured by the far-infrared camera 10 (step S1) and stored in the image memory (step S2), the temperature for detecting the person is set. A threshold value is set (step S3 ′), and a binarized image shown in FIG. 5A is acquired by performing binarization processing (step S4). Then, labeling processing is performed to obtain labels a to h as shown in FIG. 5B (step S5).

  Next, in step S11, the person detection apparatus 20 takes out the luminance value of each pixel corresponding to the labels a to h obtained by the labeling process from the thermal image stored in step S2 by the grouping target area detection unit, If the luminance value of the extracted thermal image includes a luminance value that is higher than the human temperature, the label including the pixel of the luminance value is removed.

  Specifically, first, as shown in FIG. 9A, the grouping target area detection unit sets contour lines at the outer ends of the labels a to h, and obtains luminance values of pixels surrounded by the contour lines. To do. Note that the labels a, b, and c are donut-shaped, but the luminance values of the substantially central pixels that are not detected by the binarization process are also acquired. At this time, the grouping target area detection unit reads the thermal image stored in step S2 and superimposes the contour lines set at the outer ends of the labels a to h.

  Then, as shown in FIG. 9B, the grouping target area detection unit investigates the brightness value for each label. At this time, the grouping target area detection unit uses the label a larger than the edge part of the lamp of the vehicle as the investigation area, sequentially acquires the luminance value of each pixel according to a predetermined investigation direction, and the luminance value is, for example, a person detection It is determined whether or not a predetermined luminance value that is about the temperature threshold value for use is exceeded. When it is determined that a high temperature region exists in the label, the label is removed from the grouping target. Thus, the grouping target area detection unit removes the labels a and c obtained from the lamp and the label b obtained from the muffler, and labels d, e, f, and g of the temperature of the person not including the high temperature area. , H and the image shown in FIG.

  Further, by performing the process of step S11, it becomes possible to detect the label of the vehicle lamp or the muffler as in the first embodiment described above, and based on the size, shape, etc. of the label, the vehicle 1 It can be detected that there is another vehicle ahead.

  Next, in the same manner as described above, the person detection device 20 performs the grouping process in step S7 to detect the person candidate area, and performs the person determination process in step S8 to determine whether or not the object in the person candidate area is a person. In step S9, vehicle control and warning are performed.

  As described above in detail, according to the person detection device 20 to which the present invention is applied, a label is created by performing the binarization process created using the temperature threshold for person detection, and the label is heated to a high temperature. When a pixel in a region is included, the person can be detected by removing the label and removing it from the person determination processing target. Therefore, as in the first embodiment, the vehicle and the person A person can be detected with a certain distinction.

  Further, according to the person detection device 20, unlike the first embodiment described above, the binarization process and the labeling process need only be performed once, and the vehicle has a simpler configuration compared to the first embodiment. It is possible to detect a person in distinction.

  Furthermore, according to this person detection device 20, since it is possible to determine whether or not a person is a person by examining the luminance values of the pixels included in the label, the very small label includes a high temperature region. It is also possible to detect that the label is a vehicle.

[Third Embodiment]
Next, a human detection device 20 according to a third embodiment to which the present invention is applied will be described. In addition, about the same part as the content mentioned above, the detailed description is abbreviate | omitted by using the same code | symbol and the same name.

  When the area of the label detected by the labeling process is smaller than a predetermined value, the person detection apparatus 20 according to the third embodiment performs the grouping target area detection process described in the second embodiment. When the area of the label is larger than a predetermined value, the grouping target area detection process described in the first embodiment is performed.

  That is, as shown in FIG. 10, the person detection apparatus 20 performs an image including a plurality of labels as shown in FIG. 5B by performing the processes in steps S1 to S5 described in the second embodiment. In step S21, it is determined whether the area of each label is equal to or greater than a predetermined value. Here, the predetermined value refers to processing for determining whether or not a high-temperature region is included in the region of the thermal image surrounded by the outline of the label in step S11, and for high-temperature region detection in steps S3B to S6 in FIG. The temperature threshold value is set by comparing the processing amount with the processing consisting of binarization processing, labeling processing, and grouping target area detection processing.

  The person detection device 20 then proceeds to step S11 if the label area is small such that the processing amount when the processing of step S11 is smaller than the processing amount when the processing of steps S3B to S6 is performed. Proceed with the process. On the other hand, when the area of the label is large such that the processing amount when the processing of step S11 is larger than the processing amount when the processing of steps S3B to S6 is performed, the processing proceeds to steps S3B to S6. .

  Then, by performing the processing of step S11 or the processing of steps S3B to S6, the label obtained by the lamp or the muffler can be removed, and an image including only the person's label can be created.

  As described above in detail, according to the human detection device 20 according to the third embodiment, the presence of a high temperature region is determined by examining the luminance value of each pixel of the thermal image included in the label as in the second embodiment. Of the processing to check and the processing to check the existence of the high temperature region by performing the binarization processing using the temperature threshold value for detecting the high temperature region, it is possible to select a processing with a small amount of processing, so that real-time characteristics can be selected. A vehicle and a person can be reliably distinguished without lacking.

[Fourth Embodiment]
Next, a human detection device 20 according to a fourth embodiment to which the present invention is applied will be described. In addition, about the same part as the content mentioned above, the detailed description is abbreviate | omitted by using the same code | symbol and the same name.

  As shown in FIG. 11, the person detection apparatus 20 according to the fourth embodiment performs steps S1 to S5 as described in the second embodiment, and removes only a label having a small area in step S31. After the grouping process of step S7 is performed and the person candidate area is detected, in step S32, the person candidate area including the high temperature area is removed.

  In step S7 after the labels a to h as shown in FIG. 12A to be grouped in step S31 are detected in step S31, the person detecting device 20 uses a distance threshold value in step S7. The grouping process is performed using.

  Here, in the above-described embodiment, the high temperature region is removed before the grouping process, and at the time of performing the grouping process, only the label obtained from the temperature of the person is used, but in the fourth embodiment, Labels obtained in the high temperature region are also subject to grouping processing. When the grouping process is performed by determining whether the distance between the labels is within the distance threshold value in the grouping processing unit, the person candidate area for the object A including the labels a and b The person candidate area for the object B composed of the label c and the person candidate area for the object C composed of the labels d to h are detected.

  Next, in step S32, the person detection device 20 removes the person candidate area including the high temperature area from the person candidate areas detected in step S7. At this time, as in the case shown in FIG. 9B described above, the person detecting device 20 uses each person candidate area as an investigation area, acquires a luminance value according to a predetermined investigation direction, and the luminance value is, for example, It is determined whether or not a predetermined luminance value corresponding to a temperature threshold for person detection is exceeded. If it is determined that a high temperature area exists in the person candidate area, the person candidate area is removed from the target of the person determination process in step S8. Thereby, the person detection device 20 removes the object A in the human candidate area including the lamps and the labels a and b obtained from the lamps and the object B in the human candidate area including the labels c obtained from the lamps. An image including only the object C of the human candidate area not including the area is created.

  In addition, as shown in FIG. 12B, the person detection device 20 extracts a label of a person candidate area that is determined to have a pixel in the high temperature area after the grouping process, and You may determine whether a vehicle exists from an aspect ratio and a magnitude | size. At this time, the person detection device 20 compares the aspect ratio and size of the pre-set laterally long rectangle of the vehicle with the aspect ratio and size of the area after grouping, and determines that a high temperature area exists, thereby The detection and type determination of other vehicles can also be performed.

  According to the person detection device 20 according to the fourth embodiment, when a high temperature area exists in a part of the person candidate area obtained by grouping the temperature labels corresponding to the person, the person candidate area Is excluded from the person determination process, it is possible to prevent the erroneous determination of determining the vehicle as a person as described above, and to reliably determine that the vehicle is a vehicle.

  As described above, when determining whether or not a vehicle is in the shape of a horizontally-long rectangle that is set in advance, for example, for the left and right lamps, there is a vehicle in which the high temperature region is observed only on the right side and not on the left side. In determining whether a high temperature region exists in the label, there is a possibility that the left lamp is erroneously determined as a person. On the other hand, the person detection device 20 uses a horizontally long distance threshold value for the vehicle, and first groups the left and right areas corresponding to the ramp as one object, and the grouped candidate areas are included in the group. It is possible to determine that the vehicle is a vehicle based on whether or not there is a high temperature region, and it is possible to prevent erroneous determination such that the right side including the high temperature is the vehicle and only the left side is the person.

[Fifth Embodiment]
Next, a human detection device 20 according to a fifth embodiment to which the present invention is applied will be described. In addition, about the same part as the content mentioned above, the detailed description is abbreviate | omitted by using the same code | symbol and the same name.

  In the above-described embodiment, it is possible to detect not only a person but also a vehicle by including a pixel in a high temperature region and determining that it is a vehicle when the size and shape are about the vehicle. However, the person detection device 20 according to the fifth embodiment determines that there is a vehicle ahead of the host vehicle 1 based on the results of the person and vehicle determination processing up to the previous time. Is.

  Here, since both the person and the other vehicle move with respect to the own vehicle 1 at a certain speed or less, the object continuously detected by the processing up to the previous time is based on the moving direction and moving speed of the object. Thus, the next detected position can be predicted. That is, the person detection device 20 stores the detection positions of the person and the vehicle that are temporally continuous by storing the determination results of the person and the vehicle over a plurality of times.

  In the person and vehicle determination process, the person detection device 20 predicts the detection position in the current process based on the detection positions of the person and the vehicle that are temporally continuous, and groups them with the predicted detection position. The detected position of the human candidate area obtained in this way is compared to determine whether the object is a person or a vehicle. That is, with respect to an object that is determined to be a vehicle by continuously detecting a high-temperature region that is equal to or higher than the human temperature for a certain time or longer, the object after the grouping process is again performed at the past detection position or the predicted detection position. When the area is detected, the vehicle can be determined even if the high temperature area is not detected.

  As described above, according to the human detection device 20 according to the fifth embodiment, a high-temperature region is detected for an object detected in the past and the high-temperature region is detected continuously for a certain time or longer. Even if not, it can be determined that the person is not a person. Therefore, according to this person detection device 20, since the entire body of the vehicle is not a high temperature region but a high temperature region exists in a part of a light, a muffler, etc., the direction of the other vehicle with respect to the own vehicle 1 changes, Even if the high-temperature region is not included in the thermal image, if it is determined that the high-temperature region is detected in the past and is not a person, it can be determined that the person is not a person based on the detection position of the object, An erroneous determination that determines that the vehicle is a person can be prevented.

[Sixth Embodiment]
Next, a human detection device 20 according to a sixth embodiment to which the present invention is applied will be described. In addition, about the same part as the content mentioned above, the detailed description is abbreviate | omitted by using the same code | symbol and the same name.

  The human detection device 20 according to the sixth embodiment is characterized in that when the moving speed of an object in a thermal image is equal to or higher than a predetermined speed, it is determined that the object is not a person. .

  Here, normally, even when a person walks and runs at a speed of about 5 km / h, the speed is about 20 km / h. On the other hand, the vehicle travels at a speed of about 40 km / h to 60 km / h even on a general road, depending on the court speed, road type, and situation. Accordingly, the person detection device 20 detects an object that moves in the lateral direction in the thermal image when the host vehicle 1 approaches a T-junction or an intersection, for example, and the moving speed of the object is 40 km / h or more. If it is determined that the moving speed is not a person, the object is determined to be a vehicle.

At this time, as shown in FIG. 13A, the person detection device 20 detects the temperature of the person on the imaging surface 10b via the camera lens 10a, and the distance z from the host vehicle 1 to the person and the focal length Depending on f, the person height hc (= number of pixels) appearing on the imaging surface 10b differs with the person height H. Specifically, the person height hc in the image shown in FIG.
hc = f · H / z (Formula 1)
It is expressed by the following arithmetic expression.

  That is, from the above formula 1, as the distance z between the host vehicle 1 and the person increases, the person height hc in the thermal image decreases and the height of the person existing near the host vehicle 1 in the image. The height hc in the image of the person existing far from the host vehicle 1 is smaller than hc.

  Therefore, in the human detection device 20, the height of the object in the thermal image changes as shown in Expression 1 according to the actual distance between the object and the far-infrared camera 10. That is, the actual object height H is set to 160 cm to 170 cm, which is about the same as that of a normal person, and is substituted into Equation 1 to obtain the person height hc in the thermal image. The approximate distance z to the person can be obtained.

  Further, the movement of the object in the thermal image at the X-axis coordinate position can be converted into a movement distance in real space, as shown in FIG. That is, when the object detected on the thermal image is determined to exist at the position of the distance z by the principle of Equation 1, and the X coordinate position of the object in the thermal image moves from xp to xp ′, The movement distance XP-XP ′ in the real space is obtained by the following equation as shown in FIG.

XP−XP ′ = (z · xp−z · xp ′) / f (Formula 2)
FIG. 14A is present at a forward distance z from the camera lens 10a, and a point P at a position where the distance from the optical axis is XP is a coordinate xp on the thermal image as shown in FIG. 14B. As shown in FIG. 14B, a point P ′ at a position which is present at a forward distance z from the camera lens 10a and whose distance from the optical axis is XP ′ appears as coordinates xp ′ on the thermal image. It is shown that.

  Accordingly, when the X-axis coordinate moves from xp to xp ′ in the thermal image that moves back and forth in time, the actual object movement distance XP-XP ′ is obtained multiple times, and the thermal image is acquired and moved. The moving speed of the object is obtained by dividing the result of Equation 2 for obtaining the distance by the unit time. Of the objects in the thermal image, in order to obtain the moving speed of the vehicle, the value of H in Equation 1 is set to about the vehicle.

  As described above, according to the person detection device 20, when a high temperature region is detected in a part of the object as described above and the moving speed is equal to or higher than the predetermined speed, it is determined that the object is not a person. Can do. Further, even when the other vehicle crosses in front of the host vehicle 1 and a high temperature region is not detected in a part of the object, such as when a ramp or a muffler is not detected, the moving speed of the object is equal to or higher than a predetermined speed. Since it is possible to determine that the object is not a person, it is possible to perform higher-speed processing than in the case of detecting a high-temperature region.

  In other words, it is desirable that an object with a high moving speed is shorter than usual from the time it appears on the thermal image until it is outside the thermal image, and an object with a high moving speed is usually able to be detected quickly. Since the determination can be made by a simple process of determining the moving speed of the object, the person and the vehicle can be determined at high speed.

  Further, according to the person detection device 20, when the moving speed of the object in the thermal image is high, the high temperature region is easily detected at a low temperature, and there is a risk of determining that the moving object is a person. In the case of the above, it can be determined that the person is not a person, so that it is possible to prevent an object other than a person from being erroneously determined to be a person.

  The above-described embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and various modifications can be made depending on the design and the like as long as the technical idea according to the present invention is not deviated from this embodiment. Of course, it is possible to change.

It is a figure which shows schematic structure of the own vehicle carrying the person detection apparatus to which this invention is applied, Comprising: (a) is a side view, (b) is a top view. It is a flowchart for demonstrating the process structure which detects the person and vehicle ahead of the own vehicle, and the functional structure of a person detection apparatus by the person detection apparatus which concerns on 1st Embodiment to which this invention is applied. It is a figure explaining the high temperature area | region which exists in a vehicle, Comprising: (a) is a thermal image containing a person and a preceding vehicle, (b) is a thermal image which expanded the lamp | ramp part of the vehicle. It is a figure for demonstrating a binarization process, (a) is a thermal image containing a person and a front vehicle, (b) is a temperature threshold value for person detection, (c) is (a). (D) is a temperature threshold for detecting a high temperature region, (e) is a thermal image of (a). It is the image after binarization with the temperature threshold value of (d). It is a figure for demonstrating the labeling process and noise removal process with respect to the thermal image after binarization, (a) is an image binarized with the temperature threshold value for person detection, (b) is ( (a) An image after labeling processing and noise removal processing are performed on the image, (c) is an image binarized with a temperature threshold value for detecting a high temperature region, and (c) in (d) (E) is an image obtained by subtracting a high temperature region from the image of (b). It is a figure for demonstrating a grouping process, (a) is the image after a labeling process, (b) is an image which applied the distance threshold value to the image after a labeling process, (c) is a grouping process. It is the image after doing. It is a figure for demonstrating a grouping process. It is a flowchart for demonstrating the process structure which detects the person and vehicle ahead of the own vehicle, and the functional structure of a person detection apparatus by the person detection apparatus which concerns on 2nd Embodiment to which this invention is applied. It is a figure for demonstrating the process which detects the label made into the object of grouping, Comprising: (a) is a figure which shows the case where the outline is set to the outer end of a label, (b) is a figure for every label, It is a figure explaining investigating a brightness | luminance value, (c) is a figure for demonstrating producing the image remove | excluding the high temperature area | region from the image of (a). It is a flowchart for demonstrating the process structure which detects the person and vehicle ahead of the own vehicle, and the functional structure of a person detection apparatus with the person detection apparatus which concerns on 3rd Embodiment to which this invention is applied. It is a flowchart for demonstrating the process structure which detects the person and vehicle ahead of the own vehicle, and the functional structure of a person detection apparatus with the person detection apparatus which concerns on 4th Embodiment to which this invention is applied. It is a figure for demonstrating the grouping process in 4th Embodiment to which this invention is applied, Comprising: (a) is an image after the labeling process containing a high temperature area | region and a person, (b) is an image of (a). Is an image obtained by performing a grouping process on. It is a figure for demonstrating the process which calculates | requires the person height which appears in a thermal image, Comprising: (a) is a principle figure, (b) demonstrates that the person height in a thermal image changes with the distance from the own vehicle. It is a figure for doing. It is a figure for demonstrating the process which calculates | requires the position of the X-axis direction which appears in a thermal image, Comprising: (a) is a principle figure, (b) is a coordinate in the X-axis direction in a thermal image changes with a movement of an object. It is a figure for demonstrating this.

Explanation of symbols

1 own vehicle 10 far-infrared camera 20 person detection device

Claims (9)

Imaging means for capturing a thermal image composed of pixels having luminance values corresponding to the temperature of the object;
Of the thermal image captured by the imaging means, pixels within the upper and lower threshold values of the luminance value corresponding to the person's temperature, and the upper limit threshold of the luminance value corresponding to the person's temperature. A binarization processing means for human detection that creates a binarized image for human detection in which a pixel outside the range of the value and the lower threshold is distinguished;
Referring to the binarized image for human detection, a region in which the pixel group within the range between the upper threshold value and the lower threshold value of the luminance value corresponding to the human temperature is detected is a predetermined area or more is detected. Area detecting means for detecting a person to perform,
Grouping means for grouping one or a plurality of areas existing within a predetermined distance threshold among the areas detected by the area detection means as person candidate areas;
Based on the area and shape of the person candidate area grouped by the grouping means, and a person determination means for determining whether the object of the person candidate area is a person,
Of the thermal image captured by the imaging unit, except for a high temperature region higher than the temperature of the person, including a high temperature region removal unit to be processed by the person determination unit ,
Of the thermal image captured by the imaging means, the pixels within the upper and lower thresholds of the luminance value corresponding to the high temperature region higher than the temperature of the person, and the luminance corresponding to the high temperature region Binarization processing means for detecting a high temperature region for creating a binarized image for detecting a high temperature region in which pixels outside the upper limit threshold value and the lower threshold value are distinguished from each other;
With reference to the binarized image for detecting the high temperature region, a region in which the pixel group within the range between the upper threshold value and the lower threshold value of the luminance value corresponding to the high temperature region is a predetermined area or more It further comprises an area detection means for detecting a high temperature area,
The high temperature region removing unit is configured to detect the person when the region detected by the region detecting unit for human detection includes a region detected by the region detecting unit for detecting the high temperature region. A person detecting apparatus, wherein an area detected by an area detecting means is removed . Imaging means for capturing a thermal image composed of pixels having luminance values corresponding to the temperature of the object;
Of the thermal image captured by the imaging means, pixels within the upper and lower threshold values of the luminance value corresponding to the person's temperature, and the upper limit threshold of the luminance value corresponding to the person's temperature. A binarization processing means for human detection that creates a binarized image for human detection in which a pixel outside the range of the value and the lower threshold is distinguished;
Referring to the binarized image for human detection, a region in which the pixel group within the range between the upper threshold value and the lower threshold value of the luminance value corresponding to the human temperature is detected is a predetermined area or more is detected. Area detecting means for detecting a person to perform,
Grouping means for grouping one or a plurality of areas existing within a predetermined distance threshold among the areas detected by the area detection means as person candidate areas;
Based on the area and shape of the person candidate area grouped by the grouping means, and a person determination means for determining whether the object of the person candidate area is a person,
Of the thermal image captured by the imaging unit, except for a high temperature region higher than the temperature of the person, including a high temperature region removal unit to be processed by the person determination unit ,
The high temperature region removing unit detects the person detection when a pixel having a luminance value corresponding to a high temperature region higher than the temperature of the person is included in the region detected by the human detection region detecting unit. A human detection apparatus that removes an area detected by an area detection means . The high temperature region removing means includes
If the area of the area detected by the person detecting area detecting means is greater than or equal to a predetermined value, the area detecting means for detecting the high temperature area is added to the area detected by the person detecting area detecting means. If the detected area is included, perform a process of removing the area detected by the person detection area detection means,
If the area of the area detected by the person detection area detection means is not equal to or greater than a predetermined value, the area detected by the person detection area detection means is in a high temperature area higher than the temperature of the person. The person detection apparatus according to claim 1 , wherein when a pixel having a corresponding luminance value is included, a process of removing the area detected by the person detection area detection unit is performed. Imaging means for capturing a thermal image composed of pixels having luminance values corresponding to the temperature of the object;
Of the thermal image captured by the imaging means, pixels within the upper and lower threshold values of the luminance value corresponding to the person's temperature, and the upper limit threshold of the luminance value corresponding to the person's temperature. A binarization processing means for human detection that creates a binarized image for human detection in which a pixel outside the range of the value and the lower threshold is distinguished;
Referring to the binarized image for human detection, a region in which the pixel group within the range between the upper threshold value and the lower threshold value of the luminance value corresponding to the human temperature is detected is a predetermined area or more is detected. Area detecting means for detecting a person to perform,
Grouping means for grouping one or a plurality of areas existing within a predetermined distance threshold among the areas detected by the area detection means as person candidate areas;
Based on the area and shape of the person candidate area grouped by the grouping means, and a person determination means for determining whether the object of the person candidate area is a person,
Of the thermal image captured by the imaging unit, except for a high temperature region higher than the temperature of the person, including a high temperature region removal unit to be processed by the person determination unit ,
The high temperature region removing unit includes a grouping unit when a pixel having a luminance value corresponding to a high temperature region higher than the temperature of the person is included in the person candidate region grouped by the grouping unit. A human detection device that removes human candidate regions that are grouped by . The person determination means detects the person even if the high-temperature area is not detected in the area after it is detected that the area detected by the person detection area detection means includes a high-temperature area continuously. human detection apparatus according to any one of claims 1 to 4, characterized in that determining that the object of the other times. The person determination means determines that an object in the area is an object other than a person when the movement speed of the area detected by the person detection area detection means is equal to or higher than a predetermined speed. human detection apparatus according to any one of claims 1 to 4. An imaging step of imaging means for imaging a thermal image composed of pixels of the luminance value corresponding to the temperature of the object,
Among the thermal images picked up by the image pickup means by the CPU, pixels within the range of the upper and lower thresholds of the luminance value corresponding to the person's temperature, and the luminance value corresponding to the person's temperature A binarization processing step for human detection for creating a binarized image in which pixels outside the upper threshold and lower threshold are distinguished from each other;
Wherein the CPU, by referring to the binary image, detecting a region where the pixel group is equal to or greater than a predetermined area within the range between the upper limit and lower thresholds of the luminance value corresponding to the temperature of the person An area detection step for detecting a person to perform,
A grouping step in which the CPU groups one or a plurality of regions existing within a predetermined distance threshold among the detected regions as a person candidate region;
The CPU has a person determination step of determining whether or not the object of the person candidate area is a person based on the area and shape of the grouped person candidate areas,
After the CPU creates the binarized image, a high temperature region removal step to be subject to the person determination process is provided except for a high temperature region higher than the temperature of the person in the thermal image ,
Of the thermal image captured by the imaging means, the CPU includes pixels within a range of an upper threshold value and a lower threshold value corresponding to a high temperature region higher than the temperature of a person, and the high temperature region A binarization processing step for high-temperature region detection that creates a binarized image for high-temperature region detection in which pixels outside the range between the upper threshold value and lower threshold value of the luminance value corresponding to
The CPU refers to the binarized image for detecting the high temperature region, and a pixel group within the upper limit threshold and lower limit threshold of the luminance value corresponding to the high temperature region becomes a predetermined area or more. And a region detecting step for detecting a high temperature region,
When the CPU excludes a high temperature area higher than the temperature of the person, the area detected in the area detection step for detecting a person includes the area detected in the area detection step for detecting the high temperature area. A person detecting method , wherein the area detected in the person detecting area detecting step is removed . An imaging step of imaging means for imaging a thermal image composed of pixels of the luminance value corresponding to the temperature of the object,
Among the thermal images picked up by the image pickup means by the CPU, pixels within the range of the upper and lower thresholds of the luminance value corresponding to the person's temperature, and the luminance value corresponding to the person's temperature A binarization processing step for human detection for creating a binarized image in which pixels outside the upper threshold and lower threshold are distinguished from each other;
The CPU refers to the binarized image and detects a region in which a pixel group within a range between an upper limit threshold value and a lower limit threshold value corresponding to a person's temperature is equal to or larger than a predetermined area. An area detection step for detecting a person to perform,
A grouping step in which the CPU groups one or a plurality of regions existing within a predetermined distance threshold among the detected regions as a person candidate region;
The CPU has a person determination step of determining whether or not the object of the person candidate area is a person based on the area and shape of the grouped person candidate areas,
After the CPU creates the binarized image, a high temperature region removal step to be subject to the person determination process is provided except for a high temperature region higher than the temperature of the person in the thermal image ,
When the CPU excludes a high temperature area higher than the person's temperature, the area detected in the person detection area detection step includes a pixel having a luminance value corresponding to a high temperature area higher than the person's temperature. is the case is, the person detecting method characterized by removing the above is detected in the area detecting step for person detection area. An imaging step of imaging means for imaging a thermal image composed of pixels of the luminance value corresponding to the temperature of the object,
Among the thermal images picked up by the image pickup means by the CPU, pixels within the range of the upper and lower thresholds of the luminance value corresponding to the person's temperature, and the luminance value corresponding to the person's temperature A binarization processing step for human detection for creating a binarized image in which pixels outside the upper threshold and lower threshold are distinguished from each other;
The CPU refers to the binarized image and detects a region in which a pixel group within a range between an upper limit threshold value and a lower limit threshold value corresponding to a person's temperature is equal to or larger than a predetermined area. An area detection step for detecting a person to perform,
A grouping step in which the CPU groups one or a plurality of regions existing within a predetermined distance threshold among the detected regions as a person candidate region;
The CPU has a person determination step of determining whether or not the object of the person candidate area is a person based on the area and shape of the grouped person candidate areas,
After the CPU creates the binarized image, a high temperature region removal step to be subject to the person determination process is provided except for a high temperature region higher than the temperature of the person in the thermal image ,
The high temperature region removing step includes the grouping step when pixels of luminance values corresponding to a high temperature region higher than the temperature of the person are included in the person candidate regions grouped by the grouping step. A person detection method characterized in that person candidate regions grouped by the method are removed .