JP5948983B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program that process a frame image of an imaging area captured by an infrared camera to generate a background difference image.

  2. Description of the Related Art Conventionally, a monitoring system for monitoring the intrusion of a suspicious person or leaving a suspicious object using an imaging device such as a video camera has been put into practical use. In this type of monitoring system, the imaging area of the imaging device is matched with a monitoring target area for monitoring the intrusion of a suspicious person or the leaving of a suspicious object. In addition, the image processing apparatus includes an image processing apparatus that processes a frame image of the monitoring target area captured by the imaging apparatus and detects an imaged object (such as a suspicious person or a suspicious object).

  For each pixel of the input frame image, the image processing apparatus determines whether the background pixel is a background pixel in which a background is captured or a foreground pixel in which an object that is not a background is captured. Based on the determination result, the image processing apparatus distinguishes a background difference image (binarized image) from a background region where a background is imaged and a foreground region where an object (such as a person or a vehicle) is imaged. Is generated. The foreground area in the background difference image is an area where an object is imaged.

  On the other hand, in a relatively dark place, the visible light camera has a shortage of exposure and cannot capture an object that is a detection target. In view of this, an apparatus for processing a frame image (thermal image) captured by a far-infrared camera and detecting a person being captured has been proposed (see Patent Document 1).

  This patent document 1 describes, for each pixel of a thermal image captured by a far-infrared camera, a pixel within a range between an upper threshold value and a lower threshold value of a luminance value (pixel value) corresponding to a person's temperature; In this configuration, a binarized image distinguished from pixels that are not within this range is generated, and the person being imaged is detected.

JP 2006-101384 A

  However, when it is raining, a person or vehicle that is the object to be detected gets wet in the rain, so that the contrast of the object such as a person or vehicle captured in the frame image captured by the far-infrared camera is lowered. For this reason, during clear weather or cloudy weather, even if an object such as a person or vehicle captured in a frame image captured by a far-infrared camera can be detected, it is captured in a frame image captured by a far-infrared camera during rainy weather. An object such as a person or a vehicle may not be detected.

  A change in weather in the imaging area of the far-infrared camera sometimes reduces the detection accuracy of an object such as a person or a vehicle captured in the frame image captured by the far-infrared camera.

  An object of the present invention is to suppress a decrease in detection accuracy of an object such as a person or a vehicle captured in a frame image of an imaging area captured by an infrared camera due to a change in weather in the imaging area of the infrared camera. An image processing apparatus, an image processing method, and an image processing program are provided.

  The image processing apparatus of the present invention is configured as follows in order to solve the above-described problems and achieve the object.

  A frame image of the imaging area captured by the infrared camera is input to the image input unit. For example, the infrared camera captures a frame image of 10 to 60 frames per second, and inputs the captured frame image to the image input unit.

  The background model storage unit stores a background model obtained by modeling the frequency of the pixel value of each pixel of the frame image input to the image input unit. The background model is obtained by modeling the frequency of the pixel value of each pixel of a frame image input in the past, for example, and is represented by a Gaussian density function or the like. The background model may be updated using the frame image every time the frame image is input to the image input unit.

  The background difference image generation unit, for each pixel of the frame image input to the image input unit, whether the pixel is a foreground pixel or a background pixel, a background model that the background model storage unit stores for the pixel To determine a background difference image. Specifically, for each pixel of the frame image input to the image input unit, the background difference image generation unit determines that the pixel value of the pixel is the background pixel if the frequency of the pixel in the background model is higher than the threshold value. If the frequency of the pixel in the background model is lower than the threshold value, the pixel is determined to be a foreground pixel. That is, the background difference image generation unit determines a pixel whose frequency of occurrence of the pixel value is lower than the threshold as a foreground pixel, and determines a pixel of the frequency of occurrence of the pixel value as higher than the threshold as a background pixel.

  The threshold setting unit sets a threshold that is a reference for determining whether each pixel of the frame image input to the image input unit by the background difference image generation unit is a foreground pixel or a background pixel. The threshold value setting unit calculates and sets the threshold value based on the distribution of pixel values of pixels determined to be background pixels by the background difference image generation unit.

  Since the road surface and the like get wet in the rain, the pixel values of the pixels that image the road surface and the like that are determined to be the background pixels are concentrated on a certain value. In addition, as the amount of rainfall increases, the wetness of the road surface and the like approaches uniformly, and the temperature of the road surface also becomes substantially uniform, so the pixel value of the pixel that images the road surface and the like determined to be the background pixel has a certain value The degree to concentrate on increases. This degree is obtained from, for example, the distribution of the distribution of pixel values for pixels that are determined to be background pixels by the background difference image generation unit.

  For example, the threshold setting unit calculates and sets the threshold based on the distribution of the pixel value distribution for the pixels determined to be the background pixel by the background difference image generation unit, so that the threshold setting according to the rainfall amount is set. Yes. The threshold setting unit may reduce the frequency threshold for determining whether the pixel is a background pixel or a foreground pixel as the amount of rainfall increases.

  As a result, with respect to the frame image input to the image input unit, the accuracy of determining whether the object is a foreground pixel in which an object is imaged or a background pixel in which the object is not imaged is It is possible to suppress a drop due to changes in the weather in the area. Therefore, it is possible to suppress a decrease in detection accuracy of an object such as a person or a vehicle captured in a frame image of the imaging area captured by the infrared camera due to a change in the weather of the imaging area of the infrared camera.

  In addition, you may provide the object detection part which detects the imaged object from the background difference image which the background difference image generation part produced | generated.

  The threshold setting unit is a pixel that is determined by the background difference image generation unit to be a background pixel, and that is located within a threshold calculation region that is set in advance for the frame image input to the image input unit. The threshold value may be calculated and set using the pixel value.

  In this case, an area wetted by rain when it rains may be set as a threshold calculation area. In other words, by providing a roof or the like, an area that does not get wet even when it is raining may be excluded from the threshold calculation area. Thereby, a threshold value can be set according to rainfall.

  Further, in order to prevent erroneous detection of sunlight reflected by a puddle or metal as an object, the following configuration may be added.

  A range of pixel values of the specular pixel is stored in the specular pixel value storage unit, and in the frame image input to the image input unit, the pixel value is determined as a specular pixel stored in the specular pixel value storage unit. A specular pixel determination unit that determines pixels within the range as specular pixels is provided. Furthermore, a specular region setting unit is provided for setting a specular region based on the distribution of specular pixels in the frame image input to the image input unit.

  Then, the object detection unit is configured not to detect the object being imaged for the mirror surface region set by the mirror surface region setting unit. Thereby, it is possible to prevent erroneous detection of sunlight reflected by a puddle or metal as an object.

  Further, the specular pixel determination unit continues for a predetermined first time, and in the frame image input to the image input unit, the pixel value is determined as the specular pixel stored in the specular pixel value storage unit. It is preferable to determine a pixel within the range as a mirror pixel. Accordingly, it is possible to prevent a pixel in which noise is temporarily generated or a pixel in which sunlight or the like temporarily reflected by an object is imaged as a specular pixel.

  The specular pixel determination unit continues the second predetermined time for the pixel determined to be a specular pixel, and the specular pixel value storage unit stores the pixel value in the frame image input to the image input unit. When the pixel value is outside the range of the pixel value to be determined as the mirror surface pixel to be stored, it is preferable to determine that the pixel is not a mirror surface pixel. It can be prevented that the mirror pixel is determined not to be a mirror pixel due to temporary noise.

  Moreover, you may make it determine whether the object which the object detection part detected is a person or objects, such as vehicles other than a person. In this case, based on the distribution of pixel values of pixels located in the foreground region set based on the foreground pixels corresponding to the object detected by the object detection unit, an object determination unit that determines whether the object is a person What is necessary is just to provide.

  Since the amount of heat radiated varies depending on the part of a person, the pixel values of the pixels that image the person are dispersed to some extent. On the other hand, since the amount of heat radiated from a non-human object is uniform throughout, the pixel values of the pixels that image the non-human object are concentrated on a certain value. Therefore, it is possible to determine whether or not this object is a person from the distribution of pixel values of pixels corresponding to the detected object.

  Further, if the size of the detected object is smaller than a predetermined size, it may be determined that the object is not imaged in this region, and the image input unit continues to the predetermined third time. It is good also as a structure which does not detect the object which was not detected from the input frame image as the imaged object.

  According to the present invention, the detection accuracy of an object such as a person or a vehicle captured in the frame image of the imaging area captured by the infrared camera is suppressed from being lowered due to a change in the weather of the imaging area of the infrared camera. be able to.

It is a block diagram which shows the structure of the principal part of an image processing apparatus. It is a figure explaining the background model of a certain pixel. It is a figure which shows the imaging area of a far-infrared camera. It is a figure which shows distribution of the pixel number with respect to a pixel value in the frame image which imaged the sky at the time of fine weather with a far-infrared camera. It is a flowchart which shows operation | movement of an image processing apparatus. It is a flowchart which shows a mirror surface pixel area | region setting process. It is a flowchart which shows a background difference image generation process. It is a histogram which shows distribution of the pixel value about a background pixel. It is a flowchart which shows an object detection process. It is a flowchart which shows a classification determination process.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing a configuration of a main part of the image processing apparatus. The image processing apparatus 1 includes a control unit 2, an image input unit 3, an image processing unit 4, and an input / output unit 5.

  The control unit 2 controls the operation of each part of the main body of the image processing apparatus 1.

  A frame image captured by the far-infrared camera 10 is input to the image input unit 3. The far-infrared camera 10 is installed such that a monitoring target area for monitoring the intrusion of an object such as a person or a vehicle is within the imaging area. The far-infrared camera 10 captures a frame image of 10 to 60 frames per second, and inputs the captured frame image to the image input unit 3.

  The image processing unit 4 processes the frame image input to the image input unit 3 and detects an object (non-background object) captured in the frame image. The image processing unit 4 has a memory 4 a that stores a background model used when processing the frame image input to the image input unit 3. The image processing unit 4 updates the background model stored in the memory 4 a using the frame image input to the image input unit 3. Further, the memory 4a has a range of pixel values determined to be a specular pixel, a specular pixel setting confirmation time (corresponding to the first time in the present invention), a specular pixel setting release confirmation time (first in the present invention). 2), object detection confirmation time (corresponding to the third time in the present invention), and data generated during operation (threshold value, specular area, foreground area, etc. in the present invention) are also stored. To do.

  The image processing unit 4 includes a microcomputer that executes image processing on the frame image input to the image input unit 3. The image processing program according to the present invention is a program for operating this microcomputer and is installed in advance.

  The input / output unit 5 controls data input / output with a host device (not shown). When the image processing unit 4 detects an object, the input / output unit 5 outputs a signal for notifying a higher-level device (not shown) to that effect. The input / output unit 5 may be configured to output a signal notifying the detection of the object, or may be configured to output a signal notifying the detection of the object together with the frame image in which the object is detected. The input / output unit 5 may be configured to transmit a frame image captured by the far-infrared camera 10 (a frame image input to the image input unit 3) to a higher-level device.

  When the host device is notified of detection of an object located in the monitoring target area by a signal output from the input / output unit 5 of the image processing device 1, it outputs a message to that effect to the surroundings. It may be configured to notify a staff member who is present, or may be configured to display the frame image on a display when a frame image in which an object is detected is transmitted. Further, a recording device that records a frame image captured by the far-infrared camera 10 may be provided so that it can be confirmed as necessary.

  Next, various data stored in the memory 4a will be described.

  First, the background model will be described. The background model is obtained by modeling the frequency (occurrence probability) of the pixel value of each pixel of the frame image input to the image input unit 3. Specifically, using the frame image of the latest n frames input to the image input unit 3, the frequency (occurrence probability) of the pixel value of each pixel of the frame image is calculated using a mixed Gaussian distribution. Modeled. The memory 4 a stores a background model for each pixel of the frame image input to the image input unit 3. The image processing unit 4 updates the background model of each pixel every time a frame image is input to the image input unit 3. Regarding the method of generating the background model, various methods such as a method of generating using all the pixels (foreground pixels and background pixels), a method of generating using only the background pixels (without using the foreground pixels), etc. Is already known, and the description thereof is omitted here. For the background model, a method corresponding to the characteristics of the far-infrared camera 10, the imaging environment, or the like may be selected from known methods.

  FIG. 2 is a diagram illustrating a background model of a certain pixel. In FIG. 2, the horizontal axis represents pixel values, and the vertical axis represents frequency (occurrence probability). The threshold value D shown in FIG. 2 is a boundary value for determining whether the pixel is a background pixel or a foreground pixel. When the far-infrared camera 10 processes the frame image captured at the time t, the image processing unit 4 uses the n-frame image captured by the far-infrared camera 10 between the time t-1 and the time t-n. The background model generated by

  For each pixel of the frame image input to the image input unit 3, the image processing unit 4 determines a background pixel if the frequency of the pixel value of the pixel in the background model of the pixel is equal to or higher than the threshold D, and the threshold D If it is less than that, it is determined as a foreground pixel. The threshold value D for each pixel of the frame image input to the image input unit 3 is the same value. The threshold value D is stored in the memory 4a. Further, the image processing unit 4 has a function of calculating and setting the threshold value D (updating the threshold value D stored in the memory 4a) from the frame image input to the image input unit 3. Details of the processing for setting the threshold value D will be described later.

  Next, the pixel value range of pixels determined as mirror pixels will be described. The image processing unit 4 stores a lower limit value and an upper limit value of pixel values determined as mirror pixels in the memory 4a. The mirror surface pixel referred to here is a pixel obtained by imaging sunlight reflected by a puddle or metal. For example, when a monitoring target area that is an imaging area of the far-infrared camera 10 has a puddle as shown in FIG. 3, a pixel that captures sunlight reflected by the puddle is a mirror surface pixel. Since the specular pixel has a pixel value corresponding to the far-infrared energy amount of the reflected sunlight, the specular pixel is determined to be a foreground pixel when generating the background difference image.

  The pixel value of the specular pixel approximates the pixel value when the sky is imaged. The far-infrared camera 10 picks up the water pool or sunlight reflected by metal or the like when it is fine, and is not picked up when it is cloudy or rainy. This is because, during cloudy weather or rainy weather, the sun irradiates water pools or metals and the like is scattered by clouds. For this reason, in this example, the lower limit value of the pixel value of the pixel to be determined as a mirror surface pixel based on the distribution of the number of pixels with respect to the pixel value using a frame image obtained by imaging the sky with the far-infrared camera 10 in fine weather, The upper limit value is determined and stored in advance in the memory 4a.

  The clear sky emits very low far-infrared energy. The distribution of the number of pixels with respect to the pixel value in the frame image obtained by imaging the sky in the clear sky with the far-infrared camera 10 is concentrated on a very low pixel value as shown in FIG. In FIG. 4, the horizontal axis is the pixel value, and the vertical axis is the number of pixels. In this example, the pixel value A shown in FIG. 4 is the lower limit value of the pixel value determined as the specular pixel, and the pixel value B is the upper limit value of the pixel value determined as the specular pixel.

  Further, the lower limit value A and the upper limit value B of the pixel value determined as the mirror surface pixel are determined by a clear sky image of the sky imaged by the far infrared camera 10. Therefore, the lower limit A and the upper limit B are values that take into account the characteristics of the far-infrared camera 10 and the environment of the imaging area that is the monitoring target area. Further, the lower limit value A and the upper limit value B of the pixel value determined as the mirror pixel stored in the memory 4a may be updated at appropriate intervals such as one week or one month.

  The specular pixel setting confirmation time, the specular pixel setting cancellation confirmation time, and the object detection confirmation time stored in the memory 4a are about several seconds (1 to 3 seconds). The mirror pixel setting confirmation time, the mirror pixel setting release confirmation time, and the object detection confirmation time may all be the same time, or may be different times. However, the object detection confirmation time is preferably set to be longer than the specular pixel setting confirmation time. By setting the object detection confirmation time to be equal to or longer than the mirror surface pixel setting confirmation time, it is possible to prevent erroneous detection of the mirror surface area as an object before it is determined to be the mirror surface area.

  The mirror pixel setting confirmation time, the mirror pixel setting release confirmation time, and the object detection confirmation time may be set by the number of frame images. For example, if the far-infrared camera 10 is configured to output a frame image of 10 frames per second, it may be configured to set 10 frames (number of frame images) instead of 1 second (time).

  Next, the operation of this image processing apparatus will be described. Here, an outline of the operation of the image processing apparatus 1 will be described first, and then each operation will be described in detail. FIG. 5 is a flowchart showing the operation of the image processing apparatus.

  The far-infrared camera 10 inputs a frame image obtained by imaging the imaging area to the image input unit 3. The image processing unit 4 captures the frame image (one frame) input to the image input unit 3 (s1). The image processing unit 4 captures the frame images input to the image input unit 3 in the input order, and repeats the following processing.

  The image processing unit 4 performs a specular region setting process for setting a specular region in which sunlight is reflected from a puddle or metal or the like on the frame image captured this time (s2). In s2, a mirror area may not be set for the frame image captured this time, or one or a plurality of mirror areas may be set.

  The image processing unit 4 performs background difference image generation processing for generating a background difference image for the frame image captured this time (s3).

  The image processing unit performs an object detection process for detecting an object captured in the frame image captured this time from the background difference image generated in s3 (s4). In s4, an object may not be detected, or one or more objects may be detected.

  When the image processing unit 4 detects an object captured in the frame image captured this time in s4, for each object detected in s4, a type for determining whether the object is a person or something other than a person Determination processing is performed (s5, s6). The image processing apparatus 1 outputs the determination result concerning s6 from the input / output unit 5 (s7), and notifies the host apparatus of the detection of the object.

  If the image processing apparatus 1 determines in s5 that an object has not been detected, the image processing apparatus 1 performs the following processing on s8 without performing the processing on s6 and s7.

  The image processing unit 4 performs background model update processing for updating the background model stored in the memory 4a using the frame image captured this time in s1 (s8), and returns to s1. In s8, the background model is updated for each pixel of the frame image.

  The update of the background model is not limited to a specific method, and may be performed by any known method as described above.

  The image processing apparatus 1 repeats the process shown in FIG.

  Next, details of the specular area setting process according to s2 will be described. FIG. 6 is a flowchart showing the specular area setting process.

  The image processing unit 4 performs the following processing of s11 to s17 for each pixel of the frame image captured this time in s1, and determines whether or not it is a mirror pixel.

  The image processing unit 4 determines whether or not the pixel value of the determination target pixel for determining whether or not it is a specular pixel is within the range between the lower limit value A and the upper limit value B of the specular pixel stored in the memory 4a. Determine (s11). If the image processing unit 4 determines that the pixel value is not within the range between the lower limit value A and the upper limit value B in s11, the image processing unit 4 determines whether or not it is a pixel determined to be a specular pixel in the previously captured frame image ( s12). The image processing unit 4 determines that this pixel is not a specular pixel if it is determined that it is not a specular pixel in the previously captured frame image (s13).

  Further, if the image processing unit 4 is a pixel determined to be a mirror surface pixel in the previously captured frame image, the time when the pixel value of this pixel is outside the range between the lower limit value A and the upper limit value B is a mirror surface. It is determined whether or not the pixel setting cancellation confirmation time is continued (s14). For each pixel determined to be a mirror pixel, the image processing unit 4 counts the duration for which the pixel value is outside the range between the lower limit value A and the upper limit value B of the mirror surface pixel. This count value is stored in the memory 4a.

  If the time during which the pixel value of this pixel is outside the range between the lower limit value A and the upper limit value B continues for the specular pixel setting release confirmation time, the image processing unit 4 determines that this pixel is not a specular pixel in s13. judge. On the other hand, if the time during which the pixel value of this pixel is outside the range between the lower limit value A and the upper limit value B has not continued for the mirror pixel setting cancellation confirmation time, the image processing unit 4 regards this pixel as a mirror pixel. Determine (s15).

  In addition, the image processing unit 4 determines in s11 that the pixel value is determined to be within the range between the lower limit value A and the upper limit value B, as a pixel determined as a specular pixel in the previously captured frame image. It is determined whether or not there is (s16). The image processing unit 4 determines that this pixel is a specular pixel in s15 if it is a pixel determined to be a specular pixel in the previously captured frame image.

  Further, if the image processing unit 4 is a pixel that is determined not to be a specular pixel in the previously captured frame image, the time during which the pixel value of this pixel is within the range between the lower limit value A and the upper limit value B is a specular pixel. It is determined whether the setting confirmation time is continued (s17). For each pixel determined not to be a mirror surface pixel, the image processing unit 4 counts the duration for which the pixel value is within the range between the lower limit value A and the upper limit value B of the mirror surface pixel. This count value is stored in the memory 4a.

  If the time during which the pixel value of this pixel is within the range between the lower limit value A and the upper limit value B continues for the specular pixel setting confirmation time, the image processing unit 4 determines that this pixel is a specular pixel in s15. judge. On the other hand, if the time during which the pixel value of the pixel is within the range between the lower limit value A and the upper limit value B has not continued for the specular pixel setting confirmation time, the image processing unit 4 determines that the pixel is not a specular pixel in s13. Is determined.

  As described above, the image processing unit 4 determines that each pixel of the frame image has a specular pixel when the pixel value is within the range between the lower limit value A and the upper limit value B for the specular pixel setting confirmation time. Judge that there is. Further, the image processing unit 4 determines that each pixel of the frame image is not a mirror pixel when the pixel value is outside the range between the lower limit value A and the upper limit value B and the mirror pixel setting release confirmation time continues. judge.

  Accordingly, the image processing unit 4 temporarily does not determine that a pixel whose pixel value is within the range between the lower limit value A and the upper limit value B is a specular pixel. In addition, the image processing unit 4 temporarily does not determine that a pixel whose pixel value is outside the range between the lower limit value A and the upper limit value B is not a specular pixel.

  The image processing unit 4 sets a specular region on the frame image captured this time based on the distribution of pixels determined to be specular pixels in s15 (s18). In s18, the area where the specular pixels are gathered is set as the specular area. For this reason, the set mirror surface area may include pixels determined not to be mirror surface pixels by the above processing. The image processing unit 4 stores the mirror surface area set in s18 in the memory 4a. For example, with respect to the frame image, an area where the puddle shown in FIG. 3 is imaged is set as a mirror surface area, and this area is stored in the memory 4a.

  In addition, this determination result is maintained about the pixel determined to be not a mirror surface pixel contained in the set mirror surface area | region.

  Next, details of the background difference image generation processing according to s3 will be described. FIG. 7 is a flowchart showing background difference image generation processing.

  The image processing unit 4 determines, for each pixel other than the specular area set in the above process, whether the pixel is a foreground pixel or a background pixel in the frame image captured this time in s1 (s21). In other words, it is not determined whether the pixels in the specular region set by the above processing are foreground pixels or background pixels. The image processing unit 4 determines whether the pixel is a foreground pixel or a background pixel by using the background model and the threshold value D stored in the memory 4a.

  The image processing unit 4 generates a histogram indicating the distribution of pixel values for the pixels determined to be background pixels in s21 (s22). In other words, the image processing unit 4 generates this histogram without using the pixels located in the specular area set in s18 and the pixels determined as the foreground pixels in s21.

  FIG. 8 is a histogram showing the distribution of pixel values of background pixels in a frame image. FIG. 8 (A) is a histogram when the sunlight is irradiated (daytime in fine weather), and FIG. 8 (B) is a time zone when the sunlight is not irradiated (from evening to dawn) This is a histogram of cloudy weather. However, FIGS. 8A and 8B show a state in which the background such as the road surface is not wet by rain. Further, FIGS. 8C and 8D are histograms at the time of raining, and both are in a state where the background such as the road surface is wet with rain. FIG. 8D shows a state where there is more rainfall than FIG. 8C. 8A to 8D, the horizontal axis represents the pixel value, and the vertical axis represents the number of pixels.

  As shown in FIGS. 8A to 8D, when it rains, the road surface as a background gets wet with rain, so the pixel values of the background pixels are concentrated on a certain value. Also, as the amount of rainfall increases, the degree of wetness of the road surface and the like approaches uniformly, so the degree of concentration of pixel values of pixels that image the road surface and the like that are determined to be background pixels increases to a certain value (histogram) Of the dispersion δ becomes smaller).

  On the other hand, when it rains, objects such as people and vehicles in the foreground get wet in the same manner as the road surface in the background, so that not only the background pixels but also the pixel values of the foreground pixels are lowered.

The image processing unit 4 calculates a threshold value D based on the histogram variance δ generated in s22 (s23). Specifically, the threshold value D is set to
Threshold value D = α−β / δ
(Α and β are values individually set in advance).

  The threshold D decreases as the variance δ of the histogram generated in s22 decreases, that is, as the rainfall increases.

  The image processing unit 4 updates the threshold value D stored in the memory 4a to the value calculated in s23 (s24), and ends.

  As is clear from the above description, the threshold value D calculated from the frame image captured this time is used to determine whether each pixel of the frame image captured next time is a foreground pixel or a background pixel. That is, the threshold value D used when determining whether each pixel of the frame image captured this time is a foreground pixel or a background pixel is a value calculated from the previously captured frame image.

  Thus, the threshold value D is set almost in real time with changes in the weather in the imaging area of the far-infrared camera 10. Therefore, for each pixel of the frame image captured by the far-infrared camera 10, whether or not the pixel is a foreground pixel or a background pixel depends on the weather change in the imaging area of the far-infrared camera 10. Along with this, it is possible to suppress the decrease.

  In the above description, when generating a histogram in s22, it is assumed that the pixel located in the mirror area set in s18 and the pixel determined as the foreground pixel in s21 are not used. If there is an area that is not wetted by rain when it rains because the image capturing target area is provided with a roof or the like, an area that is wetted by rain when it rains may be set as a threshold calculation area. In this case, the above-described histogram may be generated by pixels that are within the set threshold calculation area and are determined to be background pixels in s21.

  Next, details of the object detection processing according to s4 will be described. FIG. 9 is a flowchart showing the object detection process.

  The image processing unit 4 sets a foreground region on the frame image captured this time based on the distribution of pixels determined to be foreground pixels in s21 (s31). In s31, the area where the foreground pixels are gathered is set as the foreground area. For this reason, the set foreground region may include pixels determined to be background pixels in the above processing. The image processing unit 4 stores the foreground area set in s31 in the memory 4a. For example, for the frame image, the area shown in FIG. 3 where the vehicle and the person are captured is individually set as the foreground area, and this area is stored in the memory 4a.

  The image processing unit 4 determines whether each foreground area set in s31 is larger than a predetermined size (s32), and determines that the foreground area smaller than the predetermined size is noise (s33). In s33, it is determined that no object is captured in this foreground area (a foreground area smaller than a predetermined size).

  On the other hand, the image processing unit 4 determines, for each foreground area determined to be larger than the predetermined size in s32, whether or not the foreground area is continuously detected in the object detection confirmation time stored in the memory 4a. (S34). The image processing unit 4 detects, as an object, the foreground area detected continuously in the object detection confirmation time in s34 (s35). The image processing unit 4 counts the duration set on the frame image for each foreground area determined to be larger than the predetermined size in s32. This count value is stored in the memory 4a.

  Note that the image processing unit 4 does not detect the foreground area that has not been detected continuously for the object detection confirmation time as an object, but does not determine the foreground area as noise.

  As a result, the image processing unit 4 detects an object staying in the imaging area of the far-infrared camera 10 for at least the object detection confirmation time as an object. For this reason, it is possible to prevent the image processing unit 4 from detecting, as an object, a bird or insect flying near the imaging lens of the far-infrared camera 10.

  In addition, as described above, the object detection confirmation time is set to be equal to or longer than the specular pixel setting confirmation time, so that the image processing unit 4 determines that the specular area is an object before the specular area is set in s18. It is possible to prevent erroneous detection.

  Next, details of the type determination process according to s6 will be described. FIG. 10 is a flowchart showing the type determination process.

  For each foreground area detected as an object in s35, the image processing unit 4 generates a histogram indicating the distribution of pixel values of pixels in the foreground area (s41). Since an object such as a vehicle is made of a material having a substantially uniform surface, the irradiance of far-infrared energy is approximately uniform. For this reason, the foreground area of an object such as a vehicle has a relatively small histogram variance δ indicating the distribution of pixel values.

  On the other hand, the amount of radiation of far-infrared energy differs depending on the part of the body, and even if it is wet, the part where clothes and feathers are in close contact with the skin and the part away from the skin Therefore, the variance δ of the histogram indicating the pixel value distribution is relatively large.

  If the variance δ of the histogram generated in s41 is greater than or equal to the predetermined value C, the image processing unit determines that the person is a human, and otherwise (the variance δ is less than the predetermined value C) determines that the object is not a human (s42 to s44). In this type determination, the type may be determined in consideration of the size of the foreground area.

  Note that the value C used for the above determination is also stored in the memory 4a.

  As described above, the image processing apparatus 1 calculates and updates the threshold value D for determining whether the pixel is the foreground pixel or the background pixel according to the change in the weather in the imaging area of the far-infrared camera 10. Therefore, for each pixel of the frame image captured by the far-infrared camera 10, whether or not the pixel is a foreground pixel or a background pixel depends on the weather change in the imaging area of the far-infrared camera 10. Along with this, it is possible to suppress the decrease. As a result, the setting of the foreground region for the frame image captured by the far-infrared camera 10 can be performed appropriately, so that the detection accuracy of an object such as a person or vehicle captured in the frame image of the imaging area captured by the far-infrared camera 10 is improved. It can suppress that it falls with the change of the weather of the imaging area of this far-infrared camera 10. FIG.

  Further, since the image processing unit 4 does not determine whether the pixel in the set mirror area is a foreground pixel or a background pixel, the mirror area is not erroneously detected as an object.

  Further, the image processing unit 4 determines the type of the object based on the variance δ of the histogram indicating the pixel value distribution of the pixels in the foreground area detected as an object, so whether or not the detected object is a person. Judgment can be made with high accuracy.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 2 ... Control part 3 ... Image input part 4 ... Image processing part 4a ... Memory 5 ... Input / output part 10 ... Far-infrared camera

Claims (13)

An image input unit for inputting a frame image of an imaging area captured by an infrared camera;
For each pixel of the frame image input to the image input unit, a background model storage unit that stores a background model that models the frequency of the pixel value of the pixel;
For each pixel of the frame image input to the image input unit, the background model storage unit determines whether the pixel is a foreground pixel or a background pixel using the background model stored for the pixel, A background difference image generation unit for generating a background difference image;
A threshold value setting unit that sets a threshold value that is a reference for determining whether the background difference image generation unit is a foreground pixel or a background pixel for each pixel of the frame image input to the image input unit. ,
The background difference image generation unit, for each pixel of the frame image input to the image input unit, if the frequency in the background model of the pixel is a pixel value higher than the threshold set by the threshold setting unit, background pixels, If the pixel value is lower than this threshold, it is determined as a foreground pixel,
The threshold value setting unit calculates and sets the threshold value based on a distribution of pixel values of pixels determined to be background pixels by the background difference image generation unit.   The image processing apparatus according to claim 1, wherein the threshold setting unit calculates and sets the threshold based on a distribution of pixel value distributions of pixels determined to be background pixels by the background difference image generation unit.   The threshold setting unit is a pixel determined by the background difference image generation unit to be a background pixel, and is located within a threshold calculation region set in advance for a frame image input to the image input unit The image processing apparatus according to claim 1, wherein the threshold value is calculated and set using a pixel value of a pixel. The image processing apparatus according to any one of the background difference image generation unit from the background difference image generated is, claim 1 to 3 which has an object detecting unit that detects the object being imaged. A specular pixel value storage unit that stores a range of specular pixel values;
Based on the range of pixel values stored in the specular pixel value storage unit, for each pixel of the frame image input to the image input unit, a specular pixel determination unit that determines whether the pixel is a specular pixel;
A specular region setting unit configured to set a specular region for the frame image input to the image input unit based on the determination result of the specular pixel determination unit;
The image processing apparatus according to claim 4, wherein the object detection unit does not detect an object being imaged with respect to the specular region set by the specular region setting unit.   The specular pixel determination unit continues a predetermined first time, and the pixel value in the frame image input to the image input unit is a range of pixel values of the specular pixel stored in the specular pixel value storage unit. The image processing apparatus according to claim 5, wherein a pixel that is a pixel is determined as the mirror pixel.   The specular pixel determination unit continuously determines a pixel value in the frame image input to the image input unit for a predetermined second time for the pixel determined to be the specular pixel. The image processing apparatus according to claim 5, wherein when the pixel value is out of a range of pixel values of the specular pixel stored in the image processing unit, the specular pixel is determined not to be the specular pixel.   The image processing according to claim 4, wherein the object detection unit sets a foreground region in which an object is imaged based on the foreground pixels in the background difference image generated by the background difference image generation unit. apparatus.   9. The object type determination unit according to claim 8, further comprising: an object type determination unit that determines whether or not the object is a person based on a distribution of pixel values of each pixel located in the foreground region set by the object detection unit. The image processing apparatus described.   The image processing apparatus according to claim 8, wherein the object detection unit determines that no object is imaged for the foreground area if the set foreground area is smaller than a predetermined size.   The image processing according to any one of claims 8 to 10, wherein the object detection unit detects an object captured in a frame image input to the image input unit continuously for a predetermined third time. apparatus. An image processing method for processing a frame image of an imaging area captured by an infrared camera input to an image input unit and generating a background difference image,
For each pixel of the frame image input to the image input unit, the pixel value frequency of the pixel stored in the background model storage unit is modeled as to whether the pixel is a foreground pixel or a background pixel . A background difference image generation step for determining using the background model and generating a background difference image;
A threshold setting step for setting a threshold as a reference for determining whether the background difference image generation step is a foreground pixel or a background pixel for each pixel of the frame image input to the image input unit. ,
In the background difference image generation step, for each pixel of the frame image input to the image input unit, if the frequency of the pixel in the background model is higher than the threshold set by the threshold setting unit, a background pixel, If the pixel value is lower than this threshold, it is determined as a foreground pixel,
The threshold value setting step is an image processing method in which the threshold value is calculated and set based on a distribution of pixel values of pixels determined to be background pixels in the background difference image generation step. An image processing program for processing a frame image of an imaging area captured by an infrared camera input to an image input unit and causing a computer to execute image processing for generating a background difference image,
For each pixel of the frame image input to the image input unit, the pixel value frequency of the pixel stored in the background model storage unit is modeled as to whether the pixel is a foreground pixel or a background pixel . A background difference image generation step for determining using the background model and generating a background difference image;
A threshold setting step for setting a threshold as a reference for determining whether the background differential image generation step is a foreground pixel or a background pixel for each pixel of the frame image input to the image input unit; To run
In the background difference image generation step, for each pixel of the frame image input to the image input unit, if the frequency of the pixel in the background model is higher than the threshold set by the threshold setting unit, a background pixel, If it is a pixel value lower than this threshold, it is a step of determining a foreground pixel,
The image processing program, wherein the threshold setting step is a step of calculating and setting the threshold based on a distribution of pixel values of pixels determined to be background pixels by the background difference image generation step.

Images