JP2013172311A - Image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus, an image processing method, and an image processing program capable of automating luminance correction of an imaging apparatus.SOLUTION: The image processing apparatus comprises: a first extraction unit which extracts, from among plural images obtained in series by the imaging apparatus, an image of a portion expressing the clothing of a person included in each image, as a sample image; and a detection unit which detects a sample image with the largest luminance from among the sample images obtained by the first extraction unit and outputs information expressing the color represented by the detected sample image, as information representing the criterion of white in the images obtained by the imaging apparatus.

Description

  The present disclosure relates to an image processing apparatus, an image processing method, and an image processing program that process a plurality of images acquired in a time series by a photographing apparatus.

  For a monitoring system that realizes monitoring over a wide range by installing a plurality of imaging devices in different places, a technique for performing color correction between these imaging devices has been proposed (see Patent Document 1). .

  For example, Patent Document 1 discloses a technique for obtaining a color correction parameter based on an image obtained by photographing a color chart with a known color and brightness using an individual photographing device. A technique for correcting the influence is described.

JP 2007-183872 A

  The above-described conventional technology needs to photograph an object having a known luminance such as a color chart in order to guarantee the sameness of luminance in images obtained by a plurality of photographing apparatuses. However, when a large number of imaging devices are installed at locations apart from each other, the operation of causing all of these imaging devices to capture an object having a known luminance requires a great deal of manpower.

  It is an object of the present disclosure to provide an image processing apparatus, an image processing method, and an image processing program that can realize automation of luminance correction of a photographing apparatus.

  An image processing apparatus according to one aspect extracts, from a plurality of images acquired in a time series by a photographing apparatus, an image of a portion representing a person's clothes included in each image as a sample image indicating the color of the clothes. An image obtained by detecting the sample image having the maximum luminance from the sample image obtained by the first extraction unit and the first extraction unit, and obtaining information indicating the color indicated by the detected sample image by the imaging device And a detector that outputs information indicating the white reference.

  In addition, an image processing method according to another aspect includes a plurality of images acquired in a time series by a photographing device, and images of a part representing a person's clothes included in each image are used as sample images indicating the color of the clothes. The sample image having the maximum luminance is extracted from the sample image obtained by the extraction, and information indicating the color indicated by the detected sample image is used to indicate a white reference in the image obtained by the photographing apparatus. Output as information.

  Further, an image processing program according to another aspect extracts, from a plurality of images acquired in time series by a photographing device, an image of a portion representing a person's clothes included in each image as a sample image indicating the color of the clothes. Detecting a sample image having the maximum luminance from the sample images obtained by the extraction, and indicating information indicating the color indicated by the detected sample image, information indicating a white reference in the image obtained by the imaging device As a computer.

  According to the image processing apparatus, the image processing method, and the image processing program of the present disclosure, it is possible to realize automation of luminance correction of the photographing apparatus.

It is a figure which shows one Embodiment of an image processing apparatus. It is a figure explaining extraction of a sample image. It is a figure which shows another embodiment of an image processing apparatus. It is a figure explaining the correlation of the brightness | luminance of a background reference image and the brightness | luminance of a sample image. It is a figure which shows another embodiment of an image processing apparatus. It is a figure which shows the relationship between the range of the image showing a person, and a sample image. It is a figure explaining specification of the field which extracts a background reference picture. It is a figure which shows the hardware structural example of an image processing apparatus. It is a figure which shows an example of the flowchart of a process which obtains a white reference value. It is a figure which shows another example of the flowchart of a process which obtains a white reference value.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 shows an embodiment of an image processing apparatus. In FIG. 1, the image processing device 10 is arranged corresponding to each photographing device 1. The image processing apparatus 10 generates information indicating a white reference in an image obtained by the corresponding photographing apparatus 1 and passes the generated information to the color correction apparatus 2. Note that the image processing apparatus 10 may be provided corresponding to the photographing apparatus 1, and therefore, if the number of the photographing apparatus 1 is one, only one image processing apparatus 10 may be provided.

  The image processing apparatus 10 illustrated in FIG. 1 includes a first extraction unit 11 and a detection unit 12. The first extraction unit 11 uses, as a sample image indicating the color of the clothing, each of the images representing the clothing of the person included in each image from a plurality of images acquired in time series by the corresponding imaging device 1. Extract. The detection unit 12 detects a sample image having the maximum luminance from the plurality of sample images obtained by the first extraction unit 11, and obtains information representing the color of the detected sample image by the photographing apparatus 1. Is output as information indicating the white reference in the image.

  For example, the first extraction unit 11 may extract, as a sample image, an image of a portion corresponding to the upper body of the person from each of the images representing the person included in each captured image acquired by the imaging device 1.

  FIG. 2 is a diagram for explaining extraction of a specimen image. In FIG. 2, reference signs P <b> 1, Pj, and Pk indicate examples of individual captured images included in a plurality of captured images acquired in time series by the imaging apparatus 1. Moreover, the arrow which attached | subjected the code | symbol t in FIG. 2 has shown progress of time.

  The captured images P1, Pj, and Pk illustrated in FIG. 2 include human images Ci, Cj, and Ck, respectively. The first extraction unit 11 illustrated in FIG. 1 extracts, for example, human images Ci, Cj, and Ck included in the captured images P1, Pj, and Pk illustrated in FIG. And the 1st extraction part 11 extracts sample image Si, corresponding to the part showing the clothes of these persons by extracting the part corresponding to the upper body of a person among the extracted images Ci, Cj, Ck of the person. Sj and Sk are extracted, respectively.

  Here, when each photographing apparatus 1 shown in FIG. 1 is installed in a place where many people go in and out, such as a shopping center or a corporate office, a large number of people are associated with each photographing apparatus 1. Pass through the field of view. And many persons who pass through the visual field of each photographing device 1 include persons whose clothes color is white with high accuracy. For example, considering a period from several hours to several days, the probability that a photographed image acquired by each photographing apparatus 1 within this period includes an image of a person whose clothing color is white is very high. Very expensive.

  Therefore, as described above, the sample image having the maximum luminance among the many sample images extracted by the first extraction unit 11 may be a sample image extracted from an image of a person whose clothing color is white. Is very expensive. That is, the color of the sample image detected as the sample image having the maximum luminance by the detection unit 12 is white with high accuracy. Therefore, the white reference in the captured image obtained by the image capturing apparatus 1 can be indicated by the information indicating the color of the sample image detected as the sample image having the maximum luminance by the detection unit 12. For example, the average luminance value of the sample image detected by the detection unit 12 can be used as a white reference value indicating a white reference in the captured image obtained by the imaging apparatus 1.

  As described above, the image processing apparatus 10 according to the present disclosure uses the high possibility that the captured image acquired in time series by the corresponding imaging apparatus 1 includes an image of a person whose clothing color is white. Thus, a white reference value indicating a white reference in the captured image obtained by the photographing apparatus 1 is acquired. That is, the image processing apparatus 10 disclosed herein does not need to collect images obtained by photographing a specific target such as a color chart by the corresponding photographing apparatus 1. Therefore, the image processing apparatus 10 according to the present disclosure can automatically perform image processing for obtaining a white reference value indicating a white reference in a captured image obtained by the imaging apparatus 1. Similarly, the white reference value corresponding to each photographing device 1 can be automatically obtained by the image processing device 10 corresponding to each photographing device 1 illustrated in FIG.

  Then, the color correction device 2 illustrated in FIG. 1 performs brightness correction of the captured image obtained from each imaging device 1 based on the white reference value obtained corresponding to each imaging device 1 in this way. . Thereby, on the basis of a captured image obtained when an object having a known luminance is imaged by each imaging device 1, luminance correction equivalent to the case where the luminance correction of the captured image obtained by the imaging device 1 is performed is realized. can do.

  As described above, according to the image processing apparatus 10 disclosed herein, the color correction apparatus 2 acquires a white reference value that is a reference for luminance correction processing applied to the captured image obtained by each imaging apparatus 1. Processing can be automated. That is, according to the image processing apparatus 10 of the present disclosure, it is possible to realize automation of luminance correction for each of the plurality of imaging apparatuses 1 installed at locations separated from each other.

  Based on the white reference value generated by each image processing device 10, the color correction device 2 subsequently performs luminance correction on the captured image obtained by each imaging device 1, and stores the captured image after luminance correction as an image. Accumulate in device 3. In this way, in the captured image stored in the image storage device 3, at least white is expressed with the same luminance. Therefore, based on the color of the image of the portion representing the clothes of the person included in each of the photographed images obtained from the different photographing devices 1, it is determined whether the colors of the clothes of the persons who are the subjects are the same. Is possible. Thereby, for example, it is possible to extract an image of a person having the same clothing color from captured images obtained by a plurality of imaging devices 1 installed on each floor of a building.

  For example, in a system for tracking a person who moves across a floor of a building, the color of the person's clothes is an important clue for narrowing down the person to be tracked. Therefore, the image processing apparatus 10 disclosed herein is useful for realizing a tracking system based on an image as described above.

  By the way, the brightness of the installation location of each imaging device 1 may change over time. Therefore, a method for obtaining a white reference value corresponding to the photographing apparatus 1 will be described while taking into account fluctuations in brightness at the installation location where each photographing apparatus 1 is installed.

  FIG. 3 shows another embodiment of the image processing apparatus. 3 that are equivalent to the components shown in FIG. 1 are denoted by the same reference numerals and description thereof is omitted.

  The detection unit 12 illustrated in FIG. 3 includes a second extraction unit 121, a normalization unit 122, and a specifying unit 123. The second extraction unit 121 extracts, as a background reference image, an image of a region included in the background, which will be described later, from a plurality of images acquired in time series by the photographing apparatus 1. The normalization unit 122 normalizes the luminance of each sample image obtained by the first extraction unit 11 based on the average luminance of the background reference image obtained by the second extraction unit 121, thereby obtaining the luminance of the sample image. The influence by the brightness fluctuation of the installation location of the photographing device 1 that appears is corrected. The specifying unit 123 specifies the maximum value of the brightness of each normalized sample image as the maximum brightness indicating white in the image obtained by the photographing apparatus 1.

  Here, each captured image obtained by the image capturing apparatus 1 is an image of a moving subject because the trajectory of the moving subject image such as a wall or ceiling does not pass through the foreground including a moving subject such as a person. And a background corresponding to an area that does not include. This background is a region in which the locus of the image accompanying the movement of the subject does not pass when the subject including the person moves in the field of view of the photographing apparatus 1, and the image does not change even if the subject moves. It is.

  The change in brightness at the installation location of the photographing apparatus 1 is similarly reflected in the brightness of the foreground included in the field of view of the photographing apparatus 1 and the brightness of the background. That is, when the brightness of the sample image extracted from the image of a person or the like included in the foreground changes with the change in brightness at the installation location of the photographing apparatus 1, the background portion is processed by the second extraction unit 121 described above. An equivalent luminance change also appears in the background reference image extracted from.

  Therefore, there is a strong correlation between the variation in the average luminance of the background reference image extracted from each captured image by the second extraction unit 121 and the variation in the luminance of the sample image due to the change in brightness at the installation location of the imaging device 1. There is a relationship.

  FIG. 4 is a diagram for explaining the correlation between the luminance of the background reference image and the luminance of the sample image. In FIG. 4, reference symbols P <b> 1 and P <b> 2 indicate captured images captured at different times by the same imaging device 1.

  A photographed image P1 shown in FIG. 4A is an example of a photographed image obtained when the illumination at the place where the photographing apparatus 1 is installed is dark. This captured image P1 includes an image of the person C1. Moreover, the code | symbol S1 shown to FIG. 4 (A) is an example of the sample image extracted from this person C1 image. Reference sign B1 is an example of a background reference image extracted from the captured image P1.

  On the other hand, the photographed image P2 shown in FIG. 4B is an example of a photographed image obtained when the illumination of the place where the photographing apparatus 1 is installed is bright. This captured image P2 includes an image of the person C2. Moreover, the code | symbol S2 shown to FIG. 4 (B) is an example of the sample image extracted from the image of this person C2. Reference numeral B2 shown in FIG. 4B is an example of the background reference image extracted from the captured image P2.

  These background reference images B1 and B2 are portions of the same position and size included in the image area representing the background in the field of view of the corresponding captured images P1 and P2. For example, the background reference images B1 and B2 are part of an image representing a ceiling included in the captured images P1 and P2.

  When the sample image S1 shown in FIG. 4A and the sample image S2 shown in FIG. 4B are directly compared, it can be seen that the sample image S2 is photographed as a brighter color. However, when the luminances of the background reference images B1 and B2 included in the captured image P1 and the captured image P2 are compared, when the captured image P2 is acquired, compared to when the captured image P1 is acquired, the captured image is captured. It turns out that the illumination of the installation place of the apparatus 1 is bright.

  In such a case, for example, by normalizing the brightness of the photographed image P1 on the basis of the brightness of the background reference image B2 included in the photographed image P2, the luminances of the sample images S1 and S2 are compared with each other. be able to.

  FIG. 4C shows an image obtained by normalizing the brightness of the background reference image B1 of the photographed image P1 shown in FIG. 4A so as to match the brightness of the background reference image B2 described above. P1 'is shown. If the sample image S2 shown in FIG. 4 (B) is compared with the normalized sample image S1 shown in FIG. 4 (C), the clothes of the person C1 are closer to white than the clothes of the person C2. I understand that. That is, by performing such normalization, it is possible to obtain a sample image that faithfully represents the color of the clothes of the person who is the subject, regardless of the surrounding brightness when the photographing apparatus 1 acquires each photographed image. Can do.

  For example, the normalization unit 122 illustrated in FIG. 3 first calculates an average luminance for each background reference image extracted from each captured image by the second extraction unit 121. Next, the normalization unit 122 calculates the ratio between the average luminance and a predetermined reference value as a coefficient indicating the variation in brightness at the installation location of the photographing apparatus 1. In this way, the normalization unit 122 multiplies the luminance value of each pixel included in the sample image extracted by the first extraction unit 11 from the captured image by multiplying the coefficient calculated corresponding to each captured image. The brightness of the sample image can be normalized. Note that the reference value described above may be, for example, the luminance value of the background portion in the first captured image obtained when the photographing apparatus 1 is installed, or the temporal variation range of the brightness of the background portion. A luminance value corresponding to the median value may be used.

The normalization unit 122 uses, for example, the coefficient k obtained based on the average luminance of the background reference image corresponding to each captured image and the average luminance Y of each sample image extracted from the captured image. The normalized average luminance Yn expressed as in (1) may be calculated.
Yn = k × Y (1)
The normalized average brightness obtained for each sample image in this way faithfully represents the color of the clothes of the person who is the subject, regardless of the ambient brightness when each captured image was acquired. ing. Therefore, by specifying the maximum value of the average luminance of the normalized sample image by the specifying unit 123, it is possible to specify the luminance indicating white in the photographed image obtained by the photographing device 1.

  Therefore, according to the image processing apparatus 10 having the detection unit 12 illustrated in FIG. 3, the captured image acquired by the imaging apparatus 1 with high accuracy even when the brightness of the imaging range of the imaging apparatus 1 varies. A white reference value indicating white can be obtained.

  FIG. 5 shows another embodiment of the image processing apparatus. 5 that are the same as those shown in FIG. 1 or FIG. 3 are given the same reference numerals, and descriptions thereof are omitted.

  The image processing apparatus 10 illustrated in FIG. 5 uses information obtained by the area determination apparatus 3 described later for processing of the detection unit 12.

  The first extraction unit 11 illustrated in FIG. 5 includes a moving body detection unit 111 and a region extraction unit 112. The moving body detection unit 111 detects an image representing a person from each captured image acquired by the imaging device 1. The area extraction unit 112 extracts a sample image corresponding to a portion representing a person's clothes from each captured image acquired by the imaging apparatus 1 based on the detection result by the moving body detection unit 111.

For example, the moving body detection unit 111 prepares a background image that does not include a moving body in advance, and obtains a difference between each captured image and the background image to detect a portion representing a moving object. Good. As a method for detecting an object that moves based on the difference from such a background image, the moving body detection unit 111 can use, for example, the technique disclosed in Document 1 below.
Reference 1: Yutaka Sato, Shunichi Kaneko, Yoshinori Niwa, Kazuhiko Yamamoto: “Robust Object Detection by Radial Reach Filter (RRF)”, Science (D-II), Vol. J86-D-II No.5, pp 616-624, 2003.
When applying this method, the moving object detection unit 111 may determine whether or not the moving object is a person based on the size and shape of the range detected as a portion representing the moving object. desirable. When it is determined that the moving body is a person, the moving body detection unit 111 desirably performs processing for passing information indicating the range of the image corresponding to the person to the region extraction unit 112.

The moving body detection unit 111 may determine whether or not the moving body image detected from each captured image represents a person based on the characteristics of the person's outline. The moving body detection unit 111 can use, for example, the technique disclosed in Document 2 below for this determination process.
Reference 2: Satoshi Yamauchi, Takayoshi Yamashita, Hironobu Fujiyoshi “Human Detection by Co-occurrence Expression of Features Based on Boosting”, IEICE D, Vol. J92-D, No. 8, pp. 1125-1134, 2009 .
In addition, the region extraction unit 112 may extract an image of a region corresponding to the upper body of the person among the regions indicating the range of the image representing the person detected by the moving body detection unit 111 as a sample image. When extracting the sample image, the region extraction unit 112 uses the relationship between the height of the person indicated by the size of the range of the image representing the person and the position of the upper body of the person, for example, as described above. The position of the sample image included in the range may be estimated.

  FIG. 6 shows the relationship between the image range representing a person and the sample image. In FIG. 6, a symbol R <b> 1 is an example of an image range representing a person detected by the above-described moving body detection unit 111 in the captured image P <b> 1. In FIG. 6, reference numeral S <b> 1 is an example of the range of the sample image specified by the region specifying unit 112 based on the range R <b> 1. Further, the arrow shown in FIG. 6 indicates the height with respect to the lower boundary of the range R1 of the image representing the person.

The area extraction unit 112 illustrated in FIG. 5, for example, based on the height H of the range R1 of the image representing the person shown in FIG. 6, the upper limit of the range in which the image corresponding to the upper body of the person is distributed in this image And a height Hu indicating a lower limit are obtained. Then, the region extraction unit 112 uses the sample image indicating the color of the person's clothes described above as the image range having the same characteristics as the pixels in the range R1 included between the height Hu and the height Hd. May be extracted as As a method for extracting a region in this way, the region extracting unit 112 can use, for example, the technique disclosed in the following document 3.
Reference 3: Tomoyuki Nagahashi, Hironobu Fujiyoshi, Takeo Kanade “Image Segmentation Using Graph Cut by Repeating Smoothing”, IPSJ Journal CVIM, Vol. 1, No. 2, pp. 10-20, 2008.
Note that the height Hu indicating the upper limit and the height Hd indicating the lower limit of the image range corresponding to the upper body of the person can be estimated with high accuracy based on, for example, the position of the image range R1 in the captured image. it can. For the estimation of the position of the subject in three dimensions based on the position in the captured image, the region extraction unit 112 can use, for example, the technique disclosed in Document 4 below.
Reference 4: Mitsuharu Hayasaka, Hideyoshi Tominaga, Kazuzo Komiya: “Recognition and Tracking of Multiple Moving Objects Using Backprojection Method and Kalman Filter”, PRMU2001-132, pp.133-138, Nov, 2001.
For example, the region extraction unit 112 estimates the height h and the position of the top of the person in the three-dimensional space of the person who is the subject based on the position of the above-described range R1 in the captured image by using the above-described technique. . Based on the estimation result, the region extraction unit 112 may estimate a range on the image corresponding to the range corresponding to the upper half of the person in the three-dimensional space. The region extraction unit 112 desirably sets, for example, a range of h × 1/8 to h × 4/8 from the top as the range corresponding to the upper body when performing such estimation processing.

  In this way, according to the first extraction unit 11 including the moving body detection unit 111 and the region extraction unit 112 illustrated in FIG. 5, the clothing of the person is represented for each image representing the person included in each captured image. The sample image can be extracted with high accuracy without omission.

  Further, the second extraction unit 121 illustrated in FIG. 5 includes an extraction processing unit 124 and a region information holding unit 125. The area information holding unit 125 holds information indicating the position occupied by the range to be extracted as the background reference image in the captured image obtained by the imaging apparatus 1. Then, the extraction processing unit 124 extracts a background reference image from each captured image based on the information held in the area information holding unit 125.

  For example, before the image processing apparatus 10 starts the process for obtaining the white reference value, the area determination apparatus 3 determines the movement trajectory of a subject including a person in the field of view of the photographing apparatus 1 as described below. Information indicating a non-overlapping area is generated. In addition, the area determination device 3 holds the generated information in the area information holding unit 125 described above as information indicating the background reference area.

  FIG. 7 is a diagram for explaining identification of a region from which a background reference image is extracted. In FIG. 7, reference numerals P1, P2,..., Pn denote n photographed images photographed in time series by the photographing apparatus 1 before the image processing apparatus 10 starts processing for obtaining a white reference value. Show.

  The captured images P1, P2,..., Pn illustrated in FIG. 7 represent a situation where the person C1 that appears from the left end of the field of view of the photographing apparatus 1 crosses to the right end of the field of view of the photographing apparatus 1. The captured images P2,..., Pn also capture the movement of another person C2 moving from the right end to the left end of the field of view of the image capturing apparatus 1.

  In FIG. 7, reference numerals SUB1, SUB2,..., SUBn indicate difference images obtained as differences between the above-described captured images P1, P2,. In the difference images SUB1, SUB2,..., SUBn illustrated in FIG. 7, portions that match the background image are shaded, and portions that do not match are shown as white regions surrounded by a dotted line. ing. In each difference image SUB1, SUB2,..., SUBn, the portion that does not match the background image corresponds to the image of the person C1 or the person C2 included in the above-described captured images P1, P2,. In each of the difference images SUB1, SUB2,..., SUBn shown in FIG. 7, a region where a difference corresponding to the image of the person C1 is detected is indicated by a symbol TR1, and a difference corresponding to the image of the person C2 is detected. The region is indicated by the symbol TR2.

  For example, the region discriminating apparatus 3 illustrated in FIG. 5 first compares the captured images P1, P2,..., Pn acquired from the image capturing apparatus 1 with the background image, thereby obtaining the pixel values of the pixels having differences. Difference images SUB1, SUB2,..., SUBn, where “1” is set and the pixel value of a pixel having no difference is “0”, are generated. Next, the region discriminating apparatus 3 generates a composite image as indicated by reference numeral CMP in FIG. 7 by obtaining a logical sum of pixel values for each pixel included in each of the difference images SUB1, SUB,..., SUBn. May be.

  A portion TR shown in white in the composite image CMP illustrated in FIG. 7 is a portion where the logical sum of the pixel values of the above-described difference images SUB1, SUB2,. The result of synthesizing the trajectories corresponding to the images of a large number of subjects that have moved is shown. That is, the portion TR shown in white in the composite image CMP is a portion that does not match the background image in any of the above-described difference images SUB1, SUB2,. It is. On the other hand, the shaded portion BG in the composite image CMP is a portion in which the logical sum of the pixel values of the above-described difference images SUB1, SUB2,... SUBn is “0”. The portion that is supposed to match is shown. This portion BG is a portion where the locus of the image of the subject moving within the field of view of the photographing apparatus 1 does not pass in any of the photographed images P1, P2,. This corresponds to a background area in which the locus of the subject image associated with is not overlapped at all.

  Based on the composite image obtained in this way, the area discriminating apparatus 3 is configured to include a background reference image as indicated by reference numeral B1 in FIG. The region to be extracted may be specified. The region discriminating apparatus 3 may specify the position of the background reference image B1 by using a template having a desirable size as the background reference image. For example, the region discriminating apparatus 3 detects the position of the template in which the pixels in the partial BG are most contained in the template in the composite image CMP described above, and uses the detected position as the position of the background reference image B1. You may specify.

  In addition, the area discriminating apparatus 3 may store information indicating the area specified in this way in the area information holding unit 125 prior to the start of processing by the image processing apparatus 10. For details of processing for detecting a portion where the locus of the image of the subject moving within the field of view of the photographing apparatus does not pass, see Japanese Patent Application Laid-Open No. 2011-259324 by the present applicant.

  In this way, the extraction processing unit 124 illustrated in FIG. 5 performs the extraction process based on the information stored in the region information storage unit 125, so that it is not affected by the object moving within the field of view of the photographing apparatus 1. The background reference image can be extracted with high accuracy. Thereby, the normalization part 122 can normalize the average brightness | luminance of the sample image extracted by the 1st extraction part 11 with high precision. By comparing the average luminance of the sample images normalized in this way, the specifying unit 123 can specify the white reference value corresponding to the photographing apparatus 1 with high accuracy.

The image processing apparatus 10 of the present disclosure described above can be realized by using a computer apparatus such as a personal computer, for example.
FIG. 8 shows an example of the hardware configuration of the image processing apparatus 10. 8 that are the same as those shown in FIG. 1 are given the same reference numerals, and descriptions thereof are omitted.

  The computer device 20 includes a processor 21, a memory 22, a hard disk device 23, a display device 24, an input device 25, an optical drive device 26, and a local area network (LAN) interface 28. Yes. The processor 21, the memory 22, the hard disk device 23, the display device 24, the input device 25, the optical drive device 26, and the LAN interface 28 illustrated in FIG. 8 are connected to each other via a bus.

  The optical drive device 26 described above can be mounted with a removable disk 27 such as an optical disk, and reads and records information recorded on the mounted removable disk 27.

  The computer device 20 is connected to a local area network (LAN) 30 via a LAN interface 28. In addition, a plurality of photographing devices 1 are connected to the LAN 30. Therefore, the computer apparatus 20 can acquire the captured image obtained by these imaging apparatuses 1 via the LAN interface 28.

  The input device 25 illustrated in FIG. 8 is, for example, a keyboard or a mouse. An operator of the image processing apparatus 10 operates the input device 25 to instruct the image processing apparatus 10 to start image processing for obtaining, for example, a white reference value corresponding to each photographing apparatus 1. Etc. can be entered.

  The memory 22 illustrated in FIG. 8 stores an image processing program for executing the image processing for the processor 21 to obtain the above-described white reference value together with the operating system of the computer apparatus 20. The above-described image processing program can be recorded and distributed on a removable disk 27 such as an optical disk. The image processing program may be stored in the memory 22 and the hard disk device 23 by attaching the removable disk 27 to the optical drive device 26 and performing a reading process. Further, the image processing program can be read into the memory 22 and the hard disk device 23 via a communication device (not shown) connected to a network such as the Internet.

  The processor 21 performs the functions of the first extraction unit 11 and the detection unit 12 illustrated in FIG. 1 by executing the image processing program stored in the memory 22 and the hard disk device 23. Further, information obtained in the process in which the processor 21 executes the above-described image processing program is held in the memory 22 or the hard disk device 23. That is, the image processing device 10 disclosed herein can be realized by the cooperation of the processor 21, the memory 22, and the hard disk device 23 included in the computer device 20 described above.

  The processor 21 can also function as the image processing device 10 corresponding to each photographing device 1 connected via the LAN 30 by cooperating with the memory 22 and the hard disk device 23 and the LAN interface 28. .

  In this case, as illustrated in FIG. 8, the hard disk device 23 may include a reference information table 31. Then, the processor 21 holds the white reference value acquired corresponding to each photographing apparatus 1 in the reference information table 31 corresponding to the apparatus ID (IDentifier) indicating the individual photographing apparatus 1, for example. These white reference values may be output.

  Further, the hard disk device 23 may include an area information table 32 that holds information indicating the position of the background reference image corresponding to each photographing apparatus 1. Providing such an area information table 32 in the hard disk device 23 is an example of a technique for realizing the area information holding unit 125 illustrated in FIG.

  The above-described image processing program includes an application program for causing the processor 21 to execute a process of extracting a specimen image from a captured image acquired in time series by the imaging apparatus 1. The image processing program also includes an application program for causing the processor 21 to execute processing for detecting a specimen image having the maximum luminance. Therefore, the processor 21 executes this image processing program, and thereby obtains a white reference value corresponding to each photographing apparatus 1 based on the photographed image obtained from each photographing apparatus 1 according to the image processing method disclosed herein. Processing can be executed.

  The processor 21 may execute a process of detecting a sample image having the maximum luminance after completion of the process of extracting the sample image, or may have the maximum luminance in the process of executing the process of extracting the sample image. A part of the processing for detecting the sample image may be executed in parallel.

  First, when the processor 21 executes a process of detecting a specimen image having the maximum luminance after completing a process of extracting a specimen image from a photographed image obtained by a photographing apparatus selected from a plurality of photographing apparatuses 1. Will be described.

  FIG. 9 shows an example of a flowchart of processing for obtaining the white reference value. Each process of step S300 to step S305 illustrated in FIG. 9 is an example of a process included in the image processing program. In addition, each processing of step S300 to step S305 is executed by the processor 21.

  First, the processor 21 acquires a captured image from the imaging device 1 via the LAN interface 28, and adds a numerical value “1” to the acquired image number num indicating the number of acquired captured images (step S301). For example, the processor 21 may acquire a captured image obtained by the imaging apparatus 1 every predetermined time of several tens of milliseconds to several seconds. Note that the processor 21 preferably sets the initial value “0” to the number of acquired images num prior to the start of the process of obtaining the white reference value.

  Next, the processor 21 extracts a sample image from the acquired captured image as described with reference to FIG. 6 (step S302). When a plurality of person images are included in the photographed image, the processor 21 extracts clothes image included in each of these person images as sample images in the process of step S302.

  Thereafter, the processor 21 determines whether or not the number of acquired images num has reached a predetermined number N (step S303). The predetermined number N can be set in advance based on the number of captured images acquired in step S301 described above, for example, in a period ranging from several hours to several days. Note that the value of the predetermined number N may be set in advance for each photographing apparatus 1 based on the probability of appearance of a person whose clothing color is white at the installation location of each photographing apparatus 1. For example, the predetermined number N corresponding to the photographing device 1 installed in a place where there are relatively many persons wearing white clothes, such as an office, has a predetermined number N corresponding to the photographing device 1 installed outdoors. A smaller value may be set.

  When the acquired image number num is less than the predetermined number N, the processor 21 returns to the process of step S301 according to the negative determination route of step S303, and repeats the processes of step S301 to step S303. As described above, the processor 21 can perform the functions of the first extraction unit 11 illustrated in FIG. 1 by repeating the processes of steps S301 to S303.

  When the number of acquired images num reaches the predetermined number N, the processor 21 proceeds to the process of step S304 according to the affirmative determination route of step S303. In step S304, the processor 21 detects, for example, a sample image having the maximum luminance from the sample images extracted from each captured image by the processing in steps S301 to S303 described above. In step S304, the processor 21 may detect the sample image having the maximum luminance by obtaining the average luminance Y for each sample image and specifying the maximum value in the average luminance Y, for example.

  Next, the processor 21 specifies a white reference value Ys indicating white in the photographed image obtained by the photographing apparatus 1 based on the sample image detected in step S304, and outputs the white reference value Ys (step S305). In step S305, for example, the processor 21 may set the maximum value of the average luminance Y obtained for each sample image in the processing of step S304 as the white reference value Ys corresponding to the photographing apparatus 1. Further, the processor 21 stores the white reference value Ys thus obtained in the reference information table 31 provided in the hard disk device 23 in correspondence with the device ID indicating the photographing device 1. As described above, the processor 21 executes the processing of step S304 and step S305, thereby realizing the function of the detection unit 12 illustrated in FIG.

  In addition, the processor 21 may execute the processes in steps S301 to S305 described above in parallel for each of the imaging devices 1 connected to the LAN 30. Then, the white reference value obtained by the processing corresponding to each photographing apparatus 1 may be stored in the reference information table 31 in the hard disk device 23.

  In this way, by passing the white reference value stored in the reference information table 31 to the color correction device 2 illustrated in FIG. 1, the color correction device 2 handles each of the captured images obtained by the respective imaging devices 1. It is possible to execute color correction processing based on the white reference value.

  Next, in a process in which the processor 21 executes a process of extracting a specimen image from a photographed image obtained by the photographing apparatus to be processed selected from the plurality of photographing apparatuses 1, a process of detecting a specimen image having the maximum luminance A method for executing a part in parallel will be described.

  FIG. 10 shows another example of a flowchart of processing for obtaining a white reference value. Note that among the steps shown in FIG. 10, the same steps as those shown in FIG. 9 are denoted by the same reference numerals, and description thereof is omitted.

  Each process of step S300 to step S305 and step S311 to step S317 illustrated in FIG. 10 is an example of a process included in the image processing program. Each of these processes is executed by the processor 21.

  First, the processor 21 refers to the area information table 32 provided in the hard disk device 23 to obtain information indicating the position of the background reference image corresponding to the imaging device 1 to be processed (step S311). The processor 21 may set an initial value “0” for the white reference value Ys, for example, in the process of step S311.

  Next, the processor 21 acquires the captured image obtained by the imaging device 1 by executing the processing of step S301 described above, and then extracts a background reference image from the captured image (step S312). For example, the processor 21 cuts out an image having a size corresponding to the template described above from the captured image based on the information indicating the position of the background reference image acquired in step S311 described above, thereby corresponding to each captured image. A background reference image can be extracted. In this way, the function of the second extraction unit 121 illustrated in FIG. 3 can be realized by the processor 21 executing the process of step S312. Based on the luminance value of each pixel included in the background reference image extracted in step S312, the processor 21 performs normalization processing as described with reference to FIG. 4 based on the average luminance of the background reference image. Is calculated (step S313).

  Next, the processor 21 extracts the sample image included in the captured image by executing the process of step S302. Next, the processor 21 selects one of the sample images extracted in step S302, and calculates the average luminance of the sample image based on the luminance value of each pixel included in the selected sample image (step S314). ). For example, the processor 21 may calculate the average luminance Y of the extracted sample image every time the sample image is extracted corresponding to each of the human images included in the captured image. Further, the processor 21 normalizes the average luminance Y of the sample image calculated in step S314 using the coefficient k described above (step S315). In step S315, the processor 21 may obtain the normalized average luminance Yn corresponding to the average luminance Y obtained for each sample image, for example, by using the above-described equation (1).

  In this way, the processor 21 executes the processes of step S314 and step S315 in parallel with the process of extracting the specimen image from each captured image. Thus, in parallel with the process of performing the sample image extraction process by the first extraction unit 11 illustrated in FIG. 3, the process for realizing the function of normalizing individual sample images by the normalization unit 122 is executed. can do.

  Further, each time the average luminance Y of each sample image is normalized in this way, the processor 21 compares the normalized average luminance Yn of the sample image with the white reference value Ys (step S316). When it is determined that the average luminance Yn of the normalized sample image is greater than the white reference value Ys (affirmative determination in step S316), the processor 21 calculates the average of the sample image with the white reference value Ys normalized. Update with the brightness Yn (step S317). On the other hand, when it is determined that the average luminance Yn of the normalized sample image is equal to or less than the white reference value Ys (No determination in step S316), the processor 21 skips the process of step S317. In this case, the value of the white reference value Ys maintains the value set so far.

  Thereafter, the processor 21 determines whether or not the processing of Steps S314 to S317 has been executed for all the sample images extracted in Step S302 (Step S318). When there is an unprocessed sample image, the processor 21 returns to the process of step S314 according to the edge determination route of step S318. And the process of step S314-step S317 is repeated about an unprocessed sample image.

  In this way, when the processing for all the sample images extracted from the captured image acquired in step S301 is completed, the processor 21 proceeds to the processing of step S303 according to the affirmative determination route of step S318.

  In the negative determination route in step S303, the processor 21 repeats the processes in steps S312 to S318 for the captured image newly acquired in the process in step S301. When the number of acquired images num reaches the predetermined number N (Yes determination in step S303), the processor 21 proceeds to the process of step S305. In the process of step S305, the processor 21 outputs the white reference value Ys obtained in the process of repeating the processes of step S314 to step S317 for each sample image as a white reference value corresponding to the imaging device 1 to be processed. .

  In this way, in the process of executing the process of extracting the sample image from each captured image, the processor 21 executes the process of step S316 and step S317, thereby realizing the function of the specifying unit 123 illustrated in FIG. May be.

  In this way, the process of specifying the white reference value corresponding to the photographing apparatus 1 is completed almost simultaneously with the acquisition of the photographed image from the photographing apparatus 1 to be processed and the extraction of the sample image from each photographed image. be able to.

  Further, the processor 21 may execute the above-described processes on the captured images obtained from the respective imaging devices 1 connected to the LAN 30 in parallel. Thereby, even when a large number of photographing apparatuses 1 are connected to the LAN 30, white reference values corresponding to the respective photographing apparatuses 1 can be acquired in a short time.

  In the image processing device 10 disclosed herein, the detection unit 12 uses the sample image having the maximum luminance based on an index other than the average luminance of the sample image extracted from each captured image obtained by the imaging device 1. May be detected. For example, the detection unit 12 extracts a predetermined number from the sort result for the luminance value of each pixel included in the sample image in descending order of the luminance value, and uses the average value obtained for the extracted luminance value instead of the average luminance of the sample image. Can be used. In other words, the detection unit 12 is used for the process of detecting the sample image having the maximum luminance, regardless of any index, as long as the index indicates the brightness of the color of the person's clothes represented by the sample image. Also good.

DESCRIPTION OF SYMBOLS 1 ... Imaging device; 2 ... Color correction device; 3 ... Area discriminating device; 10 ... Image processing device; 11 ... First extraction part; 12 ... Detection part; 111 ... Moving body detection part; Second extraction unit; 122 ... normalization unit; 123 ... identification unit; 124 ... extraction processing unit; 125 ... area information holding unit; 21 ... processor; 22 ... memory; 23 ... hard disk device; 26: Optical drive device; 27 ... Removable disk; 28 ... Local area network (LAN) interface; 30 ... Local area network (LAN); 31 ... Reference information table; 32 ... Area information table

Claims (4)

A first extraction unit for extracting, from a plurality of images acquired in time series by the imaging device, an image of a portion representing a person's clothes included in each image as a sample image indicating the color of the clothes;
A sample image having the maximum luminance is detected from the sample images obtained by the first extraction unit, and information indicating the color indicated by the detected sample image is a white reference in the image obtained by the photographing apparatus. An image processing apparatus comprising: a detection unit that outputs information. The image processing apparatus according to claim 1.
The detector is
A second extraction unit for extracting, as a background reference image, an image of a region through which a locus of an image of a subject including a person moving within the field of view of the photographing apparatus does not pass from the plurality of images;
A normalization unit that normalizes the luminance of each sample image obtained by the first extraction unit based on the average luminance of the background reference image obtained by the second extraction unit;
An image processing apparatus comprising: a specifying unit that specifies a maximum luminance value of each normalized specimen image as a white reference value indicating white luminance in an image obtained by the imaging apparatus. From a plurality of images acquired in time series by the imaging device, a part image representing a person's clothing included in each image is extracted as a sample image indicating the color of the clothing,
A sample image having the maximum luminance is detected from the sample images obtained by the extraction, and information indicating the color indicated by the detected sample image is output as information indicating a white reference in the image obtained by the photographing apparatus. An image processing method characterized by: From a plurality of images acquired in time series by the imaging device, each part image representing a person's clothes included in each image is extracted as a sample image indicating the color of the clothes,
A sample image having the maximum luminance is detected from the sample images obtained by the extraction, and information indicating the color indicated by the detected sample image is output as information indicating a white reference in the image obtained by the photographing apparatus. An image processing program for causing a computer to execute processing.