JP2007068146A - Image processing apparatus, method for calculating white balance evaluation value, program including program code for obtaining method for calculating white balance evaluation value, and storage medium for storing the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform an accurate white balance processing when a face is detected, without erroneously recognizing a human skin color in a high color temperature as white in a low color temperature. <P>SOLUTION: An image apparatus is provided with a face detection mode, and when a face is detected, the face area and a body candidate area whose values of luminance information and color information are within predetermined ranges with respect to those values of the face area, are detected, and those areas are excluded from the acquiring area of the white balance evaluation values. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a white balance control method at the time of face detection in an imaging apparatus that performs image processing on input image data to generate output image data.

  In an imaging apparatus such as a digital camera or a digital video camera, white balance (hereinafter referred to as WB) is adjusted as described below in order to balance the color of image data.

  An analog signal output from the image sensor through the color filter is converted into a digital signal by an analog / digital (hereinafter referred to as A / D) converter, and is divided into blocks as shown in FIG. .

  However, each block is configured in units including one color signal R (red), G1 (green), G2 (green), and B (blue), as shown in FIG.

For each of these blocks, the color evaluation value Cx = {(R + G2) − (B + G1)} / Y
Cy = {(R + B) / 4- (G1-G2) / 4} / Y
Y = (R + G1 + G2 + B) / 4
Is calculated.

  Y is a luminance signal.

  FIG. 4 is a diagram showing a state of a white detection range that changes depending on the color temperature.

  FIG. 4 shows the color coordinates where the horizontal axis is Cx = (R−B) / Y and the vertical axis is Cy = (R + B) / 4Y. The white color is previously changed from a high color temperature to a low color temperature. And the white axis defined by indicating the color evaluation values Cx and Cy on the coordinates.

  Since there is some variation in white in an actual light source, a range with a width around the white axis is set as a white detection range (a region to be determined as white).

  That is, the color evaluation values Cx and Cy obtained for each block are shown on the coordinates in FIG.

  Then, it is assumed that a block of color evaluation values included in the white detection range is white. Further, the integral values SumR, SumG1, SumG2, and SumB of the color pixels that have entered the white detection range are calculated, and the WB gain is calculated by using the following equation.

However, kWB_R, kWB_G1, kWB_G2, and kWB_B are WB gains of the color signals R, G1, G2, and B, respectively.
kWB_R = 1.0 / SumR
kWB_G1 = 1.0 / SumG1
kWB_G2 = 1.0 / SumG2
kWB_B = 1.0 / SumB
However, the WB coefficient calculation described above has the following drawbacks.

  Under the high color temperature, the white color evaluation value is distributed in the vicinity of the region A in FIG.

  However, when the color evaluation values Cx and Cy of human skin are shown on the coordinates under a high color temperature light source, they are distributed on the low color temperature side in the white detection range.

  Therefore, the color evaluation value of the screen is distributed in the region B in FIG. 4 in a scene where the imaging screen has little white and the human skin is up.

  That is, there is a problem that the skin color is mistakenly identified as white under a low color temperature, and the human skin is made white.

  Japanese Patent Application Laid-Open No. 2003-189325 discloses a technique related to WB control in an imaging apparatus capable of detecting a face. The above document discloses a technique in which when a face is recognized in the face recognition mode, the WB evaluation value acquisition area is moved from the face portion so that the calculation of the WB of the face portion is not performed.

When a person is imaged using the above technology, the color of the person's face and the hue when the light of the low color temperature light source is irradiated to the achromatic part are very close, resulting in misrecognition and whitening of the face It solves the problem of end.
JP 2003-189325 A

  However, since the technique disclosed in the above-mentioned patent document excludes only the face part from the WB evaluation value acquisition area, the skin color of the body part is applied to a subject with a lot of exposed skin other than the face, such as a swimsuit. The WB evaluation value is pulled.

  Therefore, there is a problem that accurate WB control cannot be performed.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention includes a face detection unit that detects a face area from image data, and a body from the image data according to the detected face area. Then, an area extracting means for extracting the estimated body candidate area, and an arithmetic means for calculating a white balance evaluation value based on the detection result of the face detecting means and the extraction result of the area extracting unit are provided. Features.

  In order to solve the above problems and achieve the object, a control method of an image processing apparatus according to the present invention includes a face detection step of detecting a face area from image data, and the image data according to the detected face area. A region extraction step of extracting a body candidate region estimated to have a body, and a calculation step of calculating a white balance evaluation value based on the detection result of the face detection step and the extraction result of the region step unit. It is characterized by providing.

  According to the present invention, it is possible to perform WB processing with high accuracy by changing the acquisition region of the WB evaluation value according to the scene.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 2 shows a block diagram of an imaging apparatus having a conventional face detection function.

  In FIG. 2, reference numeral 1 includes a lens system, an aperture, a shutter, an image sensor having a photoelectric conversion function such as a CCD, an A / D converter, and the like, and outputs an image projected on the image sensor by the lens system as a digital signal. An imaging unit.

  Reference numeral 2 denotes a face detection processing unit that determines whether or not a human face exists in the output image data of the imaging unit 1 by a known face detection method, and detects a face region if it exists.

  As a typical face detection method, a method using learning typified by a neural network, a part having a characteristic shape such as an eye, nose, or mouth is searched from an image region by using template matching. There is a technique to regard it as a face.

  There are many other methods such as methods that detect image features such as skin color and eye shape and use statistical analysis, and face recognition is generally performed by combining a plurality of these methods.

  Specific examples include a face detection method using wavelet transform and image feature amounts described in Japanese Patent Laid-Open No. 2002-251380.

  Reference numeral 3 denotes an exposure control unit that performs control related to exposure such as setting of an aperture and a shutter speed based on information in the face detection processing unit 2. Reference numeral 4 denotes an autofocus (hereinafter referred to as “AF”) control unit that specifies the focal position of the face detection area based on information in the face detection processing unit 2. Reference numeral 5 denotes a main exposure control unit that controls the aperture and the mechanical shutter determined by the exposure control unit 3. The exposure control unit 3 and the main exposure control unit 5 are generally used in common, but are described separately here to facilitate understanding of the imaging flow. Reference numeral 6 denotes a WB control unit that performs WB processing on image data obtained by the main exposure imaging. The WB control circuit 6 can also perform saturation adjustment, edge enhancement, and the like. Reference numeral 7 denotes a color signal generation circuit that generates color difference signals U and V from data subjected to WB processing by the WB control circuit 6. Reference numeral 8 denotes a luminance signal generation circuit that generates a luminance signal Y from data subjected to WB processing by the WB control circuit 6.

  Next, operation processing in the imaging apparatus according to the first embodiment of the present invention will be described using the flowchart shown in FIG.

  FIG. 5 is a diagram showing an acquisition area of a WB evaluation value when a face is detected in the face detection mode. FIG. 5A is a diagram in the case where the entire imaging screen is used as an acquisition area for the WB evaluation value. FIG. 5B is a diagram illustrating a state where an area having substantially the same luminance information and color information as the face area is excluded from the WB evaluation value acquisition area based on the face detection result. . Here is an example of a person whose upper body is naked.

  When the image pickup apparatus is powered on, the image pickup apparatus prepares for a shooting operation starting from step S101.

  In step S101, a CPU (not shown) determines whether or not the imaging apparatus is set to the face detection mode.

  If the CPU (not shown) determines in step S101 that the imaging device is in the face detection mode, the process proceeds to step S102, and the face detection processing unit 2 applies the image data obtained from the imaging device. Perform face detection processing.

  If the CPU (not shown) determines in step S101 that the imaging apparatus is not in the face detection mode, the process proceeds to step S105, and the colored portion in FIG. 5A is set as the WB evaluation value acquisition region. .

  If the CPU (not shown) determines in step S103 that the face detection processing unit 2 has not detected a face, the process proceeds to step S105, and the colored portion (the entire screen) in FIG. Is set as the WB evaluation value acquisition area.

  In step S103, if a CPU (not shown) determines that the face is detected by the face detection processing unit 2, the process proceeds to step S104.

  In step S104, the CPU (not shown) detects a region where the values of luminance information and color information are within a predetermined range relative to the values in the face region, and this region is a body candidate that is a part of the body. This is an area. Here, the predetermined range is a range that is statistically obtained from the result of comparing a large number of face areas and body skin areas. Then, the process proceeds to step S106.

  If a plurality of faces are detected in step S102, all areas whose luminance information and color information values are within a predetermined range with respect to the values of the respective face areas are detected as body candidate areas. That is, when a plurality of faces are detected, all body candidate areas based on the luminance information and color information of the respective face areas are detected.

  In step S106, the colored portion of FIG. 5B, which is an area excluding the face area and the body candidate area, is designated as the WB evaluation value acquisition area. As described above, FIG. 5B is an example of a person whose upper body is naked. If this person is wearing a short-sleeved shirt, a region where the arm beyond the elbow is exposed is detected as a body candidate region.

  In step S107, a CPU (not shown) obtains a WB evaluation value from the WB evaluation value acquisition area specified in step S105 or step S106.

  In step S108, a final WB coefficient is calculated based on the result of the process in step S107.

  In the present embodiment, an area in which the values of the luminance information and the color information are within a predetermined range with respect to the values of the face area is extracted as the body candidate area. However, the present invention is not limited to this. For example, an area in which one of the luminance information and the color information is within a predetermined range with respect to the face area may be extracted.

  Moreover, when detecting a body candidate area | region, the time required for a detection can be shortened by making only the vicinity area | region of the detected face area | region into a detection target.

  Furthermore, a WB coefficient that optimizes the skin color of the detected face area may be selected from a group of WB coefficient tables prepared in advance.

  As described above, in the present embodiment, when a face is detected, the face area and body candidate areas whose luminance information and color information values are within a predetermined range with respect to the values of the face area are detected. Are excluded from the WB evaluation value acquisition area.

  By doing so, even when the subject is wearing a swimsuit and there is much skin exposure other than the face, more accurate WB processing without erroneously recognizing the skin color as white under low color temperature Can be performed.

(Second Embodiment)
The operation of the present embodiment will be described in detail with reference to the drawings. The same step number is assigned to the step that performs the same processing as in the first embodiment.

  In the first embodiment, a region whose luminance information and color information values are within a predetermined range with respect to the value of the face region is detected as a body candidate region, and the face region and the body candidate region are detected as WB evaluation values. Excluded from the acquisition area. On the other hand, this embodiment is different from the first embodiment in that the WB evaluation value is calculated by weighting the face area and the body candidate area with a smaller weight than other areas.

  FIG. 6 shows a flowchart of an imaging process in the second embodiment of the present invention.

  In FIG. 6, the processes in steps S101 to S104 and S108 are the same as those in the first embodiment, and a description thereof will be omitted.

  If the CPU (not shown) determines in step S101 that the imaging apparatus is not in the face detection mode, or the CPU (not shown) determines that no face is detected in the face detection processing unit 2 in step S103. If so, the process proceeds to step S207.

  In step S207, the CPU (not shown) sets the entire imaging screen as the WB evaluation value acquisition area, obtains the WB evaluation value, and advances the process to step S108.

  In step S104, if a CPU (not shown) detects a body candidate region having a face region, luminance information within a predetermined range, and color information within a predetermined range, the process proceeds to step S208.

  In step S208, the WB evaluation value obtained from the face region and the body candidate region and the WB evaluation value obtained from other regions are weighted to obtain the WB evaluation value for the entire imaging screen.

  A specific example of the weighting of the WB evaluation value acquisition area in step S208 is shown in FIG.

  In step S208, weighting is performed so that the ratio between the WB evaluation value obtained from the face area and the body candidate area and the WB evaluation value obtained from other areas is 1: 4. Specifically, the WB evaluation value obtained from the region and the body candidate region is multiplied by 0.2 as a coefficient, and the WB evaluation value obtained from other regions is multiplied by 0.8 as a coefficient to obtain the sum.

  As described above, although the weighting is small, the WB control is performed even when the face is erroneously detected in the face detection by considering the face area and the WB evaluation value obtained from the face area and the body candidate area. Can be reduced.

  As described above, in the present embodiment, by reducing the weight ratio of the WB evaluation value obtained from the face area and the body candidate area, the WB evaluation value is pulled to the skin color, thereby preventing accurate WB control. To prevent that.

(Third embodiment)
The operation of the present embodiment will be described in detail with reference to the drawings. The same step number is assigned to the step that performs the same processing as in the first embodiment.

  This embodiment is different from the first embodiment in that an area that is substantially the same distance as the face area is excluded from an area used for calculation for obtaining a WB evaluation value as a body candidate area.

  FIG. 7 shows a flowchart of the imaging process in the third embodiment of the present invention.

  In FIG. 7, steps S101 to S103, and steps S105 and S107 to S108 are the same as those in the first embodiment, and thus the description thereof is omitted.

  In step S306, the CPU (not shown) detects an area that is within a predetermined range from the face area, and advances the process to step S308. Here, the predetermined range refers to a range of values obtained statistically from the result of comparing a lot of distance information of the face area and distance information of the limb or torso area. It is conceivable to change the size of the predetermined range in accordance with the size of the detected face area.

  When a plurality of faces are detected in step S102, all areas where the value of distance information is within a predetermined range with respect to the value of each face area are detected. That is, when a plurality of faces are detected, all body candidate regions based on the distance information of each face region are detected.

  In step S308, the CPU (not shown) designates an area other than the face area detected in the processes up to step S306 and an area within a predetermined range from the face area as the WB evaluation value acquisition area. Then, the process proceeds to step S107.

  In addition, when detecting an area within a predetermined range from the face area, only a nearby area located within the predetermined range on the basis of the detected face area is set as a body candidate area detection target. Thus, the detection process can be speeded up.

  As described above, in the present embodiment, the entire imaging screen is usually the WB evaluation value acquisition area, but when a face is detected, a body candidate area whose distance information is within a predetermined range of the face area is detected. Thus, the face area and the body candidate area are excluded from the WB evaluation value acquisition area.

  By doing so, for example, even when it is difficult to accurately obtain luminance information and color information in a backlight scene or the like, accurate WB processing can be performed by accurately detecting a subject and excluding it from the WB evaluation value acquisition region. It is possible.

  Software for realizing the functions of the above-described embodiments for an apparatus or a computer in the system connected to the various devices so that the various devices are operated so as to realize the functions of the above-described embodiments. Supply the program code.

  And what was implemented by operating the said various devices according to the program stored in the computer (CPU or MPU) of the system or apparatus is also contained under the category of the present invention.

  In this case, the program code of the software itself realizes the functions of the above-described embodiment.

  The program code itself and means for supplying the program code to a computer, for example, a storage medium storing the program code constitute the present invention.

  As a storage medium for storing the program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  The functions of the above-described embodiments are not only realized by executing the program code supplied by the computer.

  For example, even when the function of the above-described embodiment is realized in cooperation with an OS (operating system) or other application software running on the computer, the program code is included in the embodiment of the present invention. included.

  Further, the supplied program code is stored in a memory provided in a function expansion board of the computer or a function expansion unit connected to the computer.

  The CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the function of the above-described embodiment is realized by the processing. It is included in the present invention.

It is a flowchart which shows the WB process of the imaging device of the 1st Embodiment of this invention. It is a figure which shows schematic structure of the imaging device of this invention. It is a figure which shows a WB evaluation value detection block. It is a figure which shows the mode of the white detection range which changes with color temperatures. It is the figure which showed the acquisition area | region of WB evaluation value. It is a flowchart which shows the WB process of the imaging device of the 2nd Embodiment of this invention. It is the figure which showed the specific example of the weighting of the acquisition area of WB evaluation value. It is a flowchart which shows the WB process of the imaging device of the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up part 2 Face detection process part 3 Exposure control part 4 AF control circuit 5 This exposure control part 6 WB control circuit 7 Color signal generation circuit 8 Luminance signal generation circuit

Claims (14)

Face detection means for detecting a face region from image data;
Region extracting means for extracting a body candidate region in which a body is estimated to exist from the image data in accordance with the detected face region;
An arithmetic means for calculating a white balance evaluation value based on the detection result by the face detection means and the extraction result of the region extraction unit;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the region extracting unit extracts the body candidate region using color information obtained from the face region.   The image processing apparatus according to claim 1, wherein the region extraction unit extracts the body candidate region using luminance information obtained from the face region.   The image processing apparatus according to claim 1, wherein the region extraction unit extracts the body candidate region using distance information obtained from the face region.   The image processing apparatus according to claim 1, wherein the calculation unit calculates a white balance evaluation value using a remaining area obtained by removing the face area and the body candidate area from the image data.   The arithmetic means calculates a white balance evaluation value by weighting the remaining area obtained by removing the face area and the body candidate area from the image data by weighting more than the face area and the body candidate area. The image processing apparatus according to claim 1.   The said area detection means extracts the body candidate area | region according to each face detected from the said image data, when the said face detection means detects a some face, The Claim 1 to 6 characterized by the above-mentioned. Image processing device.   The image processing apparatus according to claim 1, further comprising: an image sensor having a photoelectric conversion function, wherein the calculation unit calculates a white balance evaluation value for image data obtained by the image sensor. A face detection step of detecting a face area from image data;
A region extracting step of extracting a body candidate region in which a body is estimated to exist from the image data according to the detected face region;
Based on the detection result of the face detection step and the extraction result of the region process unit, a calculation step of calculating a white balance evaluation value;
An image processing apparatus control method comprising:   The method of controlling an image processing apparatus according to claim 9, wherein the region extraction step extracts the body candidate region using color information obtained from the face region.   The method of controlling an image processing apparatus according to claim 9, wherein the region extraction step extracts the body candidate region using luminance information obtained from the face region.   The method for controlling an image processing apparatus according to claim 9, wherein the region extraction step extracts the body candidate region using distance information obtained from the face region.   A program for causing a computer to execute the control method according to claim 9.   14. A recording medium on which the program according to claim 13 is recorded so as to be readable by a computer.

Images