JP4262151B2 - Image processing method, image processing apparatus, computer program, and storage medium - Google Patents

Description

  The present invention relates to an image processing method, an image processing apparatus, a computer program, and a storage medium that perform luminance value correction on an image.

  In recent years, for example, with the widespread use of digital still cameras (hereinafter simply referred to as “digital cameras”) and the like, digitization of photographic images has become easier, especially on personal computers (hereinafter simply referred to as “personal computers”). Opportunities for handling photographic images as digital image data have increased.

  Also, with the spread of application software that can correct and process images, users can freely perform image processing on personal computers. For example, correction that brightens an image that appears dark and processing that whitens the skin can be raised.

  From such a background, a more accurate image processing technique is required for image correction and processing.

  In recent years, technological development of object extraction processing has advanced, and it has become possible to detect a human face from an image. For this reason, many proposals have been made to correct the face area to an optimal brightness. In particular, in correcting a backlight image, luminance correction by face extraction is effective.

For example, Patent Document 1 proposes that a correction characteristic is created from the extracted brightness information of a subject, and a brightness condition is corrected by obtaining a control condition from a region without the subject. Japanese Patent Application Laid-Open No. 2004-228867 proposes performing gamma correction based on the skin color average luminance value and the entire image average luminance value.
JP 2003-69822 A Japanese Patent No. 3203946 JP 2000-102033 A

  However, in the brightness correction method of Patent Document 1, a plurality of regions (a subject region and a region without a subject) are processed, which increases the processing load. In the luminance correction of Patent Document 1, a correction characteristic using a straight line is created for the range of the subject central value and the subject maximum value of luminance. When luminance correction is performed according to the correction characteristic having an inflection point in the face area in this way, a pseudo contour may occur in the face in the corrected image. Furthermore, in the luminance correction of Patent Document 2, when face extraction is performed by detecting skin color, if the face is dark like a backlight image, the skin color cannot be detected well and face extraction cannot be performed. Therefore, it is preferable that the face region is spatially extracted by detecting the positions of eyes and nose.

  The present invention has been made in view of the above problems, and an object of the present invention is to suppress the generation of a pseudo contour in a predetermined area extracted from an image and to realize appropriate luminance correction for the predetermined area. To do.

  Further, the present invention makes it possible to create an appropriate correction characteristic from a predetermined area extracted from an image (for example, an area such as a subject's face), and the correction characteristic does not include an inflection point in the predetermined area extracted from the image. An object is to enable appropriate luminance correction using information of a predetermined area (for example, a face area) spatially extracted from an image.

In order to achieve the above object, an image processing method according to the present invention comprises:
An image processing method for correcting image brightness,
An obtaining step of extracting a human face area from the image and obtaining an average luminance value of the face area based on a luminance value of a pixel included in the face area;
A calculation step of calculating a corrected luminance value that is a corrected luminance value for the average luminance value of the face region based on the average luminance value of the face region and the average luminance value of the entire image;
A setting step of setting a range from a value obtained by subtracting a predetermined luminance value from the average luminance value of the face area to a value obtained by adding a predetermined luminance value to the average luminance value of the face area as a face luminance range When,
Generating a corrected luminance characteristic in which the inclination is constant within the face luminance range and an average luminance value of the face area is converted into the corrected luminance value;
A correction step of correcting the luminance of the image based on the corrected luminance characteristic generated in the generation step.

An image processing apparatus according to another aspect of the present invention for achieving the above object has the following configuration. That is,
An image processing apparatus for correcting the brightness of an image,
Acquiring means for extracting a human face area from the image and obtaining an average luminance value of the face area based on a luminance value of a pixel included in the face area;
Calculation means for calculating a corrected luminance value that is a corrected luminance value for the average luminance value of the face region based on the average luminance value of the face region and the average luminance value of the entire image;
Setting means for setting a range from a value obtained by subtracting a predetermined luminance value from the average luminance value of the face area to a value obtained by adding a predetermined luminance value to the average luminance value of the face area as a face luminance range When,
Generating means for generating a corrected luminance characteristic in which an inclination is constant within the face luminance range and an average luminance value of the face region is converted into the corrected luminance value;
Correction means for correcting the luminance value of the image based on the correction characteristic luminance characteristic generated by the generation means.

  ADVANTAGE OF THE INVENTION According to this invention, appropriate brightness correction | amendment is realizable about this predetermined area, suppressing generation | occurrence | production of the pseudo contour in the predetermined area extracted from the image.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

<First Embodiment>
An example of an outline of the image processing system according to the first embodiment is shown in FIG. For example, a digital camera 105 and a printer 106 such as an inkjet printer are connected to the host computer 100. The host computer 100 includes application software 101 such as image processing software, word processor, spreadsheet, and Internet browser, OS (Operating System) 102, and various drawing commands (image drawing commands, Software includes a printer driver 103 that generates print data by processing a text rendering command and a graphics rendering command, and a digital camera driver 104 that transfers data in the digital camera to the OS 102.

  The host computer 100 includes a central processing unit CPU 108, a hard disk driver HD 107, a random access memory 109, a read only memory ROM 110, a monitor 111, and the like as various hardware capable of operating these software.

  As an example of the host computer 100 shown in FIG. 1, Windows (registered trademark) XP of Microsoft Corporation is used as the OS 102 for a personal computer of an IBM AT compatible machine of IBM, which can be used for arbitrary printing. An example is a form in which an application is installed and a digital camera and a printer are connected.

  In the host computer 100, output image data is generated using text data classified into text such as characters by the application 101, graphics data classified into graphics such as graphics, image image data classified into natural images, and the like. Created. Such output image data can be displayed on the monitor 111 by the OS 102. When the output image data is printed out, the application 101 issues a print output request to the OS 102. The graphics data portion is a graphics rendering command, and the image image data portion is a rendering command indicating an output image composed of an image rendering command. The group is issued to the OS 102. The OS 102 receives an output request from the application and issues a drawing command group to the printer driver 103 corresponding to the printer that performs print output. The printer driver 103 processes the print request and drawing command group input from the OS 102, creates print data that can be printed by the printer 105, and transfers the print data to the printer 105. When the printer 105 is a raster printer, the printer driver 103 sequentially performs image correction processing in response to a drawing command from the OS 102 and sequentially rasterizes it into an RGB 24-bit page memory. After rasterizing all drawing commands, the contents of the RGB 24-bit page memory are converted into a data format printable by the printer 105, for example, CMYK data, and transferred to the printer.

  The image correction processing of the present embodiment executed by the host computer 100 is particularly suitable for image data obtained by photographing with the digital camera 105. In order to input such image data to the personal computer, the digital camera 105 can transmit and receive data to and from the personal computer 107. Hereinafter, the image correction processing according to the present embodiment will be specifically described with reference to the flowchart shown in FIG. Further, the image correction processing of the present embodiment is performed by the application 101 in FIG.

  Image data captured by the digital camera 105 is generally recorded and saved in an SRAM typified by a CF card or an SD card. The user can transfer the data to the HD 107 in the personal computer through the digital camera driver 104. In the application 101, image processing software is activated to correct and process the brightness of the image.

  In this embodiment, it is assumed that correction processing is performed on the YCbCr signal. Therefore, when the image captured by the digital camera driver 104 is corrected by the application 101, if the image to be processed is a JPEG image stored as a YCbCr signal, the processing can be performed as it is. However, since the bitmap image and TIFF image are stored as RGB signals, it is necessary to convert the RGB signals into YCbCr signals as shown in FIG. 2 (steps S11 and S12).

  Next, in the image of the YCbCr signal, a face region is extracted by spatially analyzing the image with the luminance value Y (step S13). In recent years, face extraction technology has been developed, and many techniques for automatically extracting a face area from an image of a person have been proposed.For example, a face is extracted by performing a filter analysis, There are various types from a color difference signal to extracting a skin color region. Further, there are those that extract a face region along the outline of the face and those that detect a center point of the face and define a predetermined range as the face. Any known method may be used for the face extraction process in step S13.

  Next, a luminance histogram is calculated from the extracted face area, and an average luminance value Y_avr of the face area is obtained (step S14). In the present embodiment, the face area is set as a rectangle as shown in FIG. 3, and the pixel values included in the face area are aggregated as shown in FIG. 4 to create a histogram, and the average luminance value Y_avr is obtained. Yes. FIG. 4 shows a state where the average luminance value Y_avr of the face area is 100. In the present embodiment, the average luminance value of a predetermined area (a face area in the present embodiment) is used as the specific luminance value for determining the corrected luminance characteristic, but the present invention is not limited to this. For example, the center value of the histogram of the face area, the maximum value of the histogram of the face area in the process of brightening a dark image, or the peak value of the histogram is used as the specific brightness value if the histogram of the face area is distributed in a mountain shape To do.

  Next, a corrected luminance value Y′_avr is calculated from the obtained average luminance value Y_avr (step S15). Y′_avr represents a corrected value of the average luminance value Y_avr. The corrected luminance value Y′_avr may be calculated functionally as Y′_avr = f (Y_avr), or may be divided according to the value of Y_avr, and Y′_avr may be obtained by LUT. In this embodiment, in order to take into account the balance with the luminance value of the entire image, for example, Y′_avr is obtained as follows.

  First, the first correction magnification k1 is calculated from the average luminance value Y_avr of the face area. For example, this may be obtained by a function as shown in FIG. 5 or may be obtained by referring to the LUT. Next, the second correction magnification k2 is calculated from the average luminance value of the entire image. This can also be obtained by a function or LUT, similar to k1 above.

When two correction magnifications k1 and k2 are obtained, Y′_avr is calculated. In the present embodiment, the corrected average luminance value is calculated by the following (Equation 1) using the multiplication of k1 and k2 as the magnification of Y_avr with respect to Y′_avr.
Y'_avr = k1 × k2 × Y_avr (Formula 1)
Thereby, the corrected luminance value for the face average luminance value can be obtained from the two parameters of the average luminance value of the face and the average luminance value of the entire image.

  In the correction brightness value acquisition method described above, for example, if the face average brightness value is low, such as an image in which the face appears dark, the value of k1 is set large to increase the magnification. If the value is high, the correction luminance value can be adjusted well by setting the value of k1 small and decreasing the magnification.

  On the other hand, if the face and the entire image are dark like an underexposed image, the correction brightness value can be further increased by increasing the value of k2.On the other hand, the face is dark but the entire image is dark like a backlight image. If it is bright, it is possible to prevent the background portion of the image from becoming too bright by reducing the value of k2. Further, as described above, instead of the average brightness value of the face, the center value of the histogram of the face area or the maximum value of the histogram of the face area in the process of brightening the dark image, the histogram of the face area is distributed in a mountain shape. If it does, the peak value of a histogram etc. can also be used.

  In the present embodiment, the correction magnification is determined from the two parameters of the average luminance value of the face and the average luminance value of the entire image, but a plurality of parameters may be used. For example, it is conceivable to use parameters such as the maximum luminance value of the face area and the minimum luminance value of the image. In the present embodiment, the calculated correction magnifications k1 and k2 are multiplied. For example, these two coefficients may be added together.

  When the corrected luminance value Y′_avr for the average luminance value Y_avr is obtained as described above, the corrected luminance value for all luminances is determined using the corrected luminance value Y′_avr, and the corrected luminance characteristic is determined (steps S16 to S16). S18). In the first embodiment, corrected luminance characteristics over the entire luminance range are created by the method described below, and corrected luminance values for all luminance values are determined. The corrected luminance value Y′_avr may be set by the user. The corrected luminance characteristic is represented by a straight line, a curve, or a combination of a straight line and a curve. This will be described below with reference to FIG.

  First, a face luminance range is determined from the entire luminance range (step S16). This represents the luminance range of the pixels constituting the face area in the image, mainly the skin color portion. In general, when the face luminance range is obtained, it is considered that the maximum luminance value and the minimum luminance value are obtained from the histogram of the face region, and the range is set as the face luminance range. However, with this method, if there are many hair and eye areas in the extracted face area, the minimum luminance value in the histogram is the minimum of the face skin area even if the face skin area is distributed in halftones. A value extremely smaller than the luminance value is taken. In addition, when there is a whiteout in the nose of the face, the maximum luminance value in the histogram becomes extremely large. That is, with this method, there is a high possibility that an error will occur when the face luminance range is obtained.

  Therefore, in the first embodiment, a fixed range including the face average luminance Y_avr is set as the face luminance range. Specifically, a face luminance range constant Y_range having a predetermined value is defined, and the range from Y_avr−Y_range to Y_avr + Y_range is set as the face luminance range as shown in FIG. 6, and the luminance values of the face area are distributed in the range. And In this way, even when dark areas such as hair and eyes and whiteout such as nose are included during face extraction, the luminance range of the facial skin area can be defined without causing a large error. . The face luminance range constant Y_range may be determined empirically by taking statistics of a face histogram from a plurality of images and examining the distribution range, or using a standard deviation or the like from the face area histogram shown in FIG. It may be determined by examining the distribution.

  Next, a corrected luminance characteristic in the obtained face luminance range Y_avr ± Y_range is determined (step S17). If the corrected luminance characteristic becomes a broken line within the face luminance range, the inclination (that is, contrast) changes at the broken point, and a pseudo contour is generated within the face luminance range. Therefore, in the first embodiment, the inclination of the corrected luminance characteristic is constant within this face luminance range. The inclination value may be determined in advance or may be varied according to the value of the face luminance range. Further, if it is desired to correct only the brightness of the original image, the inclination may be set to 1. If the contrast is to be enhanced, the inclination may be set to 1 or more. Thus, a straight line having the set inclination and passing through the corrected luminance value Y′_avr with respect to the average luminance value Y_avr is determined, and this is set as a corrected luminance characteristic within the face luminance range. In this way, luminance correction can be performed while maintaining a predetermined contrast without generating a pseudo contour in the face area in the image.

  When the corrected luminance characteristics in the face luminance ranges Y_avr−Y_range to Y_avr + Y_range are set as described above, the range 0 to Y_avr−Y_range having the next lower luminance than the face luminance range and the range Y_avr + Y_range to the luminance higher than the face luminance are set. A corrected luminance characteristic is set for 255 (step S18). First, for a range whose luminance is lower than the face luminance range, a straight line passing through a predetermined point at the point of luminance value 0 and connecting with the corrected luminance characteristic of the determined face luminance range at the boundary point Y_avr−Y_range with the face luminance range is shown. Ask. In this embodiment, as shown in FIG. 6, it is determined that (0, 0) is passed at a luminance value of 0. Next, for a range with higher brightness than the face brightness range, a straight line passing through a predetermined point at the brightness value 255 and connecting with the corrected brightness characteristic of the face brightness range at the boundary point Y_avr + Y_range with the face brightness range is obtained in the same manner as described above. . In the present embodiment, as shown in FIG. 6, it is determined that the luminance value 255 passes (255, 255).

  In this way, the range of luminance values 0 to 255 is divided into a face luminance range, a range where the luminance is lower than the face luminance range, and a range where the luminance is higher than the face luminance range, and the corrected luminance characteristics in each luminance range are It is determined that the boundary points are connected, and finally a corrected luminance characteristic including three straight lines is created as shown in FIG.

  Although the luminance signal may be corrected using the corrected luminance characteristic obtained as described above, a portion where two straight lines having different inclinations are joined at the boundary point. Therefore, if the slope of the corrected luminance characteristic changes abruptly at the boundary point, a pseudo contour is generated as described above. Therefore, in the first embodiment, smoothing is further performed at two boundary points of the corrected luminance characteristic. As a smoothing method, for example, a spline method can be considered. In this embodiment, as shown in FIG. 7, a predetermined range near the boundary point is set as a smoothing range, a circle in contact with two straight lines sandwiching the boundary point is calculated, and the circular arc is replaced with a corrected luminance characteristic for smoothing. I do.

  FIG. 8 shows a corrected luminance characteristic obtained by smoothing the two boundary points in FIG. 6 by the above method. This correction brightness characteristic corrects the average brightness value Y_avr of the face to a predetermined correction brightness value Y'_avr and keeps the slope constant within the face brightness range, thereby realizing appropriate brightness correction for the face area. In addition, the occurrence of pseudo contours can be suppressed.

  As described above, according to the first embodiment, a method of creating a corrected luminance characteristic that can extract a face from an image and appropriately correct the luminance of the face is disclosed. More specifically, in the first embodiment, the average luminance value of the face area is calculated, a face luminance range including the average luminance value is set, and a straight line having a constant inclination within the range is used as the corrected luminance characteristic. It is possible to suppress the generation of a pseudo contour on the face in the corrected image. Further, by calculating an appropriate corrected luminance value for the average luminance value and setting the corrected luminance characteristic within the face range so as to convert the average luminance value into the calculated corrected luminance value, an appropriate luminance correction can be performed. realizable. In addition, a range other than the face luminance range passes through (0, 0) and (255, 255), respectively, defines a straight line connected to the straight line in the face luminance range at the boundary point, and smoothes the boundary point. Thus, it is possible to create a corrected luminance characteristic that does not have a harmful effect such as generation of a pseudo contour or noise over the entire luminance range.

Second Embodiment
When the correction characteristic is obtained by the method of the first embodiment, the correction characteristic cannot be created when the value of Y_avr−Y_range is less than 0 or when the value of Y_avr + Y_range is greater than 255. Therefore, in the second embodiment, a luminance correction method in such a case will be described. In the following, a case where the value of Y_avr−Y_range is 0 or less, that is, a case where the corrected luminance characteristic is set for an image having an extremely dark face will be described.

  In the first embodiment, a straight line connecting the point Y_avr−Y_range and (0, 0) is part of the correction characteristic. However, when the value of Y_avr−Y_range is 0 or less, a straight line in this range cannot be created. Therefore, in the second embodiment, the point P1 (Y_avr, Y′_avr) and (0, 0) are first connected to create a straight line extending to Y_avr + Y_range. Here, the intersection of the straight line and Y_avr + Y_range is P2. Next, in the range of Y_avr + Y_range to 255, a straight line that passes through (255, 255) and is connected to the straight line at Y_avr + Y_range (a straight line that is connected to the straight line at point P2) is created. After that, smoothing is performed at the intersection of the two straight lines as described above. In this way, one correction characteristic based on the first embodiment can be created. This is called a contrast emphasis characteristic 92 in FIG. 9 because the slope of the straight line of the corrected luminance characteristic in the face luminance range is constant.

  However, when this contrast-oriented characteristic 92 is used, a correction characteristic having a very high contrast in the luminance range of 0 to Y_avr + Y_range is generated, and a pseudo contour and noise are generated in the face area in the image. Therefore, in the second embodiment, a correction characteristic emphasizing gradation is created in addition to the contrast emphasis characteristic 92, and one correction characteristic is created by combining these two correction characteristics.

  In this embodiment, a gamma curve that passes through (Y_avr, Y′_avr) and (0, 0), (255, 255) is used as the correction characteristic that emphasizes gradation. This is shown in FIG. Compared with the contrast emphasis characteristic 92, the tone emphasis characteristic 91 is a correction characteristic emphasizing gradation because the inclination in the face luminance range is generally small and the inclination does not change greatly.

  If only the contrast correction characteristic is used, white spots occur in a portion with high luminance, and the portion with low luminance becomes black and the dark portion of the face does not become bright. In addition, when only the tone emphasis characteristic is used, the brightness does not change drastically compared to the original image in the high luminance part, but the image is not sharp compared with the contrast emphasis characteristic around the average luminance of the face. It will be finished as if it was covered. From this, the two correction characteristics are combined to generate a correction characteristic. In the synthesis of a plurality of correction characteristics, a method of weighted averaging the plurality of correction characteristics over the entire luminance range from 0 to 255 can be cited. In the present embodiment, the weighting method is further varied depending on the luminance range. That is, as shown in FIG. 9, the entire luminance range is divided into a plurality of ranges, and a weighting coefficient is set for each range and averaged. By doing so, the correction characteristic can be adjusted by the luminance value. FIG. 9 shows a state in which the entire luminance range is divided into two ranges of 0 to Y_avr and Y_avr to 255 and the weighting method is changed for each.

In the following, the weighting coefficient is obtained by taking a backlight image as an example. First, since the dark part of the face area is distributed from 0 to Y_avr, it is necessary to brighten the contrast while suppressing the generation of noise. Therefore, in this range, the two correction characteristics are combined at 1: 1. Incidentally, if the contrast-oriented correction characteristic 91 is f (Y) and the gradation-oriented correction characteristic 92 is g (Y), the combined correction characteristic Y ′ (93) is
Y ′ = (f (Y) + g (Y)) / 2 (Formula 2)
It is expressed.

On the other hand, in the range of Y_avr to 255, since the background portion is mainly distributed in the backlight image, it is necessary to suppress the luminance as much as possible. Therefore, as shown in FIG. 9, the tone emphasis curve having a low correction characteristic value in this range is weighted and synthesized. For example, if the ratio is 1: 3, the combined correction characteristic Y ′ is
Y ′ = (f (Y) + 3 × g (Y)) / 4 (Formula 3)
It can be expressed as.

  In the second embodiment, the composition ratio is determined by taking a backlit image as an example. However, by changing the composition ratio according to the characteristics of the image, such as an underexposed image or a highlight image with strong flash, each image can be changed. Appropriate correction can be performed. Further, in the second embodiment, the point at which the combination ratio is switched is Y_avr, but it may be Y_avr + Y_range, or three or more switching ranges (for example, switching with both Y_avr and Y_avr + Y_range) may be set.

  In this way, correction characteristics that could not be created in the first embodiment can be created. Note that the generation of the correction characteristic by combining a plurality of characteristics described in the second embodiment can be applied even in a state where the face luminance range does not protrude from the entire luminance range. That is, the present invention can be applied to the correction characteristic generated in the first embodiment. In this case, for example, a gamma curve passing through (0, 0), (Y_avr, Y′_avr) and (255, 255) in FIG. 8 is set, and the correction characteristic and gamma curve shown in FIG. 8 are synthesized. Get correction characteristics. At this time, the weight is changed with Y_avr or the like as a boundary.

  As described above, according to the second embodiment, a curve emphasizing contrast and a curve emphasizing gradation are created, and a weighting coefficient is determined for each of the divided luminance ranges, thereby synthesizing two correction characteristics. Thus, it is possible to create a corrected luminance characteristic in which generation of pseudo contours and noise is suppressed.

<Third Embodiment>
In the third embodiment, the luminance is corrected by the correction characteristics described above in the first and second embodiments, and then saturation correction is performed. The saturation correction of this embodiment will be described below.

First, it is assumed that a corrected luminance characteristic is obtained by the method described in the first or second embodiment, luminance correction is performed on an image, and a corrected luminance value Y ′ is obtained for all pixels. In the third embodiment, saturation correction (color difference correction) is further performed. Here, the saturation value S uses the color difference values Cb and Cr, and the following equation 4,
S = (Cb ² + Cr ² ) ^1/2 (Formula 4)
It is obtained by.

  In general, saturation correction is uniformly applied to the entire image independently of luminance correction. For example, when the corrected saturation value S ′ for the saturation value S is obtained, it is generally calculated by the equation S ′ = 1.3 × S. In this case, saturation correction is applied to pixels (regions) that have not been subjected to luminance correction, and the color may be different from the original image as a whole. Alternatively, as described in Patent Document 3 (Japanese Patent Laid-Open No. 2000-102033), the luminance correction rate of the pixel is set as S ′ = (Y ′ / Y) × S from the luminance value Y and the corrected luminance value Y ′. There is also a method of performing saturation correction according to Y ′ / Y. On the other hand, there is a method in which object extraction is performed in an image and luminance correction is uniformly performed only on the extraction region. In the present embodiment, as in the method of Patent Document 3, after performing luminance correction, the degree of pixel saturation correction is also changed according to the degree of change in luminance correction. The difference from the method of Patent Document 3 is that, as described below, the degree of change in luminance correction is multiplied by a coefficient to perform saturation correction with a high degree of freedom.

In the saturation correction, first, a luminance correction amount is obtained. The luminance correction amount is a ratio of the corrected luminance value to the original luminance value, and is Y ′ / Y. Further, a normalized coefficient a that takes a range of 0 ≦ a ≦ 1 is set, and the luminance correction amount Y ′ / Y is multiplied. That is, the coefficient a is a coefficient representing the degree of cause that the luminance correction amount Y ′ / Y exerts on the saturation correction. Then, the term (1-a) is added to Y ′ / Y × a to obtain a coefficient for obtaining the corrected saturation value S ′. From the above, the saturation correction formula is
S ′ = ((Y ′ / Y) × a + 1−a) × S (0 ≦ a ≦ 1) (Formula 5)
become that way.
By doing so, the saturation correction is enhanced as the luminance correction amount is increased, and when the luminance correction is not performed, the saturation correction is not performed.

In the saturation correction, the above correction processing is performed on the saturation value S, but when the hue is not corrected,
Cb ′ = ((Y ′ / Y) × a + 1−a) × Cb
Cr ′ = ((Y ′ / Y) × a + 1−a) × Cr
Thus, it is equivalent even if correction processing is performed on the color difference values Cb and Cr.

  As can be seen from Equation 5 above, as the value of Y ′ increases compared to Y, the coefficient portion related to S also increases. In other words, the saturation correction is more strongly applied as the luminance correction amount is larger. On the other hand, when Y ′ = Y, that is, when there is no luminance correction, S ′ = S, and saturation correction is not performed.

  In the third embodiment, the saturation correction is performed according to the previously performed luminance correction amount. That is, a pixel whose luminance is corrected corrects the saturation, and a pixel whose luminance is not corrected does not correct the saturation. Further, if a region to be corrected in an image is selected with a mouse or the like, and the luminance correction characteristics described in the first or second embodiment are applied to the selected region, pixels in the selected region The luminance and the saturation are corrected for the pixel, and the luminance and the saturation are not corrected for the pixels outside the selected region. In this way, for pixels that are to be corrected compared to the original image, saturation correction is performed by performing luminance correction, and for pixels that are not to be corrected compared to the original image, saturation correction is also performed by not performing luminance correction. It can be controlled not to do so.

  The coefficient a multiplied by Y ′ / Y can be arbitrarily set. Increasing the value of a can increase the cause of Y ′ / Y, and conversely, decreasing the value of a can decrease the cause. Further, when a = 0, it means that saturation correction is not performed. Thereby, saturation correction with a high degree of freedom can be performed.

  As an application example of the third embodiment, backlight correction processing can be considered. First, a face area is extracted from an image by face extraction processing. This may be based on image frequency analysis, filtering processing, or skin color detection method. Thereafter, brightness correction is performed on the face area that appears dark as shown in FIG.

  After the luminance correction is completed, the saturation correction is performed. In the case of a backlight image, since the face area is originally dark, the saturation value may remain very small even if the luminance value is increased. Therefore, it is necessary to perform saturation correction after luminance correction to further improve the appearance of the corrected image. Further, as shown in FIG. 10, since the background of the backlight image is bright, it is not necessary to perform luminance correction or saturation correction for that portion. Therefore, it is only necessary to perform saturation correction only on the luminance-corrected region. Further, by increasing the saturation correction amount in accordance with the luminance correction amount, more accurate backlight correction can be performed.

  As described above, according to the third embodiment, as shown in Equation 5, by performing saturation correction according to the luminance correction amount, the pixel whose luminance has been corrected is also corrected for saturation and is not corrected for luminance. The pixel can be controlled so as not to correct the saturation.

<Fourth embodiment>
As described in the third embodiment, when only the luminance value Y is corrected in the YCbCr space, it may appear that the saturation is visually reduced. This is because the saturation is not maintained when the YCbCr space is converted to a uniform color space (L * a * b * space). Therefore, after correcting the luminance value Y, it is necessary to correct the saturation accordingly. In the fourth embodiment, a method for performing saturation correction from the viewpoint of maintaining color differences in the L * a * b * space will be described.

  Consider the case of converting from sRGB space to L * a * b * space. As shown in FIG. 11, the values in the original YCbCr space are Y_orig, Cb_orig, Cr_orig, and the saturation value is S_orig. When each value is converted to sRGB space and further converted from sRGB space to L * a * b * space, L * a * b * values corresponding to the YCbCr values are expressed as L * _orig, a * _orig, b * _orig. On the other hand, the corrected luminance value of Y_orig is Y_corr, and the L * a * b * values corresponding to Y_corr, Cb_orig, and Cr_orig are L * _corr, a * _corr, and b * _corr. Also, the corrected brightness value L * _corr and the YCbCr values corresponding to the L * a * b * values composed of the original color difference values a * _orig and b * _orig are Y_after, Cb_after, Cr_after, and the saturation value is S_after. To do. Further, dY and dS are used as the amounts of change in luminance and saturation, and dY = Y_corr−Y_orig and dS = S_after−S_orig.

  In the fourth embodiment, the skin color region is specified, and the conversion between the YCbCr value and the L * a * b * value is statistically examined. Here, the skin color area in the YCbCr space was set to luminance 30 to 200, hue 100 to 150 °, and saturation 10 to 80. If this skin color area is taken from 10 to 100 in the range of 10 to 100 while keeping the hue and saturation constant, statistics are taken and converted from sRGB space to L * a * b * space. A graph like 12 was obtained. Here, the horizontal axis is dY / Y_orig, and the vertical axis is dS / S_orig.

From this, it can be seen that there is a correlation between the increase in luminance and the increase in saturation. That is, in the graph of FIG.
dS / S_orig = a × dY / Y_orig (Formula 6)
Can be approximated. That is, when Expression 6 is satisfied, the color difference is maintained before and after the luminance correction. Equation 6 further
S_after = (Y_corr / Y_orig × a + 1−a) × S_orig (Expression 7)
Can be rewritten. This is equivalent to Formula 5 described in the third embodiment.

  As described above, when only the luminance is corrected brightly in the YCbCr space, the skin color tends to appear to be less saturated than the original image when confirmed on the monitor or printed matter. Therefore, it is only necessary to correct the saturation according to the luminance correction degree using the above equation 7 that corrects the saturation reduction.

  The coefficient a shown in Equation 7 may be a value calculated from the graph of FIG. 12 or a value empirically obtained by comparing the uncorrected and corrected images on a monitor or printed matter.

  In general, it has been statistically found that in a backlit image, the face appears dark and takes a luminance value of 30 to 130. Normally, backlight correction increases the brightness of the face area by about 10 to 80 to brighten the face, but in that case, even if only the brightness value is corrected as described above, the saturation is reduced visually. It was. Therefore, based on this embodiment, such a problem can be solved by correcting the saturation according to the luminance.

  As described above, as described in the third and fourth embodiments, by adjusting the saturation correction amount in accordance with the luminance correction amount, it is possible to perform image conversion with higher accuracy than in the related art. It was.

  For example, in backlight correction, when luminance correction is performed, a dark subject is corrected with the largest correction amount, while the background is corrected with a small correction amount. Therefore, when performing saturation correction, the background is not to be corrected as compared with the original image, and the saturation correction is not performed so much, and the luminance correction amount is associated with the subject to be corrected.

  Further, when the skin color is brightly changed in the YCbCr space, there is a statistical tendency to decrease the saturation in the uniform color space. Therefore, saturation correction is performed according to the luminance correction amount so that the saturation is maintained as compared with the original image.

  As described above, when only the luminance is corrected brightly in the YCbCr space, the skin color tends to appear to be less saturated than the original image when viewed on the monitor or printed matter as shown in FIG. According to the fourth embodiment, since the saturation is also corrected according to the luminance correction degree using Expression 6, it is possible to correct the saturation reduction.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the figure) that realizes the functions of the above-described embodiment is directly or remotely supplied to the system or apparatus, and the computer of the system or apparatus Is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a block diagram which shows the structure of the image processing system by 1st Embodiment. It is a figure which shows the conversion process between the RGB signal and YCbCr signal which comprise an image. It is a figure showing the face area extracted by the rectangle. It is a figure which shows the brightness | luminance histogram of the extracted face area. It is a figure showing the function which calculates 1st correction magnification k1 from face average luminance value Y_avr. It is a figure showing the correction | amendment luminance characteristic before smoothing in 1st Embodiment. It is a figure explaining the smoothing process in the intersection of two straight lines using a circle. FIG. 8 is a diagram illustrating a final corrected luminance characteristic obtained by performing the smoothing process of FIG. 7 on the corrected luminance characteristic of FIG. 6. It is a figure showing the correction | amendment luminance characteristic by 2nd Embodiment. It is a figure showing the area | region which does not carry out the area | region which correct | amends brightness | luminance and saturation in a backlight image. It is a conversion figure which shows the corresponding value in L * a * b * space when only the brightness | luminance is correct | amended in YCbCr space. It is a figure showing the relationship between luminance change rate dY / Y and saturation change rate dS / S. It is a flowchart explaining the correction process by 1st Embodiment.

Claims (13)

An image processing method for correcting image brightness,
An obtaining step of extracting a human face area from the image and obtaining an average luminance value of the face area based on a luminance value of a pixel included in the face area;
A calculation step of calculating a corrected luminance value that is a corrected luminance value for the average luminance value of the face region based on the average luminance value of the face region and the average luminance value of the entire image;
A setting step of setting a range from a value obtained by subtracting a predetermined luminance value from the average luminance value of the face area to a value obtained by adding a predetermined luminance value to the average luminance value of the face area as a face luminance range When,
Generating a corrected luminance characteristic in which the inclination is constant within the face luminance range and an average luminance value of the face area is converted into the corrected luminance value;
An image processing method comprising: a correction step of correcting the luminance of the image based on the corrected luminance characteristic generated in the generation step.   In the calculating step, a first correction magnification is calculated from an average luminance value of the face area, a second correction magnification is calculated from an average luminance value of the entire image, and the first correction magnification and the second correction magnification are calculated. The image processing method according to claim 1, wherein the correction luminance value is calculated by multiplying a correction magnification by an average luminance value of the face area.   In the generating step, the correction luminance characteristic outside the face luminance range is generated so that the correction luminance characteristic outside the face luminance range is continuous with the correction luminance characteristic within the face luminance range. The image processing method according to claim 1.   In the generating step, the face luminance range has a corrected luminance characteristic having a shape obtained by interpolating a connection portion between the corrected luminance characteristic within the facial luminance range and the corrected luminance characteristic set outside the facial luminance range with a curve. The image processing method according to claim 3, wherein corrected luminance characteristics inside and outside the face luminance range are generated. In the generation step, a first straight line having a predetermined inclination in the face luminance range and passing through the points of the average luminance value and the corrected luminance value is set.
In the range below the face luminance range, a second line connected to the first line is set through the point where the input luminance value and the corrected luminance value are minimum,
In a range equal to or greater than the face luminance range, a third straight line connected to the first straight line is set through the point where the input luminance value and the corrected luminance value are maximum, and the first, second, second The image processing method according to claim 1, wherein the corrected luminance characteristic is generated based on three straight lines.   In the generating step, the corrected luminance characteristic is generated by interpolating the connecting portion between the first straight line and the second straight line and the connecting portion between the first straight line and the third straight line with respective curves. The image processing method according to claim 5, wherein: In the generating step, when a part of the face luminance range is 0 or less,
A gamma curve that passes through the average luminance value of the face area and the point of the corrected luminance value is a first correction characteristic,
A point where the input luminance value and the corrected luminance value are minimum in the face luminance range, a first straight line passing through the average luminance value of the face region and the point of the corrected luminance value, and the face luminance range or more Correction characteristics obtained on the basis of the second straight line connected to the first straight line in the face luminance range through the point where the input luminance value and the corrected luminance value are maximum in the region As a correction characteristic,
The image processing method according to claim 1, wherein the corrected luminance characteristic is generated by performing weighted averaging of the first correction characteristic and the second correction characteristic.   The image processing method according to claim 6, wherein in the generation step, the second characteristic is generated by interpolating a connection portion between the first straight line and the second straight line with a curve.   9. The image processing method according to claim 1, wherein in the correcting step, the saturation of the pixel is corrected in accordance with a correction amount to the corrected luminance value. In the correction step, when the luminance value before correction is Y, the luminance value after correction is Y ′, and the saturation value before correction is S, the saturation value after correction is S ′ = (Y '/ Y × a + (1-a)) × S (0 ≦ a ≦ 1)
The image processing method according to claim 9, wherein the image processing method is obtained by: An image processing apparatus for correcting the brightness of an image,
Acquiring means for extracting a human face area from the image and obtaining an average luminance value of the face area based on a luminance value of a pixel included in the face area;
Calculation means for calculating a corrected luminance value that is a corrected luminance value for the average luminance value of the face region based on the average luminance value of the face region and the average luminance value of the entire image;
Setting means for setting a range from a value obtained by subtracting a predetermined luminance value from the average luminance value of the face area to a value obtained by adding a predetermined luminance value to the average luminance value of the face area as a face luminance range When,
Generating means for generating a corrected luminance characteristic in which an inclination is constant within the face luminance range and an average luminance value of the face region is converted into the corrected luminance value;
An image processing apparatus comprising: a correction unit that corrects a luminance value of the image based on the correction characteristic luminance characteristic generated by the generation unit.   A control program for causing a computer to execute the image processing method according to any one of claims 1 to 10.   A computer-readable storage medium storing a control program for causing a computer to execute the image processing method according to claim 1.

Images