WO2007032324A1

WO2007032324A1 - Imaging device, image processing device, image processing method, and image processing program

Info

Publication number: WO2007032324A1
Application number: PCT/JP2006/318009
Authority: WO
Inventors: Hiroaki Takano; Tsukasa Ito
Original assignee: Konica Minolta Photo Imaging, Inc.
Priority date: 2005-09-12
Filing date: 2006-09-12
Publication date: 2007-03-22
Also published as: JP2007081550A

Abstract

An imaging device includes: an index calculation unit for calculating the natural light index representing the outdoor imaging degree by the natural light, a brightness ratio index representing a dynamic range, and an exposure index; a brightness compensation amount calculation unit for setting a temporary correction amount for the captured image data according to a brightness analysis value of the captured image data and a brightness reproduction target value, setting a mixture coefficient as a weight coefficient to be multiplied on each of the temporary correction amounts according to the natural light index, and calculating the brightness correction amount of the captured image data according to the mixture coefficient and the temporary correction amount; and a brightness correction curve generation unit for generating a brightness correction curve for converting the captured image data according to the brightness correction amount and the respective indexes calculated by the index calculation unit.

Description

Specification

Imaging apparatus, image processing apparatus, image processing method, and image processing program

The present invention relates to an imaging device, an image processing device, an image processing method, and an image processing program that perform brightness correction processing of captured image data.

Background art

Conventionally, a captured image obtained by photoelectrically reading an image acquired by an imaging device such as a digital camera or an image recorded on a photographic film such as a negative film is output by an output device such as a printer. Output (playback) is performed. In the case of outputting (reproducing) a captured image using an output device in this way, the so-called “cover dodging” in which the low luminance portion and the high luminance portion of the image area are separately corrected and the dynamic range of the entire image is compressed. In other words, there is known a method for suppressing the collapse of the bright part and the dark part, which occurs at the limit of the reproduction range of the output device by applying the process (see, for example, Patent Document 1).

[0003] On the other hand, the brightness of captured image data is automatically adjusted so that the brightness of a main subject (for example, a human face region) in captured image data is appropriately finished. In such a brightness adjustment method, a correction amount and a correction curve for correcting the brightness of the captured image data are set based on the brightness obtained by analyzing the main subject of the captured image data. A method is known (see, for example, Patent Document 2).

Patent Document 1: Japanese Patent Laid-Open No. 2005-72930

Patent Document 2: JP 2002-247393 A

Disclosure of the invention

Problems to be solved by the invention

[0004] When the brightness correction amount and the correction curve for correcting the brightness of the captured image data are set based on the brightness of the main subject of the captured image data, In a scene with a large dynamic range, such as a shooting scene, there is a problem that as the brightness correction amount is increased, the bright and dark portions are crushed, the high-saturation color is saturated and crushed, and the hue is changed. [0005] When the above-described “drilling” process is applied as a solution to such a problem, it is difficult to maintain the brightness of the human face area properly. Improvements have been desired because of various tastes. Furthermore, there is no difference in dynamic range between backlight and strobe shooting scenes, but in the first place the brightness and color types that should be wary of crushing in each scene are determined whether to make the image brighter or darker. There is an essential problem that depends on the accuracy of how the brightness of the face area is detected o

An object of the present invention is to appropriately correct the brightness of captured image data.

Means for solving the problem

[0007] In order to solve the above-described problem, the invention described in claim 1 is a process for photographing a subject to obtain photographed image data and optimizing the brightness of a main subject in the photographed image data. An imaging device for performing

Calculates a natural light index that represents the degree of outdoor shooting with natural light in the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to

Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary corrections for the captured image data First temporary correction amount setting means for setting the amount;

First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;

A second temporary correction amount for setting a second temporary correction amount for the photographed image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts. Setting means;

Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weighting factor. 2nd mixing coefficient setting means for setting the mixing coefficient;

A second mixing factor set by the second mixing factor and the at least one first provisional compensation. Correction amount calculating means for calculating a brightness correction amount of the captured image data based on a positive amount and the second temporary correction amount;

A correction curve generating means for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating means, the natural light index, and the brightness ratio index;

Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means.

The invention described in claim 2 is an imaging apparatus that captures captured image data by capturing a subject, and performs processing for optimizing the brightness of a main subject in the captured image data.

Correction for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A quantity calculating means; Based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index, and the saturation of the captured image data is higher than a predetermined threshold value, V / A suppression coefficient setting means for setting a suppression coefficient for suppressing the brightness correction amount of the data, and a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation means by the suppression coefficient Multiplication means for calculating

A correction curve generation means for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication means, the natural light index, and the luminance ratio index;

[0009] The invention described in claim 3 is the imaging device according to claim 1 or 2, wherein

The index calculation means divides the photographed image data into regions composed of a combination of predetermined brightness and hue, and calculates a first occupancy ratio indicating the proportion of the entire photographed image data for each of the divided regions. The first calculation process and the photographed image data are divided into predetermined areas having a combination power of the distance from the outer edge of the screen of the photographed image data and the brightness, and each divided area occupies the entire photographed image data Execute at least one of the second calculation processes for calculating the second occupation ratio indicating the ratio,

Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;

By multiplying the calculated first occupancy and Z or second occupancy by the deviation amount by a coefficient set in advance according to the shooting conditions, the natural light index, the luminance ratio index, The exposure index is calculated.

[0010] The invention described in claim 4 is an image processing apparatus that performs processing for optimizing the brightness of a main subject in captured image data.

The brightness analysis value of the captured image data, the brightness reproduction target value, and the index calculation First temporary correction amount setting means for setting a plurality of first temporary correction amounts for the captured image data based on the brightness ratio index and the exposure index calculated by the stage;

Correction for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A quantity calculating means;

The invention described in claim 5 is an image processing apparatus for performing processing for optimizing the brightness of a main subject in captured image data,

Based on the natural light index calculated by the index calculating means, the plurality of first temporary complements. First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each positive amount;

Based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index, and the saturation of the captured image data is higher than a predetermined threshold value, V / A suppression coefficient setting means for setting a suppression coefficient for suppressing the brightness correction amount of the data, and a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation means by the suppression coefficient Multiplication means for calculating

The invention according to claim 6 is the image processing apparatus according to claim 4 or 5, wherein

The index calculation means divides the photographed image data into regions composed of a combination of predetermined brightness and hue, and calculates a first occupancy ratio indicating the proportion of the entire photographed image data for each of the divided regions. The first calculation process and the photographed image data are divided into predetermined areas having a combination power of the distance from the outer edge of the screen of the photographed image data and the brightness, and each of the divided areas occupies the entire photographed image data Second occupancy showing percentage Execute at least one of the second calculation processes for calculating the rate,

The invention described in claim 7 is an image processing method for performing a process of optimizing the brightness of a main subject in captured image data.

Calculates a natural light index that represents the degree of outdoor shooting with natural light in the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation process to perform,

Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and exposure index calculated by the index calculation process, a plurality of first temporary corrections for the captured image data A first provisional correction amount setting step for setting the amount;

A first mixing coefficient setting step for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculation step;

Based on the first mixing coefficient set in the first mixing coefficient setting step and the plurality of first temporary correction amounts, a second temporary correction amount for setting a second temporary correction amount for the captured image data A setting process;

Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation step, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weighting factor. (2) a second mixing coefficient setting step for setting a mixing coefficient;

Correction for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A quantity calculation step;

A correction curve generating step for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating step, the natural light index, and the brightness ratio index; And a correction step of correcting the brightness of the captured image data according to the correction curve generated by the correction curve generation step.

The invention described in claim 8 is an image processing method for performing a process of optimizing the brightness of a main subject in captured image data.

Based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index, and the saturation of the captured image data is higher than a predetermined threshold value, V / A suppression coefficient setting step for setting a suppression coefficient for suppressing the brightness correction amount of the data, and a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation step by the suppression coefficient A multiplication step for calculating

New brightness correction amount obtained by the multiplication step, the natural light index, and the luminance A correction curve generating step for generating a correction curve for converting the captured image data based on the ratio index;

And a correction step of correcting the brightness of the captured image data according to the correction curve generated by the correction curve generation step.

[0015] The invention according to claim 9 is the image processing method according to claim 7 or 8,

In the index calculation step, the photographed image data is divided into regions that are combinations of predetermined brightness and hue, and a first occupancy ratio that indicates a proportion of the entire photographed image data for each of the divided regions The first calculation process for calculating the image data and the captured image data are divided into predetermined areas having a combination power of distance and brightness from the outer edge of the screen of the captured image data, and the captured image data is divided into the divided areas. Execute at least one of the second calculation processes for calculating the second occupancy ratio indicating the proportion of the total,

[0016] An image processing program according to claim 10 is stored in a computer.

Calculates a natural light index that represents the degree of outdoor shooting with natural light in the captured image data, a luminance ratio index that represents the size of the dynamic range of the captured image data, and an exposure index that represents the exposure photographing level that results from the exposure setting at the time of shooting. An index calculation function,

Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation function, a plurality of first temporary corrections for the captured image data A first provisional correction amount setting function for setting the amount;

A first mixing coefficient setting function for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculation function;

Second temporary correction for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting function and the plurality of first temporary correction amounts. The volume setting function,

Based on the natural light index, the brightness ratio index, and the exposure index calculated by the index calculation function, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weighting factor. 2nd mixing coefficient setting function to set the mixing coefficient,

Correction for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A quantity calculation function;

A correction curve generation function for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculation function, the natural light index, and the brightness ratio index;

And a correction function for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation function.

The image processing program according to claim 11 is stored in a computer.

A second temporary correction amount for setting a second temporary correction amount for the photographed image data based on the first mixing coefficient set by the first mixing coefficient setting function and the plurality of first temporary correction amounts. Setting function,

Based on the natural light index, the brightness ratio index, and the exposure index calculated by the index calculation function, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weighting factor. 2nd mixing coefficient setting function to set the mixing coefficient, Correction for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A quantity calculation function,

Based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index, and the saturation of the captured image data is higher than a predetermined threshold value, V / A suppression coefficient setting function for setting a suppression coefficient for suppressing the brightness correction amount of the data, and a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation function by the suppression coefficient A multiplication function for calculating

A correction curve generation function for generating a correction curve for converting the photographed image data based on the new brightness correction amount obtained by the multiplication function, the natural light index, and the luminance ratio index;

[0018] The invention according to claim 12 is the image processing program according to claim 10 or 11, wherein the index calculation function is realized when the index calculation function is realized.

First calculation processing that divides photographed image data into regions that are a combination of predetermined brightness and hue, and calculates a first occupancy ratio that indicates a proportion of the entire photographed image data for each of the divided regions The photographed image data is divided into predetermined areas having a combination power of distance and brightness from the outer edge of the screen of the photographed image data, and the ratio of the divided area to the whole photographed image data is shown for each divided area. Execute at least one of the second calculation processes for calculating the occupancy ratio of 2,

Next, terms in the present invention will be described.

[0020] “Optimizing the brightness of the main subject in the captured image data” means, for example, that the brightness of the human face area, which is the main subject of the captured image data, outputs the captured image data It means that it is reproduced at the optimum brightness on the device (eg printer).

[0021] "Natural light" refers to light that uses the sun as a light source, which is not an artificial light source such as steady light or flash light (referred to as "strobe", "flash", "speed light", etc.). is there. Also known as daylight. Daylight is a term that refers to an artificial light source with a color temperature close to that of daylight (approximately 5500K), and is therefore different from “natural light”.

[0022] "Outdoor shooting degree" (also referred to as "natural light index") estimates whether or not the shot image data was taken using "natural light" and shows the result quantitatively. It is a numerical value. Many scenes that have been shot using “natural light” and that require brightness correction have many backlight scenes. Is expensive. On the other hand, for indoor or strobe shooting scenes taken without using “natural light”, a high-brightness area or low-saturation skin color area at the center of the screen, and a low-brightness area or low-brightness skin color around the screen. Highly saturated skin color tends to be distributed. Based on such experience, it is possible to quantitatively indicate whether or not the image is taken using “natural light” as “outdoor imaging degree”. It is desirable to use multivariate analysis as a statistical processing method that derives one tendency from many variables, such as the brightness at the top of the screen and the number of pixels in the sky blue hue, the brightness at the bottom of the screen and the number of pixels in the skin tone hue.

[0023] “Luminance ratio index indicating the size of the dynamic range” means that the brightness of the main subject area and the background area depends on the light source conditions at the time of shooting such as a backlight scene or strobe shooting scene. It is a value that indicates the degree of difference quantitatively. For example, in a backlight scene, the sky, which is the background area in which the human face area is dark, is a bright image due to the photographer turning the camera in the direction of the sun and shooting a person. That is, if the brightness difference value between the human face area and the background area is obtained, the difference value can be used as one of the indices. In general, it is known to obtain a difference value by drawing a histogram of image difference image data.

The “brightness analysis value” of photographed image data is a numerical value indicating the average brightness of an image obtained by examining the distribution state of pixel values. The “brightness analysis value” is preferably the brightness of the most important subject (main subject) in the captured image data. The human face area is searched by collation in the color system using hue, saturation, and brightness. For higher accuracy For this purpose, it is preferable to use so-called “face detection” in which parts constituting the face, such as the eyes “nose” mouth and face contour, are collated. In addition, an exposure index representing the exposure shooting degree (under shooting or over shooting) resulting from the exposure setting at the time of shooting may be calculated, and this exposure index may be used as the brightness analysis value. Furthermore, the analysis method of the brightness analysis value is not limited to the analysis of the captured image data. For example, the brightness obtained by analyzing the automatic exposure information of the imaging device and the additional information recorded in the automatic exposure information is used. May be.

The “reproduction target value” is, for example, a numerical value indicating the brightness necessary for the human face area, which is the main subject of the captured image data, to be optimally reproduced in the output device that outputs the captured image data. is there. That is, if the “brightness analysis value” of the photographed image data is a value that approximates the “reproduction target value”, it means that the probability that the output device is optimally reproduced is high.

The “first provisional correction amount” is a numerical value that temporarily indicates the correction amount necessary for approximating the “brightness analysis value” to the “reproduction target value”. “Set the first provisional correction amount” means creating a one-dimensional LUT (Look Up Table) that defines the relationship between the “brightness analysis value” and the “first provisional correction amount” in advance. This means that the “first provisional correction amount” is determined based on the one-dimensional LUT and the “brightness analysis value”. Furthermore, “setting multiple first provisional correction values” means “calculating brightness analysis values” under different conditions, or setting different “reproduction target values” (defining multiple 1D LUTs). Or at least two “first provisional correction amounts” temporarily.

In the present invention, two one-dimensional LUTs for setting “first provisional correction amount” are defined, assuming brightness correction of a human face area in a backlight scene and a flash photography scene. For intermediate scenes between backlight scenes and flash shooting scenes, at least one 1D LUT for setting the “first provisional correction amount” that assumes brightness correction of the entire image in under and over shooting scenes is used. It is desirable to define.

[0028] In order to fully demonstrate the effects of the present invention, when creating a one-dimensional LUT for setting the "first provisional correction amount", bright and dark portions may be crushed, or high-saturation colors may be saturated and crushed. Prepare a large number of backlight scenes and flash photography scenes where phenomena such as hue change actually occur as “learning images” (also called “teacher data”). It is desirable to do this while observing the occurrence of these phenomena.

[0029] Based on "brightness reproduction target value" and "brightness ratio index" !, "Set the first temporary correction amount" is the dynamic range power of the shot image data. This means that the brightness correction amount set based on the brightness of the main subject is weakened and the brightness correction amount is set based on the brightness ratio index so that it falls within the dynamic range of the output device. .

[0030] “Setting the first mixing coefficient” based on the natural light index is a one-dimensional definition that defines in advance the relationship between the “natural light index” and the “first mixing coefficient” by which each first temporary correction amount is multiplied. This means that LUTs, etc., are created for the number of first temporary correction amounts, and based on this one-dimensional LUT and the `` natural light index '', the `` first mixing coefficient '' to be multiplied by each first temporary correction amount is determined. To do. Note that the first mixing factor may be adjusted based on the luminance ratio index!

[0031] For example, when the natural light index indicates a high outdoor photographing value, a one-dimensional LUT for setting the “first provisional correction amount” assuming brightness correction of a human face area in a backlight scene ( Define one-dimensional LUT so that the “first mixing coefficient” of input: flesh tone average luminance, output: first temporary correction amount) becomes high.

[0032] On the other hand, if the natural light index indicates a low-light outdoor shooting degree, a one-dimensional LUT for setting the “first provisional correction amount” assuming brightness correction of the human face area in the flash shooting scene. Define a one-dimensional LUT so that the “first mixing coefficient” of (input: flesh color average brightness, output: first temporary correction amount) is high.

[0033] Also, when the natural light index shows an ambiguous outdoor shooting degree value, it is assumed that the "first provisional correction amount" is set assuming brightness correction of the entire image in the under shooting scene and over shooting scene. Define the one-dimensional LUT so that the “first mixing coefficient” of the one-dimensional LUT (input: overall average brightness or under-over shooting index, output: first temporary correction amount) is high.

[0034] When the photographed image data is in advance a backlight image, the brightness correction amount and the correction curve can be corrected based on the “brightness ratio index”, and the bright and dark parts of the photographed image data can be corrected. It is possible to suppress crushing, crushing due to saturation of a high-saturation color, or a change in hue. However, in the backlight scene and the flash shooting scene, even if the dynamic range is the same, the direction of brightness correction is completely opposite, so that In order to automate the correction process, the first step is based on a luminance ratio index that represents the size of the dynamic range according to a predicted numerical value (natural light index) that quantitatively indicates whether there is a backlit scene force strobe shooting scene. 1 The amount of provisional correction must be defined. Therefore, assuming the brightness correction of the human face area in the backlight scene, the 1st LUT (input: luminance ratio index, output: 1st temporary correction amount) for the “first temporary correction amount” setting Define a one-dimensional LUT that defines the relationship between the “natural light index” and the “first mixing coefficient” so that the “coefficient” becomes higher.

The “second provisional correction amount” is calculated based on the first provisional correction amount and the first mixing coefficient.

It is defined as (1). In Equation (1), m first temporary correction amounts among a plurality of first temporary correction amounts set by the first temporary correction amount setting means are calculated, and the first temporary correction amount of the cell is calculated as The first mixing coefficient to be multiplied by key—auto [i] and the first temporary correction amount key—auto [i] is wgt [i].

[0036] [Equation 1] m— 1

2nd temporary correction amount = X key one auto [i] x wgt [i] (l)

i = 0

“Generate a correction curve for converting captured image data based on a brightness correction amount, a natural light index, and a luminance ratio index” is a scene captured using “natural light”. The brightness of the main subject (e.g., human face area) of the captured image data is properly finished according to the predicted numerical value (natural light index) that quantitatively indicates whether or not power and the luminance ratio index that indicates the size of the dynamic range. On the other hand, by changing the shape of the brightness correction curve, it is possible to prevent the bright and dark areas from being crushed, high-saturation colors to be saturated and crushed, and the hue from being changed.

[0038] Scenes that are shot using “natural light” and require brightness correction have many backlighting scenes. The brightness correction amount is negative, and the bright part is shattered and the high-saturation sky blue is saturated. There is a high probability that the hue will change to green. On the other hand, for indoor shooting or strobe shooting scenes that were shot without using “natural light”, the brightness correction amount is positive, dark areas are crushed, high-saturated skin color is saturated, yellow, red, etc. The hue tends to change. Based on such experience, the shape of the brightness correction curve is changed. [0039] Defines a soft curve on the highlight side, assuming that when the natural light index shows a high outdoor shooting degree, the bright part in the backlight scene will be crushed or the sky will be saturated with high saturation. To do. On the other hand, if the natural light index shows a low outdoor shooting value, define a soft curve on the shadow side assuming that the dark area in the flash photography scene will be crushed or the skin color of high saturation will be saturated and crushed. .

[0040] The "predetermined hue" is a hue of a high-saturation pixel that has a high probability of being saturated and crushed or changing to a hue such as cyan or green. is there. “Predetermined hue set based on the natural light index” means that the hue of the high-saturation pixel that counts the number of pixels is changed based on the natural light index

[0041] Note that the brightness correction amount calculation means analyzes the additional information recorded in the tag area of the captured image data, and sets a new brightness correction amount by using the analyzed additional information as a supplement. Please do it.

The invention's effect

According to the present invention, while correcting the brightness of the main subject in the photographed image data, it is possible to prevent the bright part and the dark part from being crushed, the collapsing due to high saturation color saturation, and the hue change.

Brief Description of Drawings

FIG. 1 is a block diagram showing a main part configuration of an imaging apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing an internal configuration of an image processing unit.

FIG. 3 is a block diagram showing an internal configuration of a brightness correction amount calculation unit in Embodiment 1 of the present invention.

[Figure 4] LUT showing the relationship between the luminance ratio index and the temporary correction amount key—auto [0].

[Figure 5] LUT showing the relationship between the exposure index and the temporary correction amount key—auto [l].

[Fig.6] LUT showing the relationship between skin tone average brightness and provisional correction amount key—auto [2].

[Fig.7] Skin color average luminance or overall average luminance and the temporary correction amount key— auto [3]

[Figure 8] LUT showing the relationship between the brightness ratio index and the provisional correction amount key—auto [4]. 圆 9] LUT indicating the relationship between exposure index and provisional correction amount key—auto [5].

[Figure 10] LUT showing the relationship between natural light index and mixing coefficient wgt [0].

[Fig. 11] LUT showing the relationship between natural light index and mixing coefficient wgt [l].

[Figure 12] LUT showing the relationship between natural light index and mixing coefficient wgt [2].

[Figure 13] LUT showing the relationship between natural light index and mixing coefficient wgt [3].

FIG. 14 is a diagram showing a discrimination map for discriminating a scene type from a natural light index, a luminance ratio index, and an exposure index.

[FIG. 15] A diagram showing a coefficient calculation table representing the values of the mixing coefficient wgt [4] to [6] for each scene type.

FIG. 16 is a flowchart showing brightness correction processing executed in the image processing unit of the first embodiment.

[FIG. 17] A brightness correction curve (a) when the scene type is direct light and a brightness correction curve when the scene type is backlight (b).

[18] A flowchart showing the index calculation process.

[19] A flowchart showing a first occupancy ratio calculation process for calculating a first occupancy ratio for each area of brightness and hue.

FIG. 20 is a diagram showing an example of a program for converting to RGB power HSV color system.

圆 21] Lightness (V) —Hue (H) plane, and region rl and region r2 on the V—H plane.

FIG. 22 is a diagram showing the lightness (V) —hue (H) plane and regions r3 and r4 on the V—H plane. [23] A diagram showing a curve representing a first coefficient for multiplying the first occupancy ratio for calculating the index 1.

[24] A diagram showing a curve representing a second coefficient for multiplying the first occupancy ratio for calculating index 2.

25] A flowchart showing a second occupancy ratio calculation process for calculating the second occupancy ratio based on the composition of the captured image data.

FIG. 26 is a diagram showing areas nl to n4 determined according to the distance from the outer edge of the screen of captured image data.

圆 27] A curve representing the third coefficient for multiplying the second occupancy to calculate index 3 The figure shown according to area | region (nl-n4).

圆 28] A flowchart showing a bias amount calculation process.

圆 29] A block diagram showing the internal configuration of the brightness correction amount calculation unit in Embodiment 2 of the present invention.

FIG. 30 is a flowchart showing brightness correction processing executed in the image processing unit of the second embodiment.

FIG. 31 is a diagram showing a relationship between a natural light index and a hue condition of a high saturation pixel.

圆 32] A block diagram showing the main configuration of an image processing apparatus to which the present invention is applied.

Explanation of symbols

1 lens

2 Aperture

3 CCD

4 Analog processing circuit

5 AZD

6 Temporary memory

7 Image processing section

70 Pretreatment section

71 Brightness correction section

72 Image quality correction section

73 Image analysis unit

74 Indicator calculator

75 Brightness correction amount calculator

76 Brightness correction curve generator

8 Header information processing section

9 Storage device

10 CCD drive circuit

11 Control unit

12 Operation unit 13 Display

14 Strobe drive circuit

15 Strobe

16 Focal length adjustment circuit

17 Autofocus drive circuit

20 Motor

100 imaging device

101 First provisional correction amount setting section

102 First mixing coefficient setting section

103 Second temporary correction amount calculation section

104 Second mixing coefficient setting section

105 Brightness correction amount calculator

106 Suppression coefficient calculator

107 multiplier

200 Image processing device

201 Image input section

202 Image processing unit

203 Image output section

204 Image analysis unit

205 Indicator calculation section

206 Brightness correction amount calculator

207 Brightness correction curve generator

208 Image quality correction section

BEST MODE FOR CARRYING OUT THE INVENTION

[0045] Hereinafter, embodiments 1 and 2 of the present invention will be described in detail with reference to the drawings.

[Embodiment 1]

First, a configuration common to the first and second embodiments will be described.

FIG. 1 shows a main part configuration of an imaging apparatus 100 according to Embodiments 1 and 2 of the present invention. Shoot As shown in FIG. 1, the image apparatus 100 includes a lens 1, an aperture 2, a CCD (Charge Coupled Device) 3, an analog processing circuit 4, an AZD converter 5, a temporary storage memory 6, an image processing unit 7, and a header. Information processing unit 8, storage device 9, CCD drive circuit 10, control unit 11, operation unit 12, display unit 13, strobe drive circuit 14, strobe 15, focal length adjustment circuit 16, automatic focus drive circuit 17, motor 20 Is done.

The optical system of the imaging apparatus 100 includes a lens 1 that can adjust focus, a diaphragm 2 that adjusts the amount of light, and a CCD 3 that is a solid-state imaging device that converts subject information into an electrical signal.

The subject information is imaged on the light receiving surface of the CCD 3 through the lens 1 and the diaphragm 2. CCD3 photoelectrically converts subject information into an electrical signal of an amount corresponding to the amount of incident light for each sensor in CCD3. A timing pulse is output from the CCD drive circuit 10 based on a control signal input from the control unit 11, and an electrical signal (imaging signal) accumulated in the CCD 3 is sequentially output by this timing pulse to generate an analog processing circuit 4 Forwarded to

[0049] The analog processing circuit 4 performs amplification processing of an analog imaging signal and noise reduction processing. A ZD converter 5 converts the analog imaging signal input from the analog processing circuit 4 into a digital imaging signal (R, G, B signal). The digital R, G, B signals output from the AZD transformation 5 are stored in the temporary storage memory 6. Hereinafter, the digital image signal is referred to as “captured image data”.

[0050] The image processing unit 7 displays on the display unit 13 and corrects the brightness of the captured image data used for storage in the recording medium of the storage device 9, crosstalk correction of spectral sensitivity, noise suppression, sharpening, white In addition to image quality improvement processing such as balance adjustment and saturation adjustment, processing such as image size change, trimming, and aspect conversion is performed. In the processing in the image processing unit 7, ONZOFF or the setting of the adaptive amount or the like can be switched according to the operation information of the operation unit 12 force. For example, when operation information specifying brightness correction is input from the operation unit 12, the image processing unit 7 performs brightness correction processing (see FIGS. 16 and 30) for correcting the brightness of the captured image data. Start. The internal configuration relating to the brightness correction by the image processing unit 7 will be described later with reference to FIG.

[0051] The header information processing unit 8 is a header of shooting information data generated by the image processing unit 7. Write information. The R, G, and B signals for which the header information has been written are stored as image data on the recording medium by the storage device 9. The storage device 9 stores captured image image data and a control program for the imaging apparatus 100.

The control unit 11 includes a CPU (Central Processing Unit) and the like, and includes an automatic focus driving circuit 17 that controls a motor 20 that adjusts the focal length and focus (focus) of the lens 1, a focal length adjustment circuit 16, Controls the strobe drive circuit 14 that drives the CCD drive circuit 10, analog processing circuit 4, temporary storage memory 6, image processing unit 7, display unit 13 and strobe 15.

The operation unit 12 includes buttons, cursor keys, and the like (not shown), and outputs operation information of the operation unit 12 by the user to the control unit 11. The display unit 13 displays photographed image data on the liquid crystal display and also displays settings and conditions related to photography.

FIG. 2 shows an internal configuration of the image processing unit 7. As shown in FIG. 2, the image processing unit 7 includes a preprocessing unit 70, a brightness correction unit 71, and an image quality correction unit 72.

[0055] The preprocessing unit 70 performs spectral sensitivity crosstalk correction, dark current noise suppression, and temporary white balance adjustment on the captured image data input from the temporary storage memory 6, and the brightness correction unit 71 and the image quality Output to the correction unit 72.

The brightness correction unit 71 includes an image analysis unit 73, an index calculation unit 74, a brightness correction amount calculation unit 75, and a brightness correction curve generation unit 76.

[0057] The image analysis unit 73 calculates a brightness analysis value of the captured image data. The brightness analysis value is a numerical value indicating the average brightness of the captured image data obtained by examining the distribution state of the pixel values of the captured image data, and is the most important subject in the captured image data (main subject image). The brightness of the body). As brightness analysis values, for example, the average brightness value of the entire captured image data (hereinafter referred to as “total average brightness”), the average brightness value in the skin color region of the captured image data (hereinafter referred to as “skin color average brightness”). ), And an exposure index (Indicator 6 to be described later) indicating the exposure shooting degree (under, over) due to the exposure setting at the time of shooting.

[0058] The index calculation unit 74 is a natural light index that represents the degree of outdoor shooting with natural light of the captured image data.

(Indicator 4 below), luminance ratio indicator (Indicator 5 below) indicating the dynamic range of the captured image data, and exposure exposure (under, over) due to exposure settings during shooting An exposure index (index 6 described later) is calculated. The index calculation process executed by the index calculation unit 74 will be described in detail later with reference to FIGS.

The brightness correction amount calculation unit 75 calculates the brightness correction amount of the photographed image data based on the brightness analysis value calculated by the image analysis unit 73 and the calculation result of the index calculation unit 74. The internal configuration of the brightness correction amount calculation unit 75 will be described in detail later with reference to FIG. 3 (Embodiment 1) and FIG. 29 (Embodiment 2).

The brightness correction curve generation unit 76 generates a brightness correction curve for converting captured image data based on the brightness correction amount calculated by the brightness correction amount calculation unit 75.

The image quality correction unit 72 corrects the brightness of the captured image data input from the preprocessing unit 70 according to the brightness correction curve generated by the brightness correction curve generation unit 76. The captured image data whose brightness has been corrected is output to the header information processing unit 8 and the display unit 13.

FIG. 3 shows an internal configuration of the brightness correction amount calculation unit 75 of the first embodiment. The brightness correction amount calculation unit 75 includes a first temporary correction amount setting unit 101, a first mixing coefficient setting unit 102, a second temporary correction amount calculation unit 103, a second mixing coefficient setting unit 104, and a brightness correction amount calculation unit. Consists of 105.

[0063] The first temporary correction amount setting unit 101 is based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation unit 74. Set the first temporary correction amount for the captured image data. When n brightness analysis values are input from the image analysis unit 73, the provisional correction amount setting unit 101 sets the first correction amount necessary to approximate each brightness analysis value to the reproduction target value. Set n + 4 temporary correction amounts.

[0064] Figure 4 shows a one-dimensional LUT that defines the relationship between the luminance ratio index and the first temporary correction amount key—auto [0].

It is a figure which shows (Look Up Table). FIG. 5 is a diagram showing a one-dimensional LUT that defines the relationship between the exposure index and the first temporary correction amount key_auto [1]. FIG. 6 is a diagram showing a one-dimensional LUT that defines the relationship between the flesh color average luminance and the first provisional correction amount key-auto [2]. FIG. 7 is a diagram showing a one-dimensional LUT that defines the relationship between the skin tone average brightness or the overall average brightness and the first temporary correction amount key-auto [3]. FIG. 8 is a diagram showing a one-dimensional LUT that defines the relationship between the luminance ratio index and the first temporary correction amount key-auto [4]. FIG. 9 is a diagram showing a one-dimensional LUT that defines the relationship between the exposure index and the first temporary correction amount key-auto [5].

Note that the flesh color average brightness or overall average brightness shown in Fig. 7 and the first temporary correction amount key-auto [3] In the one-dimensional LUT that defines the relationship, the overall average brightness is used when the exposure index is positive, and the skin color average brightness is used when the exposure index is 0 or negative.

In this case, use the overall average brightness.

In Embodiments 1 and 2, since the brightness analysis values calculated by the image analysis unit 73 are the skin color average brightness and the overall average brightness (ie, n = 2), the first provisional correction amount setting unit 101, figure 4

~ Using the one-dimensional LUT shown in Fig. 9, six temporary correction values (key-auto [0] to [5]) are set as the first temporary correction values.

[0066] The first mixing coefficient setting unit 102 is based on the natural light index calculated by the index calculation unit 74! /

The first mixing coefficient as a weighting coefficient to multiply n + 2 (key—auto [0] to [3]) out of the n + 4 first temporary correction amounts set by the temporary correction amount setting unit 101 Set n + two.

FIG. 10 is a diagram showing a one-dimensional LUT that defines the relationship between the natural light index and the first mixing coefficient wgt [0]. Figure 11 shows a one-dimensional definition of the relationship between the natural light index and the first mixing coefficient wgt [l].

It is a figure which shows LUT. FIG. 12 is a diagram showing a one-dimensional LUT that defines the relationship between the natural light index and the first mixing coefficient wgt [2]. FIG. 13 is a diagram showing a one-dimensional LUT that defines the relationship between the natural light index and the first mixing coefficient wgt [3].

[0068] In FIG. 11, when the natural light index is larger than 2, the exposure index is not calculated, or when the exposure index is 0 or negative, the relationship is indicated by a solid line, and when the exposure index is positive, The relationship is indicated by a broken line.

[0069] Note that the one-dimensional LUTs shown in FIGS. 4 to 9 and the one-dimensional LUTs shown in FIGS. 10 to 13 are not shown in FIG. RU

[0070] Second temporary correction amount calculation unit 103 is set by 11+ two first temporary correction amounts (1 ^ — _& 1 ^ ₀ [0] to [3]) and first mixing coefficient setting unit 102 Based on the n + 2 first mixing coefficients wgt [0] to wgt [3], the second temporary correction amount for the captured image data is set. Equation (2) shows the formula for calculating the second temporary correction amount.

[0071] Second temporary correction amount = key— auto [0] X wgt [0] + key_auto [l]

X wgt [l] + key_auto [2] X wgt [2] + key_auto [3]

X wgt [3] ー（2）

Hereafter, the second temporary correction amount in Equation (2) is expressed as key-auto [6]. The second mixing coefficient setting unit 104 discriminates the scene type of the captured image data from the values of the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation unit 74 using a discrimination map. The first temporary correction amount key— auto [4], key— auto [5], and the second temporary correction amount key— auto [6] are each multiplied by three second mixing coefficients wgt [ Set [4] to [6].

[0073] FIG. 14 (a) shows a discrimination map for discriminating the scene type (forward light, backlight, strobe, under, low accuracy region) of the captured image data based on the natural light index and the luminance ratio index. Fig. 14 (b) shows a discrimination map for discriminating the scene type (forward light, backlight, strobe, under, low accuracy region) of the captured image data based on the natural light index and the exposure index. In FIG. 14, the low accuracy region represents a region where the accuracy of being determined as follow light, backlight, strobe, or under is low (than a predetermined value).

FIG. 15 shows a coefficient calculation table representing the values of the second mixing coefficient wgt [4] to [6] for each scene type. This coefficient calculation table is stored in a memory (not shown) of the imaging apparatus 100. In FIG. 15, each number shown in the scene type item is a number assigned to each scene type in FIG. For example, in FIG. 15, “3” of the scene type item represents the low accuracy region (3) of FIG.

[0075] The brightness correction amount calculation unit 105 includes two first temporary correction amounts key—auto [4], key—auto [5], second temporary correction amount key—auto [6], and second Based on the three second mixing coefficients wgt [4] to [6] set by the mixing coefficient setting unit 104, the brightness correction amount is calculated as shown in Equation (3).

[0076] Brightness correction amount = key_auto [4] X wgt [4] + key_auto [5]

X wgt [5] + key_auto [6] X wgt [6] · '· (3)

Next, the operation in the first embodiment will be described.

A brightness correction process executed in the image processing unit 7 of the first embodiment will be described with reference to the flowchart of FIG.

First, the index calculation unit 74 performs index calculation processing for calculating a natural light index, a luminance ratio index, and an exposure index (step Tl). The index calculation process in step T1 will be described in detail later with reference to FIGS.

[0079] When the index calculation process ends, the image analysis unit 73 calculates η brightness analysis values of the captured image data (step Τ2). In Step Τ2, the two brightness analysis values (skin color level) Average brightness, overall average brightness). Next, from the one-dimensional LUTs in Figs. 4 to 9, the brightness analysis value calculated in step T2, and the values corresponding to the natural light index, luminance ratio index, and exposure index calculated in the index calculation process in step T1. N + 4 (six) first temporary correction amounts key—auto [0] to [5] are set (step T3).

[0080] Next, n + 2 (4) first mixing coefficients wgt [0] are extracted from the one-dimensional LUTs of FIGS. 10 to 13 by extracting values corresponding to the natural light index calculated in step T1. ~ [3] is set (step T4). Next, using the four first temporary correction values key—auto [0] to [3] and the four first mixing coefficients wgt [0] to [3], the second temporary correction is obtained from equation (2). The quantity key_ _a uto [6] is calculated (step T5).

[0081] Next, three second mixing coefficients wgt [4] to [6] corresponding to the scene type determined using the determination map (Fig. 14) are extracted from the coefficient calculation table (Fig. 15), Set (step T6). Next, two first temporary correction amounts key—auto [4], key—auto [5], second temporary correction amounts key—auto [6], and three second mixing coefficients wgt [4] to Using [6], the brightness correction amount is calculated by equation (3) (step T7), and the scene type determined based on the calculated brightness correction amount and each index calculated in step T1. Then, a brightness correction curve for converting the captured image data is generated (step Τ8).

FIG. 17 (a) is a diagram showing brightness correction curves for each brightness correction amount when the scene type is direct light. Fig. 17 (b) shows the brightness correction curve for each brightness correction amount when the scene type is backlight. FIG. 17 shows the brightness correction curve in the practical range (the 1, 6, 1, 2, 0 buttons) of the number of correction buttons operated by the operation unit 12. In step Τ8, a brightness correction curve corresponding to the brightness correction amount calculated in step Τ7 is selected from a plurality of brightness correction curves set in advance for each scene type.

When the brightness correction curve is generated, the brightness of the captured image data is corrected according to the brightness correction curve (step Τ9), and this brightness correction process is ended. The captured image data whose brightness has been corrected in this way is output to the designated output destination. Examples of the output destination include the display unit 13 and a printer connected to the imaging device 100.

<Indicator calculation process> Hereinafter, the index calculation process in step T1 will be described with reference to the flowchart of FIG.

First, the captured image data is divided into predetermined image areas, and an occupation ratio (first occupation ratio, second occupation ratio) indicating the ratio of each divided area to the entire captured image data is calculated. Occupancy calculation processing is performed (step Sl). Details of the occupation rate calculation process will be described later with reference to FIGS.

[0085] Next, a bias amount calculation process for calculating a bias amount indicating a bias in the gradation distribution of the photographed image data is performed (step S2). The bias amount calculation process in step S2 will be described in detail later with reference to FIG.

[0086] Next, based on the occupancy calculated in step S1 and the coefficient set in advance according to the shooting conditions (scene type, etc.), the natural light representing the outdoor shooting degree by the natural light of the shot image data. An index, a luminance ratio index that indicates the size of the dynamic range of the captured image data, and an exposure index that indicates the exposure shooting degree due to the exposure setting at the time of shooting are calculated (step S3), and this index calculation process ends. . The method for calculating the index in step S3 will be described in detail later.

Next, the first occupancy rate calculation process will be described in detail with reference to the flowchart of FIG.

[0088] First, the RGB value of the photographed image data is converted into the HSV color system (step S10). Figure 20 shows an example of a conversion program (HSV conversion program) that obtains hue values, saturation values, and brightness values by converting from RGB to the HSV color system in program code (c language). is there. In the HSV conversion program shown in Fig. 20, the digital image data values that are input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, The degree value is OutS, the lightness value is OutV, and the unit is defined as 0 to 255.

Next, the photographed image data is divided into regions composed of a predetermined combination of brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step Sl l). . Hereinafter, the area division of the captured image data will be described in detail.

[0090] Lightness (V) i, Lightness value power ^) to 25 (vl), 26— 50 (ν2), 51 to 84 (ν3), 85 to 169 (ν4) , 170 to 199 (v5), 200 to 224 (v6), and 225 to 255 (v7). Hue (H) is a flesh color range (HI and H2) with a hue value of 0 to 39, 330 to 359, a green hue range (H3) with a hue value of 0 to 160, and a blue hue range with a hue value of 161 to 250. It is divided into four areas: (H4) and red hue area (H5). Note that the red hue region (H5) is not used in the calculations below, based on the knowledge that there is little contribution to scene type discrimination (see Figure 14). The flesh-color hue area is further divided into a flesh-color area (HI) and other areas (H2). Hereinafter, out of the flesh-colored hue area (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following formula (4) is defined as the flesh-colored area (HI), and the formula (4) is not satisfied! /, Let the region be (H2).

[0091] 10 <Saturation) <175,

Hue '(H) = Hue (H) + 60

(When 0 ≤ Hue (H) <300),

Hue '(H) = Hue (H) — 300

(When 300 ≤ Hue (H) <360),

Luminance (Y) = InRX0.30 + InGXO.59 + ΙηΒΧΟ.11 ··· (Α)

Hue '(Η) ΖLuminance (Υ) く 3.0Χ (Saturation (S) Z255) +0.7… （

Therefore, the number of divided areas of the captured image data is 4X7 = 28. It is also possible to use V and brightness (V) in equation (4).

[0092] When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S12). The occupation rate calculation process ends. Assuming that Rij is the first occupancy calculated in the divided area, which is the combined power of the lightness area vi and the hue area Hj, the first occupancy ratio in each divided area is shown in Table 1.

[0093] [Table 1] [First occupancy]

[0094] Next, a method for calculating the index 1 and the index 2 will be described.

[0095] Table 2 shows, for each divided area, the first coefficient necessary to calculate the index 1 that quantitatively indicates the accuracy of flash photography, that is, the brightness state of the face area during flash photography. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupancy Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the photographing conditions.

[0096] [Table 2]

[First factor]

FIG. 21 shows the brightness (V) —hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (rl) distributed in the high-luminance skin color hue region in Fig. 21, and other hues are used. Blue hue region (r2) force A negative (-) coefficient is used for the first occupancy calculated. Figure 23 shows the first coefficient in the flesh tone area (HI) and the first coefficient in the other areas (green hue area (H3)) as curves (coefficient curves) that change continuously over the entire brightness. It is shown. According to Table 2 and Fig. 23, the sign of the first coefficient in the skin color region (HI) is positive (in the region of high brightness (V = 170 to 224)) +), And the sign of the first coefficient in other areas (for example, the green hue area (H3)) is negative (one), indicating that the signs of the two are different.

[0098] If the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Equation (5).

[0099] [Equation 2]

Sum of H k regions =:: Rik X Cik (5)

i = 1

[0100] Therefore, the sum of the H1 to H4 regions is expressed by the following equations (5-1) to (5-4).

HI area sum = R11X (—44.0) + R21X (-16.0) + (omitted) ...

+ R71X (-11.3) --- (5-1);

Sum of H2 regions = R12X0.0 + R22X8.6 + (omitted) ...

+ R72X (-11.1) --- (5-2);

Sum of H3 regions = R13XO.0 + R23X (-6.3) + (omitted) ...

+ R73X (-10.0) --- (5-3);

Sum of H4 region = R14X0.0 + R24X (-1.8) + (omitted) ...

+ R74X (14.6) --- (5-4).

Index 1 is defined as equation (6) using the sum of the H1 to H4 regions shown in equations (5-1) to (5-4).

[0101] Indicator 1 = H1 region sum + H2 region sum + H3 region sum + H4 region sum +4.424 ー (6)

Table 3 shows the second coefficient required for each divided area to calculate the index 2 that quantitatively shows the accuracy of backlighting, that is, the brightness state of the face area during backlighting. The coefficient of each divided area shown in Table 3 is a weighting coefficient by which the first occupancy ratio Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

[0102] [Table 3] [Second factor]

FIG. 22 shows the lightness (V) —hue (H) plane. According to Table 3, a negative (one) coefficient is used for the occupancy calculated from the area (r4) distributed in the intermediate lightness of the flesh-colored hue area in Figure 22, and the low lightness (shadow) of the flesh-colored hue area. A positive (+) coefficient is used for the occupation ratio calculated from the region (r3). FIG. 24 shows the second coefficient in the skin color region (HI) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and Fig. 24, the sign of the second coefficient in the intermediate lightness region of the flesh hue region with the lightness value of 85 to 169 (v4) is negative (-), and the lightness value is 26 to 84 (v2, The sign of the second coefficient in the low lightness (shadow) region of v3) is positive (+), which indicates that the sign of the coefficient in both regions is different.

[0104] If the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 2 is defined as in Equation (7).

[0105] [Equation 3]

Sum of H k regions => Rik X Dik (7)

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (7-1) to (7-4).

HI area sum = R11X (--27.0) + R21X4.5 + (omitted) ...

+ R71X (-24.0) to (7-1);

Sum of H2 regions = R12X0.0 + R22X4.7 + (omitted) ...

+ R72X (-8.5) --- (7-2);

Sum of H3 regions = R13XO.0 + R23XO.0 + (omitted) ... + R73 X 0. 0--(7- 3);

H4 region sum = R14 X 0. 0 + R24 X (-5.1) + (omitted) ...

+ R74 X 7. 2--(7-4).

Index 2 is defined as equation (8) using the sum of the H1 to H4 regions shown in equations (7-1) to (7-4).

[0107] Indicator 2 = sum of H1 regions + sum of H2 regions + sum of H3 regions + sum of H4 regions + 1. 554…

Since index 1 and index 2 are calculated based on the brightness and hue distribution amount of the captured image data, they are effective in determining the scene type when the captured image data is a color image.

Next, the second occupancy rate calculation process executed to calculate the index 3 will be described in detail with reference to the flowchart in FIG.

First, the RGB value of the photographed image data is converted into the HSV color system (step S 20). Next, the captured image data is divided into areas where the combined power of the distance from the outer edge of the captured image screen and the brightness is determined, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided area. (Step S21). Hereinafter, the area division of the captured image data will be described in detail.

[0110] FIGS. 26 (a) to (d) show four regions nl to n4 divided according to the distance from the outer edge of the screen of the captured image data. The area nl shown in FIG. 26 (a) is the outer frame, the area n2 shown in FIG. 26 (b) is the area inside the outer frame, and the area n3 shown in FIG. A further inner area, an area n4 shown in FIG. 26 (d) is an area at the center of the captured image screen. In addition, the lightness is divided into seven regions from vl to v7 as described above. Therefore, when the captured image data is divided into regions that are a combination of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

[0111] When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S22). The occupation rate calculation process ends. If the second occupancy calculated in the divided area, which is the combined power of the brightness area vi and the screen area nj, is Qij, the second occupancy in each divided area is The share is shown in Table 4.

[0112] [Table 4]

[Second occupancy]

[0113] Next, a method of calculating the index 3 will be described.

[0114] Table 5 shows the third coefficient necessary for calculating index 3 for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy Qij of each divided area shown in Table 4 is multiplied, and is set in advance according to the photographing conditions.

[0115] [Table 5]

[Third coefficient]

[0116] FIG. 27 shows the third coefficient in the screen areas nl to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

[0117] If the third coefficient in the lightness area vi and screen area nj is Eij, the sum of the nk area (screen area nk) for calculating index 3 is defined as in equation (9).

[0118] [Equation 4] nk region sum = J Qik x Eik

Accordingly, the sum of the nl to n4 regions is expressed by the following formulas (91) to (94).

nl region sum = Q11X40. 1 + Q21X37.0 + (omitted) ...

+ Q71X22.0 --- (9-1);

Sum of n2 regions = Q12X (-14.8) + Q22X (— 10.5) + (omitted) ...

+ Q72X0.0 --- (9-2);

n3 area sum = Q13X24.6 + Q23X12.1 + (omitted) ...

+ Q73X10.1 --- (9-3);

n4 region sum = Q14X1.5 + Q24X (-32.9) + (omitted) ...

+ Q74X (-52.2) --- (9-4).

Index 3 is defined as equation (10) using the sum of nl to n4 regions shown in equations (91) to (94).

Index 3 = Sum of nl regions + Sum of n2 regions + Sum of n3 regions + Sum of n4 regions 12.6201 (10)

Index 3 is calculated based on the compositional characteristics (distance from the outer edge of the screen of the captured image data) based on the brightness distribution position of the captured image data, so the scene type of the monochrome image as well as the color image is selected. It is also effective for discrimination.

Next, the bias amount calculation process (step S2 in FIG. 18) will be described with reference to the flowchart in FIG.

[0121] First, the luminance Y (brightness) of each pixel is calculated from the RGB (Red, Green, Blue) value of the captured image data using Equation (A), and the standard deviation (xl) of the luminance is calculated. (Step S23). The standard deviation (xl) of luminance is expressed as shown in Equation (11).

[0122] [Equation 5]

In Expression (11), the sum is calculated for each pixel of the captured image data. In equation (11), pixel The brightness value is the brightness of each pixel of the captured image data, and the overall average brightness is the average value of the brightness of the entire captured image data. The total number of pixels is the number of pixels of the entire captured image data.

Next, as shown in Expression (12), the luminance difference value (x2) is calculated (step S24).

[0125] Luminance difference value (x2) = (Maximum luminance value Overall average luminance) Z255… (12)

In equation (12), the maximum luminance value is the maximum luminance value of the captured image data.

[0126] Next, the average luminance value (x3) of the flesh color region at the center of the screen of the photographed image data is calculated (step S25), and further, the average luminance value (x4) at the center of the screen is calculated (step S25). S26). Here, the center of the screen is, for example, an area composed of an area n3 and an area n4 in FIG.

[0127] Next, the flesh color luminance distribution value (x5) is calculated (step S27), and this deviation amount calculation processing ends. When the maximum brightness value of the skin color area of the captured image data is Yskin—max, the minimum brightness value of the skin color area is Yskin—min, and the average brightness value (skin color average brightness) of the skin color area is Yskin—ave, the skin color brightness distribution value ( x5) is expressed as in Equation (13).

[0128] x5 = (Yskin― max― Yskin― min) / 2― Yskin― ave

•••(13)

Let x6 be the average luminance value of the skin color area in the center of the screen of the captured image data. Here, the center of the screen is, for example, a region composed of region n2, region n3, and region n4 in FIG. At this time, index 4 is defined as in equation (14) using index indexes 3 and x6, and index 5 is defined as in equation (15) using index 2, index 3, and x6.

[0129] Indicator 4 = 0. 46 X indicator 1 + 0. 61 X indicator 3 + 0. 01 χ χ6 0. 79

•••(14)

Indicator 5 = 0. 58 Χ Indicator 2 + 0.18 X Indicator 3 + (— 0. 03) Χ χ6 + 3. 34---(15) Here, each indicator in Equation (14) and Equation (15) The weighting coefficient multiplied by is preset according to the shooting conditions.

[0130] The index 6 is obtained by multiplying the deviation amounts (xl) to (x5) calculated in the deviation amount calculation processing by a fourth coefficient set in advance according to the imaging conditions. Table 6 shows the fourth coefficient, which is a weighting coefficient by which each deviation is multiplied. [0131] [Table 6]

[Fourth coefficient]

[0132] The index 6 is expressed as in Expression (16).

[0133] Indicator 6 = xlX0.02 + x2Xl. 13 + x3XO.06 + x4X (-0.01)

+ x5XO.03-6.49 --- (16)

This index 6 has luminance histogram distribution information that is based only on the compositional characteristics of the captured image data screen, and is particularly effective in distinguishing between a flash photography scene and an under photography scene.

[0134] The index 4 shown in Expression (14) corresponds to a natural light index, the index 5 shown in Expression (15) corresponds to a luminance ratio index, and the index 6 shown in Expression (16) corresponds to an exposure index.

As described above, according to the imaging apparatus 100 of the first embodiment, the brightness of the main subject (for example, a human face area) in a backlight scene or strobe shooting scene is an output device that outputs shot image data. It is possible to obtain a brightness correction amount that can be reproduced with an optimum brightness (for example, a printer).

[0136] Further, by generating a brightness correction curve based on the brightness correction amount calculated as described above and the determined scene type, a captured image such as a backlight scene or a flash shooting scene is generated. When outputting a scene with a large data dynamic range, the degree to which the output device's dynamic range power is off will be small, so the brightness of the main subject of the captured image data will be properly finished, It is possible to suppress the dark part from being crushed, high-saturated colors being saturated and crushed, and the hue from being changed.

[Embodiment 2]

Next, Embodiment 2 of the present invention will be described.

[0137] Since the functions of the constituent elements common to the first and second embodiments have been described, Hereinafter, functions unique to the second embodiment will be described.

Hereinafter, the internal configuration of the brightness correction amount calculation unit 75 of Embodiment 2 will be described with reference to FIG. Note that, among the components of the brightness correction amount calculation unit 75 according to the second embodiment, the same components as those in the first embodiment shown in FIG. 3 are denoted by the same reference numerals, and the functional description thereof is omitted. As shown in FIG. 29, the brightness correction amount calculation unit 75 of Embodiment 2 includes a first temporary correction amount setting unit 101, a first mixing coefficient setting unit 102, a second temporary correction amount calculating unit 103, and a second mixing coefficient. It comprises a setting unit 104, a brightness correction amount calculation unit 105, a suppression coefficient calculation unit 106, and a multiplication unit 107.

The suppression coefficient calculation unit 106 calculates the number of high-saturation pixels having a predetermined hue that is higher than a predetermined threshold and has a predetermined hue that is set based on the natural light index. Based on the ratio of the number of high-saturation pixels to the total number of pixels in the image data, a suppression coefficient Re (0 <0 <Re≤l. 0) for suppressing the brightness correction amount of the captured image data is calculated. (Set). The calculation method of the suppression coefficient Re will be described in detail later with reference to FIG.

[0140] The multiplication unit 107 calculates (sets) the brightness correction amount (Equation (3)) calculated by the brightness correction amount calculation unit 105 by the suppression coefficient calculation unit 106 as shown in Equation (17). By multiplying the suppression coefficient Re thus obtained, a new brightness correction amount is calculated, and this new brightness correction amount is output to the brightness correction force curve generator 76.

[0141] New brightness correction amount = Re X brightness correction amount (17)

The brightness correction curve generation unit 76 according to the second embodiment converts captured image data based on the scene type determined by the determination map (see FIG. 14) and the new brightness correction amount input from the multiplication unit 107. To generate a brightness correction curve.

Next, the operation in the second embodiment will be described.

A brightness correction process executed in the image processing unit 7 of the second embodiment will be described with reference to the flowchart of FIG.

First, the index calculation unit 74 performs an index calculation process (see FIGS. 18 to 28) for calculating a natural light index, a luminance ratio index, and an exposure index (step T20). Next, the image analysis unit 73 calculates η brightness analysis values of the captured image data (step Τ21). Step In step T21, two brightness analysis values (skin color average brightness, overall average brightness) are calculated.

[0145] Next, from the one-dimensional LUT shown in Figs. 4 to 9, the brightness analysis value calculated in step T21 and the natural light index, luminance ratio index, and exposure index calculated in the index calculation process in step T20 are supported. By extracting the value, n + 4 (six) first temporary correction amounts key—auto [0] to [5] are set (step T22).

[0146] Next, n + 2 (4) first mixing coefficients wgt [0] are extracted from the one-dimensional LUTs in Figs. 10 to 13 by extracting the values corresponding to the natural light index calculated in Step T20. ~ [3] is set (step T23). Next, using the four first temporary correction values key—auto [0] to [3] and the three first mixing coefficients wgt [0] to [3], the second temporary correction is obtained from Equation (2). the amount key_ _a uto [6] is calculated (step T24).

[0147] Next, three second mixing coefficients wgt [4] to [6] corresponding to the scene type determined using the determination map (Figure 14) are extracted from the coefficient calculation table (Figure 15). Set (step T25). Then, two first temporary correction amounts key—auto [4], key—auto [5], second temporary correction amounts key—auto [6], and three second mixing coefficients wgt [4] to Using [6], the brightness correction amount is calculated by equation (3) (step T26).

[0148] Next, it is determined whether or not the luminance ratio index calculated in step Τ20 is larger than the threshold value Thl (step T27). Here, the threshold Thl = l.5. If it is determined in step T27 that the luminance ratio index> Thl (step T27; YES), the hue condition of the pixel to be counted later as a high saturation pixel is determined according to the natural light index calculated in step T20. Set (step T28).

FIG. 31 shows the hue condition (hue range) set according to the value of the natural light index in step T28. As shown in Fig. 31, the hue condition is -6. 0 <natural light index ≤ 1.5, hue H = 161-250, and 1.5 <natural light index <2.0, hue H = 0 to 39, 161 to 250, 330 to 359, and 2.0 ≦ natural light index 6.0, hue H = 0 to 39, 330 to 359.

[0150] When the hue condition is set, the number of pixels having the hue condition set in step T28 and having a saturation (S) greater than the threshold Th2 among the total number of pixels of the captured image data is calculated. Saturation)> The saturation level depends on the ratio (%) of the high saturation pixels that satisfy Th2 to the total number of pixels. Is set (step T29). Here, Th2 = 130 (HSV color system, maximum value 255). Table 7 shows the ratio (%) of high-saturation pixels satisfying saturation)> Th2 to the total number of pixels and the suppression coefficient for each saturation level (saturation 1 to 4).

[Table 7]

[0152] When the saturation level is set, according to Table 7, the suppression coefficient Re corresponding to the set saturation level is set (step T30). Note that the processing of steps Τ27 to Τ30 is executed by the suppression coefficient calculation unit 106. Next, as shown in equation (17), a new brightness correction amount is calculated by multiplying the brightness correction amount calculated in step Τ26 by the suppression coefficient Re set in step Τ30 (step (T31), a brightness correction curve for converting the captured image data is generated from the new brightness correction amount and the scene type determined based on each index calculated in step Τ20 (step S31). (Τ32).

[0153] If it is determined in step Τ27 that the luminance ratio index ≤ Thl (step Τ27;

NO), a brightness correction curve for converting captured image data from the brightness correction amount calculated in step T26 and the scene type determined based on each index calculated in step T20! Is generated (step T32).

[0154] When the brightness correction curve is generated, the brightness of the captured image data is corrected according to the brightness correction curve (step T33), and the brightness correction processing is terminated. The captured image data whose brightness has been corrected in this way is output to the designated output destination. Examples of the output destination include a display unit 13 and a printer connected to the imaging apparatus 100.

[0155] As described above, according to the imaging apparatus 100 of Embodiment 2, the brightness of captured image data is determined using the suppression coefficient calculated based on the ratio of the number of high-saturation pixels to the total number of pixels of captured image data. By suppressing the correction amount, it is possible to prevent saturation of high saturation pixels in captured image data having many high saturation pixels. In particular, since the hue of a pixel to be counted as a high saturation pixel is determined based on the value of the natural light index, saturation of the high saturation pixel can be prevented with high accuracy. [0156] Note that the description in each of the above embodiments can be changed as appropriate without departing from the spirit of the present invention.

For example, the case where the brightness correction processing shown in each of the above embodiments is performed by an imaging apparatus such as a digital camera has been described. However, an image processing apparatus (personal computer) installed in a minilab (small-scale development laboratory) is shown. However, it is possible to install the function of the image processing unit 7 in FIG.

FIG. 32 shows the main configuration of an image processing apparatus 200 to which the present invention is applied. As shown in FIG. 32, the image processing apparatus 200 includes an image input unit 201, an image processing unit 202, and an image output unit 203.

The image input unit 201 acquires photographed image data obtained by photographing a subject and outputs it to the image processing unit 202. The image output unit 203 has a display such as an LCD (Liquid Crystal Display) and a CRT (Cathode Ray Tube), and displays captured image data processed by the image processing unit 202. Further, the image output unit 203 may include a printing unit that prints out the captured image data processed by the image processing unit 202.

As shown in FIG. 32, the image processing unit 202 includes an image analysis unit 204, an index calculation unit 205, a brightness correction amount calculation unit 206, a brightness correction curve generation unit 207, and an image quality correction unit 208. The functions of the respective units are the same as those of the image analysis unit 73, the index calculation unit 74, the brightness correction amount calculation unit 75, the brightness correction curve generation unit 76, and the image quality correction unit 72 of FIG.

[0161] However, in the case of the image processing apparatus 200 installed in the minilab, the brightness correction curve has many more types than just the four types shown in Fig. 17 (1-6, --4, -2, 0 buttons). It is possible to generate a brightness correction curve.

Claims

The scope of the claims

[1] An imaging device that captures captured image data by capturing an image of a subject and performs processing for optimizing the brightness of a main object in the captured image data.

Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means;

An imaging apparatus comprising:

[2] Take a picture of the subject, obtain the shot image data, and then capture the main subject in the shot image data An imaging device that performs processing to optimize body brightness,

According to the correction curve generated by the correction curve generating means, the captured image data is displayed. Correction means for correcting the brightness of the data;

An imaging apparatus comprising:

[3] The index calculation means includes:

By multiplying the calculated first occupancy rate and Z or second occupancy rate and the deviation amount by a coefficient set in advance according to the shooting conditions, the natural light index, the luminance ratio index, The imaging apparatus according to claim 1, wherein the exposure index is calculated.

[4] An image processing device that performs processing for optimizing the brightness of a main subject in captured image data.

A second temporary correction amount for setting a second temporary correction amount for the photographed image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts. Setting means; Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weighting factor. 2nd mixing coefficient setting means for setting the mixing coefficient;

An image processing apparatus comprising:

[5] An image processing apparatus that performs processing for optimizing the brightness of a main subject in captured image data.

Multiplying the second temporary correction amount by at least one of the plurality of first temporary correction amounts based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculating means. A second mixing coefficient setting means for setting a second mixing coefficient as a weighting coefficient; a second mixing coefficient set by the second mixing coefficient; the at least one first temporary correction amount; and the second temporary coefficient. Correction amount calculating means for calculating a brightness correction amount of the captured image data based on a correction amount;

An image processing apparatus comprising:

[6] The index calculation means includes

By multiplying the calculated first occupancy and Z or second occupancy by the deviation amount by a coefficient set in advance according to the shooting conditions, the natural light index, the luminance ratio index, 6. The image processing apparatus according to claim 4, wherein the exposure index is calculated.

[7] An image processing method that optimizes the brightness of the main subject in the captured image data. There,

A correction curve generating step for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating step, the natural light index, and the brightness ratio index;

A correction step of correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation step;

An image processing method comprising:

An image processing method for performing processing for optimizing the brightness of a main subject in captured image data,

A natural light index indicating the degree of outdoor photographing with natural light of the photographed image data, and the photographing An index calculation step for calculating a brightness ratio index that represents the size of the dynamic range of the image data, and an exposure index that represents the exposure shooting degree due to the exposure setting at the time of shooting;

A correction curve generating step for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication step, the natural light index, and the luminance ratio index;

An image processing method comprising: [9] In the index calculation step,

By multiplying the calculated first occupancy and Z or second occupancy by the deviation amount by a coefficient set in advance according to the shooting conditions, the natural light index, the luminance ratio index, The image processing method according to claim 7, wherein the exposure index is calculated.

[10] On the computer,

A correction function for correcting the brightness of the captured image data according to the correction curve generated by the correction curve generation function;

An image processing program for realizing

[11] On the computer,

Correction for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A quantity calculation function, Based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index, and the saturation of the captured image data is higher than a predetermined threshold value, V / A suppression coefficient setting function for setting a suppression coefficient for suppressing the brightness correction amount of the data, and a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation function by the suppression coefficient A multiplication function for calculating

An image processing program for realizing

When realizing the index calculation function,

By multiplying the calculated first occupancy and Z or second occupancy by the deviation amount by a coefficient set in advance according to the shooting conditions, the natural light index, the luminance ratio index, The image processing program according to claim 10, wherein the exposure index is calculated.