JP2014121018A - Imaging apparatus and control method for the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus and a control method for the same, capable of performing a dynamic range control allowing for color component level.SOLUTION: A target dynamic range is determined by the degree of halation over an entire image. If the image includes an area with a deviation in color component level, the target dynamic range is corrected in proportion to the degree of halation within the area, and a dynamic range control is performed on the basis of the target dynamic range corrected. To determine the degree of halation in an area with a deviation in color component level, the degree of halation allowing for color component level is determined, in addition to the luminance of pixels.

Description

  The present invention relates to an imaging apparatus and a control method thereof, and more particularly to an imaging apparatus that performs dynamic range control of an image and a control method thereof.

  A conventional digital still camera, digital video camera, or the like uses a luminance conversion characteristic (gamma characteristic) for converting a luminance value of an input image into a luminance value of an output image. If the input image contains many pixels with saturated brightness values (having an upper limit brightness value), the brightness value is saturated by first reducing the exposure amount by controlling the aperture and shutter speed. Reduce pixels. Then, the gain and the luminance conversion characteristic applied to the captured image are corrected to compensate for the reduced exposure amount.

  By combining the exposure control and the correction after shooting, it is possible to suppress overexposure due to saturation of the luminance value of the pixel in the high luminance region and to express gradation. Such a series of controls is called gradation control or dynamic range (D range) control. Although the apparent dynamic range of the image sensor can be expanded by D range control, the influence of the amount of pixels with saturated luminance values (saturated pixels) on image quality may vary depending on the subject. . For example, the impression of image quality deterioration due to the presence of saturated pixels in the face area of a person in the image is different from the impression of image quality deterioration given by saturated pixels existing in other subjects.

  For this reason, there has been proposed an image processing apparatus that performs D range control by changing a threshold value for detecting a whiteout region composed of saturated pixels between a human face region and other subjects (Patent Document 1).

JP 2010-193099 A

  However, especially in areas where the color component level (RGB value) of the pixel is biased, such as a human face area, a high-level color in a high-luminance part where the luminance value is close to the upper limit even if the luminance is not saturated. The component may approach the upper limit value and result in an unnatural gradation. In the prior art described in Patent Literature 1 and the like, the overexposed area is detected based only on the level of the luminance component of the pixel, and therefore, image quality deterioration due to the color component level approaching the upper limit value is detected. It was difficult to correct.

  The present invention has been made in view of such problems, and an object thereof is to provide an imaging apparatus capable of gradation control in consideration of the level of color components and a control method therefor.

  The above-described object is an imaging apparatus that performs dynamic range control of an image by performing exposure control and gain control according to the whiteness of the image, and sets a target dynamic range based on the whiteness of the entire image. Determining means for determining; detecting means for detecting an area having a biased color component level from the image; and correcting means for correcting the target dynamic range in accordance with a whiteout degree of the area having a biased color component level And control means for performing dynamic range control based on the corrected target dynamic range, wherein the correction means determines the whiteness of the area biased in the level of the color component, the brightness level of the pixels included in the area, and In addition to calculating based on the color component level, if the overexposure degree of the area where the color component level is biased is larger than the threshold, the target dynamic range is expanded. Correction was carried out for, when overexposed degree in the area of the bias in the level of a color component is smaller than the threshold value is achieved by an imaging device and performs a correction to reduce the target dynamic range.

  With such a configuration, according to the present invention, it is possible to provide an imaging apparatus capable of gradation control in consideration of the level of color components and a control method therefor.

1 is a block diagram illustrating a functional configuration example of a digital video camera as an example of an imaging apparatus according to an embodiment of the present invention. The figure which shows the typical image for demonstrating the dynamic range control in the digital video camera which concerns on embodiment of this invention (A) is a figure which shows the example of a relationship between the number of overexposure pixels in the embodiment of this invention, and the width of D range, (b) is a figure which shows the example of a relationship between exposure change amount and control amount, (c) is The figure which shows the example of the gamma characteristic change accompanying the change of D range The flowchart which shows the detail of D range correction process in embodiment of this invention (A) is a figure for demonstrating the selection process of the largest color component in embodiment of this invention, (b) is a figure which shows the example of calculation of the overexposure pixel number in embodiment of this invention. The figure for demonstrating the detection method of the overexposure pixel in embodiment of this invention

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.
In the following, a digital video camera will be described as an example of an imaging apparatus to which the present invention can be applied. However, the present invention is applicable to any imaging apparatus including a digital still camera and an apparatus incorporating the imaging apparatus. . Non-limiting examples include mobile phones, personal computers, game machines, media players, navigation systems, automobiles, home appliances, and the like.

  FIG. 1 is a block diagram showing a functional configuration example of a digital video camera as an example of an imaging apparatus according to an embodiment of the present invention. The arrows in the figure represent the flow of signals between functional blocks. The optical system 101 has a diaphragm (including an ND filter) function, an autofocus function, a zoom function, and the like, and forms an optical image of a subject on the imaging surface of the image sensor 102. The imaging element 102 is a photoelectric conversion element such as a CCD image sensor or a CMOS image sensor. The image sensor 102 converts the subject image formed on the imaging surface into an electrical signal in units of pixels and outputs an analog image signal.

  The A / D conversion unit 103 converts the analog image signal into a digital image signal having a digital value in accordance with the determined gain control value. The signal processing unit 104 applies white balance adjustment, color interpolation (demosaic) processing, and the like to the digital image signal. The face detection unit 105 applies a known face detection method to the image data generated by the signal processing unit 104, and detects a region (face region) having a human face feature included in the captured image.

  The image feature acquisition unit 106 extracts features of an arbitrary region in the image from the image data output from the signal processing unit 104. The gamma circuit 107 applies a gamma curve having predetermined input / output characteristics to the image data output from the signal processing unit 104, and converts the luminance level of the image data. The system control unit 100 controls the above-described functional blocks to realize operations described later. Further, other functional blocks are controlled as necessary to realize the operation of the entire digital video camera.

  The system control unit 100 further includes a plurality of control blocks. The exposure control unit 108 controls the optical system 101, the image sensor 102, and the A / D conversion unit 103. A dynamic range control unit (D range control unit) 109 determines a target D range according to an image feature such as a whiteness degree of the entire image, a whiteness degree of a face area detected by the face detection unit 105, and the like. The D range control unit 109 controls the exposure control unit 108 based on the determined target D range to change the exposure amount, and controls the luminance conversion characteristic (gamma characteristic) so as to correct the exposure change amount. The detection parameter determination unit 110 determines overexposed pixel detection parameters according to the control of the D range control unit 109. The whiteout pixel detection unit 111 detects a whiteout pixel in the subject area output by the image feature acquisition unit 106 using the whiteout pixel detection parameter determined by the detection parameter determination unit 110.

  The system control unit 100 can be realized, for example, by a combination of a programmable processor such as a CPU or MPU, a ROM storing a control program, and a RAM for developing the control program or using it as a work area. Accordingly, one or more functional blocks included in the system control unit 100 can be realized by software. Of course, one or more of the functional blocks may be realized by dedicated hardware.

  The operation during shooting will be briefly described. An optical image of the subject whose brightness is controlled by the aperture is formed on the imaging surface of the imaging element 102 by the optical system 101. The image sensor 102 is exposed by the optical image for the time determined by the exposure control unit 108, and an analog image signal is acquired. The exposure time is adjusted using a mechanical shutter or an electronic shutter.

  The A / D conversion unit 103 converts the analog image signal into a digital image signal. At this time, the signal is also amplified by the gain value determined by the exposure control unit 108. For example, in the case of a dark subject, a control signal is output so that the level of the output signal is increased from the exposure control unit 108 to the A / D conversion unit 103 and output to the signal processing unit 104. The signal processing unit 104 performs image processing such as black level correction, white balance adjustment, and edge correction on the digital image signal. The signal processing unit 104 outputs the digital image data during or after the image processing to the face detection unit 105 and the image feature acquisition unit 106, and the gamma of the image data after the image processing as an image signal for recording and display. Output to the circuit 107.

  The face detection unit 105 detects a region (face region) having a human face feature from the image, and outputs the position and size of the face region, luminance information, and the like to the image feature acquisition unit 106. The image feature acquisition unit 106 uses the image data output from the signal processing unit 104 and the face detection result from the face detection unit 105 to calculate the average brightness value, brightness frequency distribution, and RGB pixel values of the face area and the entire image. Image feature information such as frequency distribution is acquired.

  The face detection in the face detection unit 105 can be performed using a known technique. Image features suitable for face detection include feature vectors projected on eigenspace, gradient-based features, HAAR-LIKE features, SALIENCY MAP extracted by Sobel filters and average color features, and the like. A face detection method based on the movement of a person based on the CHLAC feature and the position of the face image may be used. A known speed-up method such as a boosting algorithm by ADA-BOOST, discrimination by SVM, reduction of the number of dimensions by concatenation of cascade type weak classifiers or projection to an eigenspace may be applied. In the present embodiment, for the sake of convenience, the case where the subject to be detected using the image feature is a human face will be described. However, since the above-described problem occurs in a subject region with biased RGB pixel values, An arbitrary subject that meets various conditions can be detected. For example, a pet, an infant, a ball, etc. can be illustrated.

  When the face area of a person is included in the image, gradation control (D range control) of the image is performed using the exposure control unit 108, the D range control unit 109, the detection parameter determination unit 110, and the overexposure pixel detection unit 111. I do. Here, a specific example of the D range control will be described for a case where an image as shown in FIG.

  First, the image feature acquisition unit 106 divides the entire image into a plurality of rectangular areas as illustrated in FIG. 2B, for example, and acquires a representative value of luminance for pixels included in each rectangular area. There is no particular limitation on what value is used as the representative value, but it may be, for example, an average luminance level or a maximum luminance level. The D range control unit 109 determines the degree of saturation (overextraction) in the image, such as the number of overexposed pixels and the ratio of overexposed areas, from the number of rectangular areas (excessive overexposed areas) whose luminance representative value is equal to or greater than the threshold Th. A value (overexposure degree) is calculated. For example, it is possible to calculate the number of overexposed pixels as all overexposed pixels as pixels that are included in a rectangular area whose luminance representative value is equal to or greater than the threshold value Th. In the example of FIG. 2B, a whiteout region exists at the top of the image.

  The D range control unit 109 sets the target D based on the correspondence between the whiteness and the range of the D range designed in advance so as to keep the whiteness within a predetermined value according to the calculated whiteness. The range Dnext is determined. FIG. 3A shows an example of the correspondence relationship between the whiteout degree and the width of the D range when the number of overexposed pixels is used as the whiteout degree. The larger the number of overexposed pixels, the wider the D range. However, in the region where there are many overexposed pixels, the increase rate of the D range with respect to the increase in overexposed pixels is low.

  The D range control unit 109 outputs the exposure change amount E corresponding to the determined value of the target D range Dnext to the exposure control unit 108 and changes the gamma characteristic of the gamma circuit 107 according to the exposure change amount E. In the exposure control unit 108, for example, based on the exposure control diagram as shown in FIG. 3B, the exposure control amount Enext after the change and the exposure control that realizes it from the current exposure control amount Enow and the exposure change amount E. Determine the elements. Here, the shutter speed, the ND filter, and the aperture value are associated as exposure control elements according to the range of the exposure control amount. The exposure control unit 108 controls the optical system 101 and the image sensor 102 from the changed exposure control amount Enext and the associated exposure control element to realize the changed exposure control amount Enext.

  Changing the exposure control amount changes the brightness of the entire captured image (in this case, it becomes dark). Therefore, the exposure control unit 108 applies a gain (−E) corresponding to the exposure change amount E to the A / D conversion unit 103 so that the final image brightness does not change before the exposure control amount is changed. Set to. This prevents the brightness of the image from changing before and after the change of the exposure control amount at the time of output from the A / D conversion unit 103. In this way, by changing the exposure control amount, the saturation of the pixels of the image sensor 102 is suppressed, and a gain that compensates for the exposure control amount is applied, so that the brightness of the entire image does not change. Range control can be realized.

  Further, in the gamma circuit 107, for example, as shown in FIG. 3C, the gamma characteristic Gnow is changed to the gamma characteristic Gnext according to the exposure change amount E. Thereby, the luminance conversion characteristic corresponding to the maximum input pixel level Dnext higher than the current maximum input pixel level Dnow is realized. That is, the luminance conversion characteristic corresponding to the input D range wider than the present is realized.

  Moreover, although the above-mentioned D range control expanded the D range of an image, in the D range compression process (D range suppression process) which compresses (suppresses) the D range of an image, the exposure condition is not necessarily changed. It is not necessary and may be changed only by the luminance conversion characteristic. Since Dnow> Dnext in the D range compression process, the characteristic shown in Dnext in FIG. 3C is changed to the characteristic shown in Dnow. Furthermore, the D range of the image may be expanded by a known HDR synthesis technique that combines a plurality of images with different exposure conditions to generate a wide dynamic range image.

  Subsequently, the D range control unit 109 uses the detection parameter determination unit 110 and the overexposure pixel detection unit 111 to calculate the overexposure degree of the face region detected by the face detection unit 105 (here, the number of overexposed pixels). To do. Then, the D range control unit 109 obtains a corrected target D range Dnext ′ obtained by correcting the target D range Dnext in accordance with the whiteness of the face area.

  As described above, in a subject area with a biased color component level, such as a human face, a color component with a high level is close to saturation even if the luminance is not saturated in a portion having luminance close to the saturation level. The gradation expression becomes poor, and pixels that give a visually uncomfortable feeling are generated. This sense of incongruity is noticeable when the D range is widened and the amount of change in the high-brightness portion of the gamma characteristic is reduced.

  Therefore, in the present embodiment, the degree of overexposure is detected in consideration of the luminance and the level frequency distribution of the color components for the pixels in the face area as an example of the subject area with a biased color component level. For example, a threshold value for determining whether or not the image is overexposed is provided for the luminance and the color component, and the threshold value is changed according to the width of the D range. Also, an evaluation value obtained by adding the number of pixels exceeding the luminance threshold and the number of pixels exceeding the color component threshold with a weight according to the width of the D range is obtained, and the final overexposure is determined from the size of the evaluation value. Judgment is made. The details of this whiteout determination process will be described later.

  The detection parameter determination unit 110 determines a detection parameter based on the target D range Dnext first determined by the D range control unit 109, and the overexposed pixel detection unit 111 detects overexposed pixels. The D range control unit 109 corrects the target D range Dnext according to the number of overexposed pixels detected by the overexposed pixel detection unit 111 to obtain a corrected target D range Dnext ′. The target D range correction process from Dnext to Dnext ′ is performed by the D range control unit 109 in the same manner as when Dnext is determined.

Next, the D range correction processing in the present embodiment will be described using the flowchart shown in FIG.
First, in S200, the D range control unit 109 acquires size and position information for each of the face regions detected by the face detection unit 105 in the image. If no face area is detected, the correction of the target D range is unnecessary, and the D range control unit 109 ends the process. On the other hand, if one or more face regions are detected, the D range control unit 109 outputs the target D range Dnext to the detection parameter determination unit, and instructs determination of a detection parameter according to the size of the target D range.

  In S201, the detection parameter determination unit 110 obtains a threshold and a weight for detecting overexposed pixels in the face area according to the target D range. The wider the target D range, the smaller the slope of the gamma characteristic applied by the gamma circuit 107 in the region where the input level (gradation value) is close to saturation. In other words, the wider the D range, the smaller the change in the output level with respect to the change in the input level in the region close to saturation, so that it looks more saturated. For example, in the face area, the R component level is higher than the G component and B component levels among the RGB components. Therefore, in the high luminance part, even if the luminance level is not saturated, the gradation property of the R component is poor, and it becomes easy to look unnatural. In this specification, such a pixel that appears to be saturated is referred to as a pixel that is visually uncomfortable near saturation.

In S201, the detection parameter determination unit 110 sets the color component threshold Th _CDnext and the luminance component threshold Th _YDnext to values corresponding to the target D range Dnext for detecting overexposed pixels. Further, the detection parameter determination unit 110 obtains a weight α _Dnext between the color component and the luminance component.

In the present embodiment, instead of a fixed threshold value, luminance and color component levels that are constant with respect to the target D range (upper limit level) are determined as threshold _values Th _CDnext and Th _YDnext . Then, the luminance component weight α _Dnext (0 ≦ α _Dnext ≦ 1) in overexposure pixel detection is determined to be large when the target D range Dnext is narrow and small as the target D range becomes wide. Note that the weight of the color component is (1-α).

In step S <b> 202, the overexposed pixel detection unit 111 detects overexposed pixels using the threshold value and weight determined in step S <b> 201 and the face area information detected by the face detection unit 105. Specifically, the whiteout pixel detection unit 111 obtains integrated values C _R , C _G , and C _B for each color component for the pixel values (R, G, B) in the face region, and the integrated value is the maximum. The obtained color component is selected as the maximum component C _Max . This can be expressed by the following formula.
_{C Max = Max (C R,} C G, C B) ... (1. 1)
For example, when integral values C _R , C _G , and C _B as shown in FIG. 5A are obtained, the overexposure pixel detection unit 111 selects the R component as the maximum component C _Max .

The whiteout pixel detection unit 111 also generates a pixel level frequency distribution (histogram) of the luminance component Y and the maximum color component _CMax . Then, the overexposed pixel detection unit 111 _{obtains the} number of pixels having a luminance level exceeding the threshold Th _YDnext and the number of pixels having a color component level exceeding the threshold Th _CDnext . FIG. 6A and FIG. 6B schematically show this pixel count calculation processing.

なお、Count()は、条件を満たす画素の数を与える関数である。 Next, the _whiteout pixel detection unit 111 calculates the final _whiteout pixel number score _Dnext by weighting and adding the number of pixels with a weight α _Dnext corresponding to the D range as follows.

Note that Count () is a function that gives the number of pixels that satisfy the condition.

FIG. 5B shows an example of calculating the overexposure pixel count score _Dnext . This is a case where the number of pixels having a luminance level exceeding the threshold Th _YDnext is 300, the number of pixels of the maximum color component C _Max having a color component level exceeding the threshold Th _CDnext is 500, and the weight α is 0.7. In this case, the overexposed pixel count score _Dnext is 360 according to the above equation.

The overexposed pixel detection unit 111 calculates the overexposed pixel ratio R _Dnext in the face area from the overexposed pixel count score _Dnext and the total face area face size facesize calculated as described above.

  Here, if the corrected target D range is too narrow, the overexposure of the face area will increase and the visual impression will deteriorate. On the other hand, if the corrected target D range is too wide, the overexposed pixels disappear, but the whole face area is covered and the gradation is lowered, and the visual impression is also deteriorated. From such a point of view, the target D range is corrected so that the D range has a good balance between the ratio of overexposed pixels and gradation.

D range control unit 109 in S203, the overexposed pixel ratio R _Dnext it is determined whether or larger than a predetermined value D _{Stay up-,} corrects the Dnext 'target D range from Dnext in S204 is larger. Here, correction for expanding the D range is performed in order to reduce overexposed pixels. After performing the D range expansion process, the process returns to S201, and the process based on the corrected target D range Dnet ′ is repeated.

On the other hand, if the overexposed pixel ratio R _Dnext is determined to be below a predetermined value D _{Stay up-at} S203, D range control unit 109 at S205, determines overexposed pixel ratio R _Dnext whether certain value D _Down smaller . When the overexposed pixel ratio R _Dnext is smaller than the predetermined value D _Down , the D range control unit 109 corrects the target D range from Dnext to Dnext ′ in S206. Here, correction for reducing the D range is performed to increase the number of overexposed pixels. After performing the D range reduction process, the process returns to S201, and the process based on the corrected target D range Dnet ′ is repeated.

If it is determined in S203 and S205 that the overexposed pixel ratio R _Dnext is equal to or less than the constant value D _{Up and} equal to or greater than the constant value D _Down, it is determined that the target D range has been reached and the D range control is performed. The unit 109 ends the D range correction process.

  As described above, in the present embodiment, the target dynamic range for suppressing overexposure according to the luminance of the entire image is corrected according to the overexposure degree in an area where the color component level is biased like the face area. To do. For this reason, it is possible to perform appropriate dynamic range control while suppressing the influence on the image quality of the region where the color component level is biased.

  In addition, when detecting the whiteness of an area with a biased color component level, a threshold corresponding to the target dynamic range is set for both the luminance and the color component, and Find the degree of overexposure for the level. Furthermore, the target dynamic range can be appropriately corrected by adding these whiteout degrees with weights corresponding to the target dynamic range to obtain the final whiteout degree.

  The target dynamic range and the correction process described above may be performed periodically or in response to a trigger event such as detection of a scene change. Accordingly, appropriate dynamic range control can be realized even when continuous shooting is performed as in a moving image.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (7)

An imaging device that performs dynamic range control of an image by performing exposure control and gain control according to the degree of whiteout of the image,
A determination means for determining a target dynamic range based on a whiteout degree of the entire image;
Detecting means for detecting a region having a biased color component level from the image;
Correction means for correcting the target dynamic range in accordance with the whiteness of the area biased in the level of the color component;
Control means for performing dynamic range control based on the corrected target dynamic range,
The correction means calculates a whiteness degree of a region biased in the color component level based on a luminance level and a color component level of a pixel included in the region, and is biased in the color component level. When the whiteout degree of a certain area is larger than the threshold value, a correction for expanding the target dynamic range is performed, and when the whiteout degree of the area where the color component level is biased is smaller than the threshold value, the target dynamic range is changed. An imaging apparatus that performs correction for reduction.   The correction means calculates a whiteness degree of a region biased in the color component level using a threshold value for luminance and a threshold value for the color component level, and a threshold value for the luminance and a threshold value for the color component level. Is set according to the target dynamic range.   An area in which the correction means is biased to the level of the color component with a value obtained by weighting and adding the number of pixels having a luminance exceeding the threshold for the luminance and the number of pixels having a color component level exceeding the threshold for the color component level 3. The imaging according to claim 1, wherein a weight for the number of pixels having a brightness exceeding a threshold value for the brightness is set to be larger as the target dynamic range is narrower. apparatus.   The correction means, when correcting the target dynamic range, calculates a whiteout degree of an area biased in the color component level using the corrected target dynamic range as a new target dynamic range. The imaging device according to any one of claims 1 to 3.   The correction means integrates the color component level of the pixel included in the region where the level of the color component is biased for each color component, and determines the level of the color component having the maximum integrated value and the pixel included in the region. 5. The image pickup apparatus according to claim 1, wherein a whiteout degree of an area where the color component level is biased is calculated based on a luminance level. 6.   The imaging apparatus according to claim 1, wherein the detection unit detects a human face area as an area having a biased level of the color component. A control method for an imaging apparatus that performs dynamic range control of an image by performing exposure control and gain control according to the whiteness of the image,
A determining step in which the determining means determines the target dynamic range based on the overexposure degree of the entire image;
A detection step in which the detection means detects a region having a bias in the level of the color component from the image;
A correcting step for correcting the target dynamic range in accordance with a whiteout degree of a region biased in the level of the color component;
A control means for performing dynamic range control based on the corrected target dynamic range,
In the correction step, the correction means calculates a whiteness degree of an area biased in the level of the color component based on a luminance level and a color component level of a pixel included in the area, and When the whiteout degree of the area biased in the level is larger than the threshold value, the target dynamic range is corrected, and when the whiteout degree of the area biased in the level of the color component is smaller than the threshold value, the correction is performed. A control method for an imaging apparatus, wherein correction for reducing a target dynamic range is performed.

Images