JP4948303B2 - Imaging apparatus and signal processing method - Google Patents

Description

The present invention is a signal processing method in an imaging apparatus such as a digital camera, and more specifically, processing at high speed during still image shooting by calculating WB correction coefficients for both moving image and still image during moving image processing. The present invention relates to a signal processing device that enables the above.

  2. Description of the Related Art Conventionally, shooting methods performed by an imaging device such as a digital camera are roughly classified into still image shooting for shooting still images and moving image shooting for displaying moving image recording or live view.

  When shooting a still image, for example, as disclosed in Patent Document 1, shooting is performed using an electronic shutter and a mechanical shutter. Here, a still image capturing sequence generally performed conventionally will be briefly described.

  First, a solid-state imaging device (for example, CCD) is reset (electronic shutter), charge accumulation is started, and after a desired charge accumulation time is over, the mechanical shutter is closed to complete exposure, and the accumulated charge is read from the CCD. When reading the charge from the CCD, smear components on the CCD transfer path are removed at high speed prior to the reading. Then, after removing the smear component, the charge is read out on the transfer path and transferred to take a still image.

  On the other hand, when shooting a moving image, the charge of the image for one field or one frame is sequentially read onto the transfer path and transferred to the outside of the CCD at a preset cycle determined by the moving image standard. The mechanical shutter is not used when shooting a moving image, and the charge accumulation time is from the reading of one image to the reading of the next image.

  As described above, in moving image shooting, unlike a still image, since smear components generated on the transfer path are not discharged prior to reading of charges, there is a problem that the smear components adversely affect the image. For example, if the subject brightness is high and the subject contains a large amount of smear components, the exposure may be set bright due to smear components, or the white balance may be magenta. Occurs.

  On the other hand, Patent Document 2 discloses a method of removing smear components by paying attention to the fact that smear is generated at substantially the same level in the vertical direction from the generation principle. In this method, first, vertical OB pixels (optical black) are provided in a light-shielded area outside the effective pixel area of the CCD, and an average value obtained by averaging the pixels of a plurality of lines in the vertical direction is calculated. And the method of subtracting from the signal obtained from the pixel of an effective screen by making the calculated average value into a reference signal is disclosed.

  Further, Patent Document 3 discloses a method for performing similar correction using a dummy pixel when a vertical OB pixel region cannot be set.

  Furthermore, Patent Document 4 discloses that the correction amount is changed according to the luminance level, and it is shown that the correction in the saturation region can be improved to some extent by using these methods.

JP-A-2-162976 JP 7-67038 A JP 2000-50165 A JP 2001-24943 A

  However, with the recent increase in the number of pixels and the definition of CCD, the smear level has deteriorated, and even if correction is performed using the above-described method, correction cannot be performed sufficiently. In that case, for example, as described in Patent Document 4, if the smear correction is performed by changing the correction level according to the luminance level, the linearity of the signal level is deteriorated and the image is deteriorated. Although it is possible to perform correction to such an extent that the linearity does not deteriorate, there is a problem in that an image with sufficient image quality cannot be obtained because sufficient smear correction cannot be performed.

  Some recent digital cameras have a moving image still image function for shooting a still image during moving image shooting. In the case of such a recording method, as described above, the mechanical shutter is used to shoot a still image, and the movie shooting is temporarily interrupted. It is required to shoot.

  There is also an imaging device having a live view function. Live view images are taken at a predetermined cycle in the same manner as the moving picture shooting described above, and the live view can be realized by sequentially displaying the shot images on a display provided in the imaging apparatus. In such an imaging apparatus, a user observes a live view image and instructs to take a still image at a desired timing, thereby taking a still image. At this time, in order to avoid the effect of smear, if white balance and exposure correction are calculated after image analysis after reading out charges for a plurality of fields as described above, it takes time to calculate. was there.

  Analysis information necessary for shooting still images from video or live view image data when shooting still images while shooting a movie or observing a live view image to speed up still image shooting Can be considered. However, because the output balance of the signal output from the CCD differs due to smearing between still image shooting and movie shooting, the exposure and white balance control information calculated from the movie and live view image is used as a still image. It cannot be used as it is for shooting. That is, in order to acquire analysis information necessary for capturing a still image from moving image or live view image data, it is necessary to sufficiently eliminate the influence of smear.

  However, for the reasons described above, if the smear correction is sufficiently performed on the moving image, the linearity is deteriorated and the image quality is deteriorated, so that the smear correction is sufficiently performed to shoot a moving image having sufficient image quality. There was a problem that could not.

In order to solve the above problems, a solid-state imaging device that performs photoelectric conversion and outputs an image signal, a smear detection unit that detects a smear component from a continuous image signal that is continuously output from the solid-state imaging device, and From the smear component detected by the smear detection means and a predetermined first smear correction characteristic representing the correction amount for correcting smear corresponding to the detected amount of the smear component, a first correction amount is obtained. A first determining means for determining; a smear component detected by the smear detecting means; and a second smear correction characteristic having a correction characteristic for reducing smear more strongly than the first smear correction characteristic. A second determination unit that determines a correction amount of 2, a first correction unit that performs smear correction on the continuous image signal by the first correction amount, and smear correction by the first correction unit. Signal processing means for performing signal processing for at least one of recording and display of the continuous image signal with respect to the continuous image signal, and smear correction with the second correction amount for the continuous image signal And a parameter for white balance correction processing of an image signal obtained when still image shooting is performed based on the continuous image signal smear corrected by the second correction unit. An imaging apparatus having a determining unit for determining is provided.

Further, in an imaging apparatus having a solid-state imaging device that performs photoelectric conversion and outputs an image signal, a smear detection unit detects a smear component from a continuous image signal continuously output from the solid-state imaging device; 1 is a smear component detected by the smear detection means, and a predetermined first smear correction characteristic representing a correction amount for correcting smear corresponding to the detected amount of the smear component, A step of determining a first correction amount; a second detection means having a smear component detected by the smear detection means; and a correction characteristic for reducing smear more strongly than the first smear correction characteristic. A step of determining a second correction amount from the two smear correction characteristics, a step of performing a smear correction by the first correction amount on the continuous image signal by the first correction unit, and a signal processing unit. A step of performing signal processing for at least one of recording and display of the continuous image signal on the continuous image signal smear corrected by the first correction unit; and a second correction unit includes the continuous image signal. A step of performing smear correction on the signal by the second correction amount, and when the determination unit performs still image shooting based on the continuous image signal smear corrected by the second correction unit. And determining a parameter for white balance correction processing of the obtained image signal.

  According to the present invention, it is possible to perform smear correction suitable for continuous images and to perform smear correction suitable for obtaining parameters for processing a still image.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a signal processing function in the imaging apparatus according to the embodiment of the present invention. Note that the imaging apparatus according to the present embodiment has a function of capturing a still image while observing a live view image displayed on an electronic viewfinder (EVF).

  In FIG. 1, reference numeral 100 denotes a solid-state image pickup device (hereinafter referred to as a CCD) represented by a CCD, which converts a subject optical image into electric charges by photoelectric conversion. As shown in FIG. 2, the CCD 100 includes an effective area that is exposed and outputs an image signal at the time of shooting, and an OB (optical black) portion that is shielded from light. Of the OB portion, the OB portion formed below the effective region is used to detect smear in the present embodiment, and is particularly called a smear detection region. Reference numeral 101 denotes a buffer that holds an image signal read from the CCD 100.

  Reference numeral 102 denotes a smear detection unit that detects smear from the image signal in the smear detection area among the image signals held in the image buffer 101. The detection result by the smear detection unit 102 is output to an EVF processing system and a still image WB (white balance) processing system, respectively.

  First, the EVF processing system will be described. Reference numeral 103 denotes an EVF smear correction amount determination unit (first determination unit) that determines an EVF smear correction amount according to the detection result of the smear detection unit 102. Reference numeral 105 denotes an EVF smear correction unit. Based on the EVF smear correction amount determined by the EVF smear correction amount determination unit 103, an image signal in the effective area among the image signals held in the image buffer 101. Correct smear. Reference numeral 106 denotes an EVF WB detection unit which calculates an EVF WB correction coefficient from the image signal smear corrected by the EVF smear correction unit 105. Reference numeral 107 denotes a signal processing unit, which performs WB correction on a smear-corrected image signal using the WB correction coefficient calculated by the EVF WB detection unit 106, converts it into image data suitable for EVF display, etc. Performs predetermined signal processing.

  Next, a processing system for the still image WB will be described. Reference numeral 104 denotes a still image smear correction amount determination unit (second determination unit) that determines a still image smear correction amount according to the detection result of the smear detection unit 102. Reference numeral 108 denotes a still image WB detection smear correction unit. Based on the still image smear correction amount determined by the still image smear correction amount determination unit 104, the effective region of the image signal held in the image buffer 101 is determined. Correct the image signal. Reference numeral 109 denotes a still image WB detection unit (determination unit) that calculates a still image WB correction coefficient based on the image signal smear corrected by the still image WB detection smear correction unit 108.

  Next, the operation of the imaging apparatus having the above-described configuration in this embodiment will be described with reference to the flowchart of FIG. Here, a description will be given assuming that EVF display is performed to capture a still image.

  First, in step S <b> 1, image signals are continuously read from the CCD 100 and accumulated in the image buffer 101. As described above, here, the CCD 100 is driven and the image signal (continuous image signal) is read for EVF display. The resolution and update rate of the image signal necessary for EVF display are determined by the display device that is the display destination, and the resolution is usually lower than that of a still image that is not subject to restrictions such as the display destination. For this reason, the image signal read in step S1 is an image signal with a low resolution (a small amount of data) read by performing addition reading and / or thinning reading or the like on the CCD 100.

  Next, in step S2, the smear detection unit 102 calculates the average value of the image signals in the OB area shown in FIG. 2 among the image signals accumulated in the image buffer 101 in step S1, and acquires the black signal level. . Next, in step S3, the smear detection unit 102 further adds the image signals in the smear detection area shown in FIG. 2 in the vertical direction and then calculates an average value. In step S4, a smear component in the horizontal direction (for each column) is detected by subtracting the black signal level calculated in step S2 from the average value calculated in step S3. In step S4, when the smear component is detected by the smear detection unit 102, using the detected smear component, the moving image processing by the EVF processing system and the still image WB processing system by the processing system for the still image WB are used. WB detection processing is performed.

  First, the moving image processing will be described.

  In step S5, the EVF smear correction amount (first correction amount) is determined by the EVF smear correction amount determination unit 103 in accordance with the amount of the smear component detected in step S4. The EVF smear correction amount determined in step S5 will be described later in detail. Subsequently, in step S6, the EVF smear correction unit 105 uses the EVF smear correction amount determined in step S5, and among the image signals in the effective area shown in FIG. The image signal is taken out and smear correction is performed. (First smear correction).

  In step S7, the EVF WB detection unit 106 calculates a WB correction coefficient from the smear-corrected image signal. In step S8, the signal processing unit 107 uses the calculated WB correction coefficient to perform the smear-corrected image. Perform WB correction of the signal. In step S9, the signal processing unit 107 performs signal processing necessary for EVF display on the WB-corrected image signal. The signal signal subjected to signal processing is displayed on a display (not shown) in step S10, and the above-described series of processing ends.

  Next, the still image WB detection process will be described.

  In step S11, the still image smear correction amount determination unit 104 determines a still image smear correction amount (second correction amount) according to the amount of the smear component detected in step S4. Here, a value that can reduce the smear component more strongly than the smear correction amount for EVF is determined as the smear correction amount for still images. The still image smear correction amount determined in step S11 will be described later together with the EVF smear correction amount determined in step S5. Subsequently, in step S12, the still image WB detection smear correction unit 108 uses the still image smear correction amount determined in step S11, and among the image signals read out in step S1, the image in the effective area shown in FIG. The signal is extracted and smear correction is performed. (Second smear correction).

  In step S13, the still image WB detection unit 109 calculates a WB correction coefficient from the smear-corrected image signal. Thereafter, in step S14, sensitivity / color correction is performed in view of the difference between the imaging drive modes of the still image and the EVF image, particularly the difference in charge accumulation time, and the final WB correction coefficient of the still image is calculated. In step S15, the still image WB correction coefficient calculated in step S14 is stored. This makes it possible to start still image signal processing without performing WB correction coefficient detection again during still image shooting. When the WB correction coefficient of the still image is stored, the above-described series of processing ends.

  Further, since the WB correction coefficient is a result of estimating the color temperature, other image parameters may be determined according to the estimated color temperature.

  For example, the correspondence between the WB correction coefficient and the color temperature is obtained in advance, and image processing parameters such as gain values for RY and BY signals when performing color signal processing are held for each color temperature. deep. Thus, it is possible to determine a parameter for signal processing by selecting a parameter according to the WB correction coefficient and the estimated color temperature.

  Next, a method for calculating the WB correction coefficient that can be divided in steps S7 and S13 will be described. Note that the calculation of the WB correction coefficient is a common technique, and a known calculation method can be used, and an example thereof will be briefly described here. Although the operation of the EVF WB detection unit 106 will be described below, the still image WB detection unit 109 can similarly detect the WB correction coefficient. At that time, as an image to be detected, a still image image smear corrected by the still image WB detection smear correction unit 108 may be used.

Assume that the CCD 100 is covered with a primary color Bayer array filter shown in FIG.
Ex = (RB) / Y
Ey = ((R + B) / 2− (G1 + G2) / 2) / Y
However, Y = 0.3R + 0.59G + 0.11B

  FIG. 5 shows a color evaluation value at each color temperature obtained by photographing a white subject in advance at a plurality of color temperatures existing in nature.

  FIG. 6 is a block diagram showing a function for calculating the WB correction coefficient in the present embodiment. In addition, the same reference number is attached | subjected to the structure similar to FIG. In FIG. 6, the WB detection parameter determination circuit 110 converts the WB detection parameter to the WB detection parameter candidate group 111 according to the WB control setting signal instructing acquisition of the WB correction coefficient output from the control unit (not shown) or the drive mode. Select more and decide. Then, the determined WB detection parameters (white portion detection area and detection color temperature setting range on the color evaluation value coordinates) are set in the EVF WB detection unit 106. The EVF WB detection unit 106 detects white pixels using the set WB detection parameters.

  To detect white pixels, first, an image is divided into a plurality of small areas (sample areas). Here, the small area indicates one area obtained by dividing one screen into m areas, and a signal obtained by averaging the RGB signals existing in the small area for each pixel is acquired.

  Next, the color evaluation value obtained based on the averaged signal acquired in each sample area is compared with the set WB detection parameter, and whether each sample area is included in the white discrimination range 500 of FIG. Each is determined. The white discrimination range 500 is an example of a white discrimination range set on the color evaluation value space, and the white discrimination range can be designated by a coordinate value or the like on a color space such as chromaticity coordinates. These WB detection parameters are generally changed according to the subject brightness, the white balance mode, and the like.

  When the sample area is included in the white discrimination range 500, the output value output from the pixel in the sample area is added for each color. The above-described determination is performed at all the sample points on the screen, and the gain is obtained for each color so that the average value of the RGB signals becomes the same level from the added average value, and the gain is set as the WB correction coefficient. As described above, the WB correction coefficient can be obtained.

  Next, the signal subjected to smear correction for EVF is subjected to WB correction by the WB correction coefficient calculated by the WB control unit 112 in the signal processing unit 107. The signal subjected to WB correction is subjected to control of saturation and hue by performing a color matrix, high luminance achromaticity, high saturation color suppression, and the like in the color signal processing unit 113, and a color signal is created. In addition, the luminance signal processing unit 114 performs luminance gradation correction, edge enhancement signal processing, and the like on the signal subjected to WB correction to generate a luminance signal.

  Next, the smear correction amount determination method performed in step S5 and step S11 will be described.

  FIG. 7 shows smear correction characteristics for moving images and still images. The horizontal axis represents the smear detection value as input, and the vertical axis represents the smear correction amount as output. As shown in FIG. 7, in the present embodiment, when the amount of smear is smaller than a predetermined amount of smear (predetermined amount) (in FIG. 7, the smear detection signal is up to 100), the detected smear is detected. Correction is performed as it is using the detected value. And it sets so that a smear correction amount may become small as a smear amount increases. Further, the smear correction amount for still images has a characteristic of becoming larger than that for moving images when a predetermined smear amount is exceeded. The reason for setting the smear correction amount in this way will be described below.

  As described above, in principle, the smear component can be removed and corrected by subtracting the signal amount detected as the smear component from the signal value output from the solid-state imaging signal.

  FIG. 8A is a diagram showing the relationship between smear and signal value in the normal exposure area (area of the image signal level output from the pixel that is not saturated among the image signals output from the effective area). FIG. 8B is a diagram showing signal values after the smear component is subtracted from the image signal shown in FIG. As can be seen from FIG. 8, since the normal linearity is maintained in the normal exposure region, it is possible to appropriately correct the smear by subtracting the smear component as it is.

  On the other hand, if the smear component becomes too large, the signal value is unnecessarily lowered by subtracting the smear component in a region where saturation occurs. Subtraction of smear components results in image degradation that does not reach the saturation level and is colored.

  In general, black level adjustment and A / D conversion are performed on the sensor output within an AFE (Analog Fornt End). For example, if AFE performs A / D conversion with 10 bits, the output of the sensor is very high. If a saturation signal exceeding 10 bits is input to AFE, 10 bits will be generated when A / D conversion is performed. Clipped in range.

  FIG. 9A shows the relationship between smear and signal value in the saturation region (region of the image signal level output from the saturated pixel in the image signal output from the effective region). FIG. 9B shows a signal value after clipping by AFE, and FIG. 9C shows an example in which smear correction is performed on the image signal shown in FIG. 9B. As can be seen from FIG. 9, when smear removal is performed on the image signal in the saturated region, the signal value becomes insufficient.

  Due to the phenomenon described above, if the smear component detected from the display image displayed on the liquid crystal or the like or the image recorded in the moving image is removed as it is, the high luminance portion is expressed darkly, and the image quality is impaired. Also, considering that the amount of smear generated has increased with the recent miniaturization of sensors, it is even more difficult to completely remove the smear.

  On the other hand, the corrected image signal is not displayed for the image signal for detecting the still image WB. Further, in actual still image shooting, smear on the transfer path is discharged prior to reading of the image signal, so that it is not affected by smear. For this reason, it is necessary to sufficiently remove the smear component in order to calculate the WB correction coefficient for a still image from the image obtained during EVF display. In order to sufficiently remove smear components from an image obtained during EVF display, the correction characteristic for still images is made stronger than that for moving images, that is, the value of the correction amount is increased.

  For the EVF display image, the correction amount of the smear component is set so as not to deteriorate the linearity. By doing so, it is possible to obtain an image signal from which a smear component has been sufficiently removed for calculating a WB correction coefficient for a still image while suppressing deterioration in image quality of the image for EVF.

  The EVF smear correction amount determination unit 103 and the still image smear correction amount determination unit 104 hold the smear correction amount for the amount of smear components (smear detection signal) shown in FIG. Thereby, it is possible to determine the smear correction amount according to the amount of the smear component detected by the smear detection unit 102. Alternatively, the smear correction amount for the smear detection signal shown in FIG. 7 is held in a memory (not shown) so that the EVF smear correction amount determination unit 103 and the still image smear correction amount determination unit 104 can refer to them as necessary. You may comprise.

  As described above, according to the present embodiment, when performing smear correction, in an image for EVF display, it is sufficient to suppress deterioration in image quality due to smear correction and to acquire parameters for processing a still image. An image signal subjected to smear correction can be obtained.

  In the above-described embodiment, the case where the EVF display is performed to capture a still image has been described. However, the present invention is similarly applied to the case where a still image is captured during moving image recording. Is possible.

It is a block diagram which shows the signal processing function in the imaging device of embodiment of this invention. It is a figure for demonstrating the area | region of CCD in embodiment of this invention. It is a flowchart which shows the operation | movement in embodiment of this invention. It is a figure which shows an example of a color filter arrangement | sequence. It is a figure which shows an example of the color evaluation value space for white balance, and a white discrimination range. It is a block diagram which shows the calculation function of the WB correction coefficient in embodiment of this invention. It is a figure which shows the smear correction characteristic for moving images and still images. It is a figure which shows the relationship between the smear and signal value in a normal exposure area | region. It is the figure which showed the mode of the smear correction | amendment in a saturation area | region.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Solid-state image sensor 101 Image buffer 102 Smear detection part 103 EVF smear correction amount determination part 104 Still image smear correction amount determination part 105 EVF smear correction part 106 EVF WB detection part 107 Signal processing part 108 For still image WB detection Smear correction unit 109 WB detection unit for still image 110 WB detection parameter determination circuit 111 WB detection parameter candidate group 112 WB control unit 113 color signal processing unit 114 luminance signal processing unit

Claims (3)

A solid-state imaging device that performs photoelectric conversion and outputs an image signal;
Smear detection means for detecting a smear component from a continuous image signal continuously output from the solid-state imaging device;
From the smear component detected by the smear detection means and a predetermined first smear correction characteristic representing the correction amount for correcting smear corresponding to the detected amount of the smear component, a first correction amount is obtained. First determining means for determining;
A second correction amount is determined from a smear component detected by the smear detection means and a second smear correction characteristic having a correction characteristic for reducing smear more strongly than the first smear correction characteristic. And means for determining
First correction means for performing smear correction on the continuous image signal with the first correction amount;
Signal processing means for performing signal processing for at least one of recording and display of the continuous image signal with respect to the continuous image signal smear corrected by the first correction means;
Second correction means for performing smear correction on the continuous image signal by the second correction amount;
Determining means for determining a parameter for white balance correction processing of an image signal obtained when a still image is taken based on the continuous image signal smear corrected by the second correction means. An imaging device that is characterized. The second determining means sets the second correction amount to the same value as the first correction amount when the detection amount of the smear component detected by the smear detection means does not exceed a predetermined amount. When the smear component exceeds the predetermined amount, the second correction amount larger than the first correction amount is determined based on the second smear correction characteristic. Item 2. The imaging device according to Item 1. A signal processing method in an imaging apparatus having a solid-state imaging element that performs photoelectric conversion and outputs an image signal,
A step of detecting a smear component from a continuous image signal continuously output from the solid-state imaging device by a smear detection means;
The first detection means includes a smear component detected by the smear detection means, and a predetermined first smear correction characteristic representing a correction amount for correcting smear corresponding to the detected amount of the smear component. Determining a first correction amount;
The second detection means includes a smear component detected by the smear detection means, and a second smear correction characteristic having a correction characteristic for reducing smear more strongly than the first smear correction characteristic. Determining a correction amount;
A first correction unit performing smear correction on the continuous image signal by the first correction amount;
A signal processing unit performing signal processing for at least one of recording and display of the continuous image signal on the continuous image signal smear corrected by the first correction unit;
A step in which a second correction unit performs smear correction on the continuous image signal by the second correction amount;
Determining a parameter for white balance correction processing of an image signal obtained when a still image is captured based on the continuous image signal smear corrected by the second correction unit; A signal processing method comprising:

Images