JP5169318B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method including a technique for performing strobe white balance control, and a computer-readable recording medium.

  Conventionally, for images taken with a flash, a white balance that applies white to the subject, which is supposed to be white, is applied to the entire screen. The area located at is strongly affected by strobe light, and the area located far from the camera is less affected by strobe light and strongly influenced by outside light. For this reason, when the white balance gain matched to the strobe light is applied to the entire screen, it is not possible to apply the optimum white balance to an area where the influence of external light is strong. Conversely, when a white balance gain that matches external light is applied to the entire screen, an optimal white balance cannot be applied to an area that is strongly affected by strobe light. Therefore, in a scene where both areas exist, there is a problem that it is difficult to apply an optimal white balance to the entire screen.

  In order to solve the above-described problem, there is a technique that does not cause a color shift with respect to the entire screen by performing area division by comparing an image in which a strobe light is emitted and an image in which light is not emitted, and applying white balance for each area. (For example, Patent Document 1)

Japanese Patent No. 3569352

According to the above-described conventional technology, since the white balance gain is set for each block, an unnatural color shift occurs at the boundary between the blocks, and a photographed image with and without strobe light emission is required. There is a problem that a memory for storing two captured images is required, and a processing time required for photographing increases.
The present invention has been made in view of the above-described circumstances, and good white balance correction that does not cause color misregistration on the entire screen in strobe shooting is performed in a processing time comparable to that of normal single-shooting and is small. An object is to provide an imaging apparatus and an imaging method that can be executed in a memory, and a computer-readable recording medium.

As a result of intensive studies in order to solve the above problems, the inventors of the present invention have specifically described at least the imaging apparatus and imaging method and computer-readable recording medium according to the present invention as described in (1) to ( 1) below. It has been found that the above-mentioned problems can be solved by having the technical features described in 9) , and the present invention has been completed.

(1): an imaging element that converts light incident from the optical system into an electrical signal and outputs the electrical signal, an imaging unit that stores the imaging signal in a memory, and a white that extracts a white balance evaluation value from the imaging signal In the imaging apparatus including a balance evaluation value acquisition unit, the imaging unit stores a first imaging function for storing an imaging signal without strobe light emission in the memory as a first imaging signal, and a first imaging signal with strobe light emission. A second imaging function for storing the first imaging signal and the second imaging signal into a plurality of blocks, and a divided block. Luminance signal integration means for integrating the luminance signal every time to calculate the luminance integration result, the luminance integration result of the block of the first imaging signal, and the block of the second imaging signal In the luminance integration result of the image, an area dividing unit that divides the area by comparing a calculated value calculated by calculating each block at the same position with a predetermined reference value, and the division Correction coefficient calculation means for calculating a correction coefficient for white balance for each area based on the white balance evaluation value for each area, and white balance calculation means for setting a correction coefficient for white balance for each of the divided areas And an exposure correction coefficient such that an exposure amount is the same between the first imaging signal and the second imaging signal, and a shutter speed, an aperture value, and an AGC of the first imaging function at the time of imaging. Calculate from the gain value, ND filter position, pixel addition, shutter speed, aperture value, AGC gain value, and ND filter position at the time of imaging by the second imaging function, An imaging apparatus characterized by having the exposure compensation coefficient calculating means for setting the serial luminance accumulation result for each block of the first image pickup signal and / or the luminance accumulation result for each block of the second image pickup signal.

(2): a white detection unit that performs white detection based on the white balance evaluation value and a preset white detection range; and the correction coefficient calculation unit includes a white detection unit that detects white detected by the white detection unit. The imaging apparatus according to (1), wherein a correction coefficient for white balance for each region is calculated based on a detection result and the white balance evaluation value for each of the divided regions.

(3) : The exposure correction coefficient calculation means calculates a block whose luminance signal is a predetermined value or less among the block of the first imaging signal and / or the block of the second imaging signal. The imaging apparatus according to (1) or (2) , wherein the exposure correction coefficient is weighted.

(4) : a photoelectric conversion step of converting light incident from the optical system into an electric signal by an image pickup device and outputting it as an image pickup signal, an image pickup step of storing the image pickup signal in a memory, and a white balance evaluation value from the image pickup signal A white balance evaluation value acquisition step of extracting the image, wherein the imaging step includes a first imaging step of storing an imaging signal without strobe emission in the memory as a first imaging signal, and a strobe emission A second imaging step of storing the imaging signal as a second imaging signal in the memory, and a block dividing step of dividing the first imaging signal and the second imaging signal into a plurality of blocks, A luminance signal integration step of calculating a luminance integration result by integrating luminance signals for each of the divided blocks, a luminance integration result of the block of the first imaging signal, and the second In the luminance integration result of the blocks of the imaging signal, an area dividing step of dividing the area by comparing the calculated value calculated by calculating each block between the same position and a predetermined reference value, A correction coefficient calculation step of calculating a correction coefficient for white balance for each area based on the white balance evaluation value for each of the divided areas, and white for setting a correction coefficient for white balance for each of the divided areas A balance calculation step, and an exposure correction coefficient such that an exposure amount is the same between the first imaging signal and the second imaging signal, and a shutter speed and an aperture value at the time of imaging by the first imaging function. , AGC gain value, ND filter position, pixel addition, shutter speed, aperture value, AGC gain value, ND filter position at the time of imaging by the second imaging function, And al calculation, imaging and having an exposure correction coefficient calculation step of setting a luminance accumulation result for each block of the first luminance accumulation result for each block of the image signal and / or the second image pickup signal Is the method.

(5) : a white detection step of performing white detection based on the white balance evaluation value and a preset white detection range, wherein the correction coefficient calculation means detects the white detected by the white detection means a detection result, an imaging method according to the item (4), and calculates a correction coefficient for white balance for each region based on said white balance evaluation value of the divided each region.

(6) : In the exposure correction coefficient calculation step, a block whose luminance signal is a predetermined value or less is calculated in the block of the first imaging signal block and / or the second imaging signal block. The imaging method according to (4) or (5) , wherein the exposure correction coefficient is weighted.

(7) : Photoelectric conversion processing in which light incident from an optical system is converted into an electrical signal by an imaging device and output as an imaging signal in a computer included in an apparatus having an image processing function, and imaging in which the imaging signal is stored in a memory And a white balance evaluation value acquisition process for extracting a white balance evaluation value from the imaging signal, and a computer-readable recording medium storing a program for executing the processing, wherein the imaging process is performed without flash emission. A first imaging process for storing an imaging signal as a first imaging signal in the memory; and a second imaging process for storing an imaging signal with strobe light emission as a second imaging signal in the memory; A block division process for dividing one image pickup signal and the second image pickup signal into a plurality of blocks, and a luminance signal for each divided block. The luminance signal integration processing for calculating the luminance integration result by calculating, the luminance integration result of the block of the first imaging signal, and the luminance integration result of the block of the second imaging signal A region division process for dividing the region by comparing a calculated value calculated by calculating each other with a predetermined reference value, and for each region based on the white balance evaluation value for each of the divided regions A correction coefficient calculation process for calculating a white balance correction coefficient and a white balance calculation process for setting a white balance correction coefficient for each of the divided areas , the first imaging signal, The exposure correction coefficient is set so that the exposure amount is the same as that of the second imaging signal, and the shutter speed, aperture value, AGC gain value, ND filter position, image at the time of imaging by the first imaging function. Calculated from the added amount and the shutter speed, aperture value, AGC gain value, and ND filter position at the time of imaging by the second imaging function, and the luminance integration result for each block of the first imaging signal and / or the above A computer-readable recording medium storing a program for executing a white balance calculation process, characterized in that it comprises an exposure correction coefficient calculation process for setting a luminance integration result for each block of a second imaging signal .

(8) : a white detection process for performing white detection based on the white balance evaluation value and a preset white detection range, wherein the correction coefficient calculation process includes the white detection process detected by the white detection process; The white balance calculation processing according to (7) is executed, wherein a correction coefficient for white balance for each region is calculated based on the detection result and the white balance evaluation value for each of the divided regions. A computer-readable recording medium on which a program for causing the program to be recorded is recorded.

(9) : In the exposure correction coefficient calculation process, a block whose luminance signal is a predetermined value or less is calculated in the block of the first imaging signal block and / or the second imaging signal block. A computer-readable recording medium storing a program for executing the white balance calculation process according to (7) or (8) , wherein the exposure correction coefficient is weighted.

  According to the present invention, an imaging device and an imaging method capable of executing a good white balance correction that does not cause a color shift on the entire screen in strobe shooting in a processing time comparable to that of normal single-frame shooting, and computer-readable A recording medium can be provided.

  An imaging apparatus according to the present invention includes an imaging element that converts light incident from an optical system into an electrical signal and outputs the electrical signal, an imaging unit that stores the imaging signal in a memory, and a white balance evaluation value from the imaging signal. An image pickup apparatus comprising: a white balance evaluation value acquisition unit that extracts a first image pickup function that stores an image pickup signal without strobe light emission in the memory as a first image pickup signal; and image pickup with strobe light emission. A second imaging function for storing a signal in the memory as a second imaging signal, and a block dividing unit that divides the first imaging signal and the second imaging signal into a plurality of blocks, and the division Luminance signal integration means for calculating the luminance integration result by integrating the luminance signal for each of the blocks, the luminance integration result of the block of the first imaging signal, and the second imaging In the luminance integration result of the block of the signal, an area dividing unit that divides the area by comparing a calculated value calculated by calculating each of the blocks at the same position with a predetermined reference value, and Correction coefficient calculation means for calculating a correction coefficient for white balance for each area based on the white balance evaluation value for each divided area, and white balance for setting a correction coefficient for white balance for each of the divided areas And an arithmetic means.

Next, the basic configuration of the imaging apparatus according to the present invention will be described in more detail with reference to the drawings.
The embodiments described below are preferred embodiments of the present invention, and thus various technically preferable limitations are given. However, the scope of the present invention particularly limits the present invention in the following description. Unless otherwise stated, the present invention is not limited to these embodiments. However, among the embodiments described below, examples other than the first embodiment are examples corresponding to reference examples not belonging to the scope of the present invention.

[First Embodiment]
(Appearance structure of digital camera)
FIG. 1 is a schematic diagram showing an external appearance of a digital camera which is a first embodiment of an imaging apparatus according to the present invention.
As shown in FIG. 1, a release button (shutter button), a power button, and a shooting / playback switching dial are provided on the upper surface side of the digital camera according to the present embodiment, and the front (front) side of the digital camera. Includes a lens barrel unit having a photographing lens system, a strobe light emitting unit, and an optical viewfinder.

  On the back side of the digital camera is a liquid crystal monitor (LCD), the eyepiece of the optical viewfinder, wide-angle zoom (W) switch, telephoto zoom (T) switch, menu (MENU) button, and enter button (OK button) Etc. are provided. In addition, a memory card storage unit for storing a memory card for storing captured image data is provided inside the side surface of the digital camera.

(Digital camera system configuration)
FIG. 2 is a block diagram showing a configuration of a digital camera which is the first embodiment of the imaging apparatus according to the present invention.
As shown in FIG. 2, in this digital camera, a CCD, a CCD as a solid-state imaging device, on which a subject image incident through a photographing lens system (optical system) installed in a lens barrel unit is formed on a light receiving surface. An analog front end unit (AFE unit) that processes electrical signals (analog RGB image signals) that are output into digital signals (imaging signals), a signal processing unit that processes digital signals output from the AFE unit, and data temporarily An SDRAM (memory) to be stored, a ROM storing a control program, a motor driver, and the like are provided.

  The lens barrel unit includes a photographic lens system having a zoom lens, a focus lens, and the like, an aperture unit, and a mechanical shutter unit. Each drive unit (not shown) of the photographic lens system, the aperture unit, and the mechanical shutter unit is a motor driver. Driven by. The motor driver is driven and controlled by a drive signal from a control unit (CPU) of the signal processing unit.

  In the CCD, RGB primary color filters as color separation filters are arranged on a plurality of pixels constituting the CCD, and electrical signals (analog RGB image signals) corresponding to the three primary colors of RGB are output.

  The AFE unit includes a TG (timing signal generation unit) that drives the CCD, a CDS (correlated double sampling unit) that samples an electrical signal (analog RGB image signal) output from the CCD, and an image signal sampled by the CDS. An AGC (analog gain control unit) that adjusts the gain and an A / D that converts the image signal gain-adjusted by the AGC into a digital signal (RAW-RGB data) are provided.

The signal processing unit outputs a screen horizontal synchronizing signal (HD) and a screen vertical synchronizing signal (VD) to the TG of the AFE unit, and the RAW− output from the A / D of the AFE unit according to these synchronizing signals. A CCD interface (CCD I / F) that captures RGB data, a memory controller that controls the SDRAM, a YUV converter that converts the captured RAW-RGB data into YUV format image data that can be displayed and recorded, and display and recording A resizing processing unit that changes the image size according to the size of the image data to be displayed, a display output control unit that controls display output of the image data, a data compression unit for recording image data by JPEG formation, and the like Media interface that writes data to a memory card or reads image data written to a memory card ( And Deer I / F), based on the operation input information from the operation unit, and a control section for entire digital camera system control or the like based on the control program stored in a ROM (CPU).
A part of the CCD I / F functions as an image pickup means in the present invention.

  The operation unit includes a release button, a power button, a shooting / playback switching dial, a wide-angle zoom switch, a telephoto zoom switch, a menu button, a confirmation button, and the like provided on the external surface of the digital camera (see FIG. 1). A predetermined operation instruction signal is input to the control unit by a photographer's operation.

  In the SDRAM, the RAW-RGB data captured by the CCD I / F is stored, and YUV data (YUV format image data) converted by the YUV conversion unit is stored, and further compressed by the data compression unit. The image data such as JPEG formation is stored.

  Note that YUV of the YUV data is luminance data (Y), color difference (difference (U) between luminance data and blue (B) component data, and difference (V) between luminance data and red (R) component data). It is a format that expresses color with information.

(Digital camera monitoring and still image shooting)
Next, the monitoring operation and still image shooting operation of the digital camera will be described. This digital camera performs a still image shooting operation while performing a monitoring operation as described below in the still image shooting mode.

  First, when the photographer turns on the power button and sets the photographing / playback switching dial to the photographing mode (still image photographing mode), the digital camera is activated in the recording mode. When the control unit detects that the power button is turned on and the shooting / playback switching dial is set to the shooting mode, the control unit outputs a control signal to the motor driver, and moves the lens barrel unit to the shooting enabled position. In addition, the CCD, the AFE unit, the signal processing unit, the SDRAM, the ROM, the liquid crystal monitor, and the like are activated.

  Then, by directing the photographic lens system of the lens barrel unit toward the subject, a subject image incident through the photographic lens system is formed on the light receiving surface of each pixel of the CCD. An electrical signal (analog RGB image signal) corresponding to the subject image output from the CCD is input to the A / D via the CDS and AGC, and converted into 12-bit (RAW) RGB data by the A / D. Convert.

  This RAW-RGB data (first image pickup signal) is taken into the CCD I / F of the signal processing unit (first image pickup function) and stored in the SDRAM via the memory controller. The RAW-RGB data read from the SDRAM is input to the YUV converter and converted into YUV data (YUV signal) that can be displayed, and then the YUV data is stored in the SDRAM via the memory controller. Is done.

  The YUV data read from the SDRAM via the memory controller is sent to the liquid crystal monitor (LCD) via the display output control unit, and a captured image (moving image) is displayed. At the time of monitoring in which a photographed image is displayed on the above-described liquid crystal monitor (LCD), one frame is read out in a time of 1/30 second by thinning out the number of pixels by the CCD I / F.

  In this monitoring operation, the photographed image (moving image) is only displayed on the liquid crystal monitor (LCD) functioning as an electronic viewfinder, and the release button is not yet pressed (including half-pressed). .

  By displaying the photographed image on a liquid crystal monitor (LCD), the composition for photographing a still image can be confirmed. In addition, it can output as a TV video signal from a display output control part, and a picked-up image (moving image) can also be displayed on external TV (television) via a video cable.

Then, the CCD I / F of the signal processing unit calculates an AF (automatic focus) evaluation value, an AE (automatic exposure) evaluation value, and an AWB (auto white balance) evaluation value from the captured RAW-RGB data.
In the present invention, the white balance evaluation value is an AWB evaluation value, the luminance integration result is an AE evaluation value, and each of the white balance evaluation value acquisition unit and the luminance signal integration unit is a signal processing unit. Part of the CCD I / F.

  The AF evaluation value is calculated by, for example, the output integrated value of the high frequency component extraction filter or the integrated value of the luminance difference between adjacent pixels. When in the in-focus state, the edge portion of the subject is clear, so the high frequency component is the highest. By using this, at the time of AF operation (focus detection operation), AF evaluation values at each focus lens position in the photographing lens system are acquired, and AF operation is performed with the maximum point as the focus detection position. Is executed.

  The AE evaluation value and the AWB evaluation value are calculated from the integrated values of the RGB values in the RAW-RGB data. For example, a screen corresponding to the light receiving surface of all the pixels of the CCD is equally divided into 256 areas (blocks) (horizontal 16 divisions, vertical 16 divisions) by a detection circuit which is a part of the CCD I / F of the signal processing unit which is a block division means. ), The RGB integrated value of each area (block) is calculated by the CCD I / F.

  Then, the control unit reads the calculated RGB integrated value, and in the AE process, calculates the luminance of each area (block) of the screen and determines an appropriate exposure amount from the luminance distribution. Based on the determined exposure amount, exposure conditions (the number of electronic shutters of the CCD, the aperture value of the aperture unit, the insertion and removal of the ND filter, etc.) are set. In the AWB process, an AWB control value that matches the color of the light source of the subject is determined from the RGB distribution. By this AWB process, white balance is adjusted when the YUV conversion unit performs conversion processing to YUV data. The AE process and AWB process described above are continuously performed during the monitoring.

  When a still image shooting operation in which the release button is pressed (half-pressed to fully pressed) is started during the monitoring operation described above, an AF operation that is a focus position detection operation and a still image recording process are performed.

  That is, when the release button is pressed (half-pressed to fully pressed), the focus lens of the photographic lens system is moved by a drive command from the control unit to the motor driver. For example, a contrast evaluation method called so-called hill-climbing AF AF operation is executed.

  So-called hill-climbing AF means that when the AF (focusing) target range is the entire range from infinity to close, the focus lens of the taking lens system is at each focus position from close to infinity or from infinity to close. The controller reads the AF evaluation value at each focus position calculated by the CCD I / F. Then, the focus lens is moved to the in-focus position with the point where the AF evaluation value at each focus position is maximized as the in-focus position, and in-focus.

  Then, the AE process described above is performed, and when the exposure is completed, the mechanical shutter unit is closed by a drive command from the control unit to the motor driver, and an analog RGB image signal for a still image is output from the CCD. Then, as in the monitoring, the A / D conversion unit of the AFE unit converts the data into RAW-RGB data.

  The RAW-RGB data (second image pickup signal) is taken into the CCD I / F of the signal processing unit (second image pickup function), converted into YUV data by the YUV conversion unit, and SDRAM via the memory controller. Saved in. The YUV data is read from the SDRAM, converted into a size corresponding to the number of recorded pixels by the resizing processing unit, and compressed to image data in JPEG format or the like by the data compression unit. The compressed image data such as JPEG format is written back to the SDRAM, read out from the SDRAM via the memory controller, and stored in the memory card via the media I / F.

Next, an imaging operation (operation after the start of operation of the second imaging function) that is a feature of the present invention will be described.
FIG. 3 is a flowchart showing a series of processing for strobe WB correction in the first embodiment.
First, RAW-RGB at the time of photographing by flash emission is stored in the SDRAM (S1). The amount of stroboscopic light emission is adjusted by detecting the reflected light from the object obtained by the pre-emission before the main emission with the CCD.

The RAW-RGB stored in the SDRAM is divided into 256 blocks of horizontal 16 × vertical 16 by a detection circuit which is a part of the CCD I / F of the signal processing unit which is a block dividing means.
At the same time, an AWB evaluation value (white balance evaluation value), an AE evaluation value (luminance signal integration result) and an AE control value (shutter speed, aperture value, AGC gain value, ND filter position) at the time of photographing are stored in the internal RAM. Stored (S2). The AWB evaluation value is expressed in the form of G / R and G / B after calculating R, G, and B per pixel by integrating R, G, and B for each block. The AE evaluation value is expressed as a result of the following equation after calculating R, G, and B per pixel by integrating R, G, and B for each block.

      AE evaluation value = 0.299R + 0.587G + 0.114B

  Next, an exposure correction coefficient (k) calculated by a calculation method to be described later is used for each block corresponding to the monitoring image so that the exposure amount at the time of monitoring immediately before shooting is the same as the exposure amount at the time of shooting. An evaluation value and / or an AE evaluation value for each block at the time of shooting is set (S3; exposure correction coefficient calculation means). The AE evaluation value corresponding to the monitoring image (first image pickup signal) immediately before shooting is stored in the RAM, which is calculated from the last frame of the release using the same calculation formula as when shooting. As in the case of shooting, the AE control values (shutter speed, aperture value, AGC gain value, ND filter position, pixel addition amount) at this time are also stored in the internal RAM. In addition, when adjusting the exposure amount, there is no change even if the exposure correction coefficient is multiplied to the block with no light amount, and the division method that divides according to the difference size makes division difficult, so the exposure amount of the image with a small exposure amount is reduced. It is preferable to multiply an image with a large exposure amount by an exposure correction coefficient (k) so as to match.

Although the example of the AE control value in this Embodiment is shown in FIG. 4, the AE control value in this invention is not limited to this.
Since the amount of change in brightness per step of each AE control value is the same, the difference in brightness between shooting and monitoring (delta Ev) by subtraction from the addition result of each AE control value during shooting and monitoring. ). In the example of the AE control value in the present embodiment, the difference in control value is 16 and 1 EV difference, that is, the brightness is twice different. The calculation method of the exposure correction coefficient (k) is shown in the following formula 1.

  When the AE evaluation value is calculated from the result of adding horizontal and vertical pixels at the time of monitoring, it is necessary to consider the pixel addition when calculating the exposure correction coefficient. The calculation method of the exposure correction coefficient (k) It becomes. For example, when the result of summing two horizontal pixels and two vertical pixels (for four pixels) is read out from the CCD during monitoring, 2 EV (4 pixels during monitoring) compared to shooting (for one pixel) without pixel addition. Times) will be brighter.

Then, the AE evaluation value of the monitoring image immediately before shooting is subtracted from the AE evaluation value of the captured image of the same block, and compared with a reference value set in advance. Area division is performed using the block as a non-flash irradiation area (S4; area division means).
The determination method of the strobe irradiation area in the present invention is not limited to the determination method of S4, and any determination method can be used as long as it is a calculation method that can determine a block that is sufficiently irradiated with strobe light. The reference value set in advance can be arbitrarily set.

  With S1 to S4, it is possible to extract a block having a strong influence of a strobe even when the exposure amount differs between shooting and monitoring.

  FIG. 5 shows an area divided into a strobe irradiation area and a strobe non-irradiation area in block units. 5 in FIG. 5 represents a strobe irradiation region, and 2 in FIG. 5 represents a strobe non-irradiation region.

  Next, white extraction (also referred to as white detection in the present invention; hereinafter the same) is performed for each region (S5; white detection means). In the white extraction, an AWB evaluation value acquired in units of blocks for each region is compared with a threshold value, and blocks below the threshold value are stored as white extraction blocks.

  If a specific example is shown in the captured image as shown in FIG. 5, the white extraction block is a block that exists in the area of white clothes in the image of the person who is the subject, and a white cloud area in the background image. Examples include (existing) blocks.

Here, in the white extraction processing in the present embodiment, the white extraction block is determined based on whether the AWB evaluation value of the extracted block is equal to or greater than a predetermined threshold value or less than the predetermined threshold value. The predetermined threshold can be arbitrarily set.
The present invention is not limited to the above-described embodiment of the white extraction process, and any form can be used as long as a block in which white is present in a subject or background can be extracted with high reliability according to a certain standard. There may be. In the present embodiment and the embodiments described later, the white extraction range (white detection range) can be an arbitrary range in which the white extraction block can be extracted with high reliability.

When the number of white extraction blocks is equal to or greater than a certain standard in both the strobe irradiation area and the strobe non-irradiation area, the AWB evaluation value in each white extraction block for each of the strobe irradiation area and the strobe non-irradiation area By calculating based on the AE evaluation value, the white balance correction coefficient can be calculated.
The white balance correction coefficient is preferably a correction coefficient that places importance on an area where the AE evaluation value is high, and can be calculated, for example, by weighting the AE evaluation value to the AWB evaluation value as shown in Equation 3 below. .

In addition, when the number of white extraction blocks is less than a certain reference in either one of the strobe irradiation area and the strobe non-irradiation area, the area where the number of white extraction blocks is less than a certain reference is the AWB evaluation value in each block. By calculating based on the AE evaluation value, the white balance correction coefficient can be calculated.
The white balance correction coefficient is preferably a correction coefficient that places importance on an area where the AE evaluation value is high, and can be calculated, for example, by weighting the AE evaluation value to the AWB evaluation value as shown in Equation 4 below. .

  Alternatively, when the number of white extraction blocks is less than a certain standard in either one of the strobe irradiation area and the strobe non-irradiation area, the area where the number of white extraction blocks is less than the certain standard has the other white extraction block number. You may use the white balance correction coefficient of the area | region beyond a fixed reference | standard.

When the number of white extraction blocks is less than a certain standard in both the strobe irradiation area and the strobe non-irradiation area, the white balance coefficient can be calculated by calculating based on the AWB evaluation value and the AE evaluation value in each block of the entire screen.
The white balance correction coefficient is preferably a correction coefficient that places importance on an area where the AE evaluation value is high, and can be calculated, for example, by weighting the AE evaluation value to the AWB evaluation value as shown in Equation 5 below. (S6; correction coefficient calculation means).

  Next, RAW-RGB is read from the SDRAM, and the pixel located at the center of the block as shown in FIG. 6 is used as the block center pixel, and the WB correction coefficients Rgain and Bgain calculated for each region are used as the block center pixel corresponding to each region. (S7; white balance calculating means). 6 in FIG. 6 is a block center pixel for setting white balance correction coefficients Rgain and Bgain calculated in the strobe irradiation area, and 4 in FIG. 6 is a block center pixel for setting white balance correction coefficients Rgain and Bgain calculated in the strobe non-irradiation area. It is.

  The white balance correction coefficients Rgain and Bgain of the non-block center pixel are preferably obtained by interpolation based on the white balance correction coefficients set in the block center pixel of the block and the surrounding block center pixels and the distance from the pixel (S8). ).

  A method for calculating the white balance correction coefficient of the non-block center pixel will be described with reference to FIG. The non-block center pixel that is the target of white balance correction coefficient calculation is 11, and the peripheral block center pixels are R0, R1, R2, and R3. R3 is normalized by 1 and the position of the non-block center pixel for which the white balance correction coefficient is to be calculated is represented by x and y, and the white balance correction coefficient is calculated by the following formula.

Rgain = (1-x) (1-y) R0 + x (1-y) R1 + (1-x) yR2 + xyR3
Bgain = (1-x) (1-y) B0 + x (1-y) B1 + (1-x) yB2 + xyB3

  In the case where there is only one peripheral pixel in the peripheral block like the 12-part pixel in FIG. 8, the white balance correction coefficient for the block is set.

  When there are only two peripheral block center pixels like the pixels 13, 14, 15, and 16 in FIG. 9, they are obtained by interpolation using two correction coefficients and the distance from the pixel. The calculation formula is shown below.

In the case of region 13, R _gain = (1-x) R2 + xR3
B _gain = (1-x) B2 + xB3

In the case of region 14, R _gain = (1-x) R0 + xR1
B _gain = (1-x) B0 + xB1

In the case of region 15, R _gain = (1-y) R1 + yR3
B _gain = (1-y) B1 + yB3

In the case of region 16, R _gain = (1-y) R0 + yR2
B _gain = (1-y) B0 + yB2

  The R and B data of the entire screen are multiplied by the white balance correction coefficients Rgain and Bgain calculated for each pixel (S9).

  Thereby, even when the color temperature of the light source irradiated in the strobe irradiation region and the strobe non-irradiation region is different, a good white balance can be applied to each.

  As a result, since the low-pass filter is applied to the correction coefficient between the pixels, it is possible to avoid an unnatural white balance being applied at the boundary between the areas in an image in which each area has a different white balance correction coefficient. .

  In addition, since the region is divided from the difference between the AE evaluation values in units of blocks, the amount of calculation can be reduced.

  Further, by using the monitoring image immediately before the flash photography, it is possible to divide the area without taking a plurality of pictures, and to shorten the time required for photography.

  Further, since the white balance correction coefficient is calculated for each region, the amount of calculation required for calculating the white balance correction coefficient can be reduced as compared with the method of calculating for each block or pixel.

  In the present embodiment, the captured image and the monitoring image are divided into 16 horizontal 16 × vertical blocks, but may be divided by an arbitrary number of divisions. In that case, when dividing the area, it is necessary to match the number of divisions of the captured image and the monitoring image for each same block in the captured image and the monitoring image, based on the AE evaluation value, that is, the strobe irradiation area or the strobe non-irradiation area. is there. In addition, the division is preferably in a lattice shape, but may be divided into any shape as long as the corresponding blocks in the captured image and the monitoring image have the same shape. The same applies to the embodiments described later.

[Second Embodiment]
In the first embodiment, the strobe irradiation area and the strobe non-irradiation area are divided in units of blocks, and the white balance correction coefficient is calculated for each area. In the present embodiment, the strobe light amount is calculated for each block. The white balance correction coefficient corresponding to the ratio of the strobe light amount is calculated for each block.

FIG. 10 is a flowchart showing a series of processing for strobe WB correction of the second embodiment of the imaging apparatus according to the present invention.
First, RAW-RGB at the time of photographing by flash emission is stored in the SDRAM (S1). The amount of stroboscopic light emission is adjusted by detecting the reflected light from the object obtained by the pre-emission before the main emission with the CCD.
The RAW-RGB stored in the SDRAM is divided into 256 blocks of horizontal 16 × vertical 16 by a detection circuit which is a part of the CCD I / F of the signal processing unit which is a block dividing means.
At the same time, an AWB evaluation value (white balance evaluation value), an AE evaluation value (luminance signal integration result) and an AE control value (shutter speed, aperture value, AGC gain value, ND filter position) at the time of photographing are stored in the internal RAM. Stored (S2). The AWB evaluation value is expressed in the form of G / R and G / B after calculating R, G, and B per pixel by integrating R, G, and B for each block. The AE evaluation value is expressed as a result of the following equation after calculating R, G, and B per pixel by integrating R, G, and B for each block.

      AE evaluation value = 0.299R + 0.587G + 0.114B

  Next, an exposure correction coefficient (k) calculated by a calculation method to be described later is used for each block corresponding to the monitoring image so that the exposure amount at the time of monitoring immediately before shooting is the same as the exposure amount at the time of shooting. An evaluation value and / or an AE evaluation value for each block at the time of shooting is set (S3; exposure correction coefficient calculation means). The AE evaluation value corresponding to the monitoring image (first image pickup signal) immediately before shooting is stored in the RAM, which is calculated from the last frame of the release using the same calculation formula as when shooting. As in the case of shooting, the AE control values (shutter speed, aperture value, AGC gain value, ND filter position, pixel addition amount) at this time are also stored in the internal RAM.

  In addition, when adjusting the exposure amount, there is no change even if the exposure compensation coefficient is multiplied to the block with no light quantity, and it becomes difficult to calculate the strobe contribution ratio by the difference. Therefore, the exposure is adjusted to match the exposure amount of the image with a small exposure amount. It is preferable to multiply an image having a large amount by an exposure correction coefficient (k).

In addition, when the exposure correction coefficient (k) is multiplied to an image with a small exposure amount so as to match the exposure amount of an image with a large exposure amount, the ratio of noise increases with an AE evaluation value less than a certain value, and therefore the lower 1 EV or less. It is preferable to weight the AE evaluation value block so that the exposure correction coefficient (k) calculated by the calculation method described later changes linearly in conjunction with the AE evaluation value.
Specifically, for example, it can be calculated using an operation for weighting the AE evaluation value with respect to the exposure correction coefficient (k) as shown in the following equation.

      k ′ = k × (AE evaluation value / 1 AE evaluation value for 1 EV)

  Further, when the shading correction parameters are different between the monitoring image and the captured image, it is preferable to add the difference between the shading correction parameters of the monitoring image and the captured image to the exposure correction coefficient (k ′) of each block.

Although the example of the AE control value in this Embodiment is shown in FIG. 4, the AE control value in this invention is not limited to this.
Since the amount of change in brightness per step of each AE control value is the same, the difference in brightness between shooting and monitoring (delta Ev) by subtraction from the addition result of each AE control value during shooting and monitoring. Can be requested. In the example of the AE control value in the present embodiment, the difference in control value is 16 and 1 EV difference, that is, the brightness is twice different. The calculation method of the exposure correction coefficient (k) is shown in the following formula 1.

  When the AE evaluation value is calculated from the result of adding the horizontal and vertical pixels at the time of monitoring, it is necessary to consider the pixel addition when calculating the exposure correction coefficient. The calculation method of the exposure correction coefficient (k) Become. For example, when the result of summing two horizontal pixels and two vertical pixels (for four pixels) is read out from the CCD during monitoring, 2 EV (4 pixels during monitoring) compared to shooting (for one pixel) without pixel addition. Times) will be brighter.

Next, the stroboscopic light quantity for each block is calculated by subtracting the AE evaluation value of the captured image of the same block and the AE evaluation value of the monitoring image immediately before shooting (S4).
The determination method of the strobe irradiation area in the present invention is not limited to the determination method of S4, and any determination method can be used as long as it is a calculation method that can determine a block that is sufficiently irradiated with strobe light. The reference value set in advance can be arbitrarily set.

  With S1 to S4, it is possible to calculate the stroboscopic light amount for each block even when the exposure amount differs between shooting and monitoring.

  Further, for each block, the strobe contribution ratio in the block is calculated from the ratio between the external light amount and the strobe light amount. The external light quantity refers to the AE evaluation value of the monitoring image after exposure correction. The strobe contribution rate of each block indicates the ratio of the strobe light amount when the sum of the external light amount and the strobe light amount is taken as 100%.

  Next, white extraction is performed (S5; white detection means). For white extraction, a block below the threshold is stored as a white extraction block for the AWB evaluation value acquired in units of blocks in the entire screen, or a block below the threshold is extracted from among blocks with a strobe contribution ratio below a certain value. Store as a block.

When the number of white extraction blocks is equal to or greater than a certain standard, the first white balance correction coefficient can be calculated by calculating based on the AWB evaluation value and the AE evaluation value in each white extraction block.
The first white balance correction coefficient is preferably a correction coefficient that places importance on an area where the AE evaluation value is high. For example, as shown in the following Equation 3, the AWB evaluation value is weighted with the AE evaluation value. It can be calculated.

Furthermore, by calculating based on the first white balance coefficient and the second white balance coefficient that can be corrected to a good white balance when the effect of the strobe is 100% according to the strobe contribution rate, A third white balance correction coefficient can be calculated.
The third white balance correction coefficient is preferably a correction coefficient such that a block having a high strobe contribution ratio places more importance on the second white balance coefficient than a block having a low strobe contribution ratio. In addition, a weighted average of the first white balance coefficient and the second white balance coefficient can be calculated for each block.

When the number of white extraction blocks is less than a certain standard, the first white balance correction coefficient can be calculated by calculating the entire screen based on the AWB evaluation value and the AE evaluation value of each block.
The first white balance correction coefficient is preferably a correction coefficient that places importance on a region with a high AE evaluation value. For example, as shown in the following Equation 4, the AWB evaluation value is weighted with the AE evaluation value. It can be calculated.

  Alternatively, the white balance correction coefficient set in the monitoring image immediately before shooting is set as the first white balance correction coefficient.

Furthermore, by calculating based on the first white balance coefficient and the second white balance coefficient that can be corrected to a good white balance when the effect of the strobe is 100% according to the strobe contribution rate, A third white balance correction coefficient can be calculated (S6).
The third white balance correction coefficient is preferably a correction coefficient such that a block having a high strobe contribution ratio places more importance on the second white balance coefficient than a block having a low strobe contribution ratio. In addition, the weighted average of the first white balance coefficient and the second white balance coefficient can be calculated for each block (S6; first to third white balance correction coefficient calculating means).

  Here, the second white balance correction coefficient may be set in advance as a predetermined value as a correction coefficient that can be corrected to a good white balance when the effect of the strobe is 100%, but the strobe is emitted. If the change in the color temperature of the strobe light is large due to the voltage at the time, the WB when emitting light at the lower limit voltage and the WB when emitting light at the upper limit voltage are held as internal data in advance, and the voltage at the time of strobe emission From this, it is preferable to calculate the second white balance correction coefficient by linear interpolation.

  Next, RAW-RGB is read from the SDRAM, and the third white balance correction coefficients Rgain and Bgain calculated for each block are set as the center pixel of the corresponding block, with the pixel located at the center of the block as the block center pixel as shown in FIG. (S7; white balance setting means).

  Thereafter, the method for setting the white balance correction coefficient for each block (S8 to S9 in FIG. 10) is the same as that in the first embodiment (S8 to S9 in FIG. 3).

According to the present embodiment, it is possible to set a fine white balance correction coefficient corresponding to the effect of the strobe for each block.
Further, even when the color temperature of the light source irradiated in the strobe irradiation region and the strobe non-irradiation region is different, a good white balance can be applied to each.
Furthermore, since a low-pass filter is applied to the correction coefficient between pixels, it is possible to avoid an unnatural white balance being applied at the boundary between the areas in an image in which there are areas having different white balance correction coefficients.
In addition, since the region is divided from the difference between the AE evaluation values in units of blocks, the amount of calculation can be reduced.
Further, by using the monitoring image immediately before the flash photography, it is possible to divide the area without taking a plurality of pictures, and to shorten the time required for photography.

  In this embodiment, the captured image and the monitoring image are divided into 16 horizontal 16 × vertical 16 blocks, but may be divided by an arbitrary number of divisions.

[Third Embodiment]
In the first embodiment, region division is performed based on the difference between the AE evaluation values of each block at the time of shooting and at the time of monitoring. In this embodiment, the shot image after image processing (WB, gamma correction, color correction, etc.) is performed. Then, the area is divided in units of pixels based on the luminance of the monitoring image immediately before shooting.

  FIG. 12 is a flowchart showing a series of processing for strobe WB correction in the third embodiment of the imaging apparatus according to the present invention.

  First, RAW-RGB and YUV at the time of shooting by strobe light emission are stored in the SDRAM (S1). In addition, the amount of stroboscopic light emission is adjusted by detecting reflected light from the object obtained by pre-emission before the main emission with the CCD.

  At the same time, the AWB evaluation value and the AE control value (shutter speed, aperture value, AGC gain value, ND filter position) at the time of shooting are stored in the internal RAM (S2).

  Next, the captured image is read out from the SDRAM, and subjected to resizing processing so as to have the same size as the monitoring image is stored in an SDRAM area different from the captured image before resizing (S3).

  De-gamma processing is performed on YUV during monitoring and immediately before shooting (S4). Since YUV for monitoring at the time of shooting and immediately before shooting is data after gamma correction, the output is linear with respect to the input and does not change. Usually, gamma has a characteristic that it is lifted from low luminance to medium luminance to make the gradation change abrupt and make the gradation on the high luminance side gentle. FIG. 11 shows the gamma characteristic. Because of this characteristic, the output changes greatly with a slight change on the low-brightness side, so when dividing the area by the luminance difference for each pixel of the captured image and the monitoring image immediately before shooting, obtain the exact degree of strobe impact I can't. Therefore, a process of returning to a linear value by degamma is performed. If the gamma table used for shooting is different from that for monitoring, de-gamma processing corresponding to the used gamma table is performed.

  Next, luminance information in pixel units of the captured image and the monitoring image immediately before capturing is calculated (S5). The calculation method of luminance information is shown below.

      Luminance information = 0.299R + 0.587G + 0.114B

  Next, the exposure correction coefficient (k) is set to the luminance information of each pixel of the monitoring image and / or the captured image so that the exposure amount of the monitoring image (first imaging signal) immediately before capturing is the same as the exposure amount of the captured image. (S6; exposure correction coefficient calculation means). The method for calculating the exposure correction coefficient is the same as that in the first embodiment. The monitoring image is subjected to image processing through the CCD I / F and is overwritten and saved in the SDRAM as needed as YUV. Also, the AE control values (shutter speed, aperture value, AGC gain value, ND filter position, pixel addition amount) at the time of monitoring immediately before shooting are stored in the internal RAM.

  In the same pixel, the luminance information of the monitoring image is subtracted from the luminance information of the photographed image, and compared with a preset reference value. Is divided (S7). Furthermore, based on the result of area division in units of pixels, the area is divided in units of blocks composed of a plurality of pixels (S8; area dividing means).

  FIG. 5 shows an area divided into a strobe irradiation area and a strobe non-irradiation area in block units. 5 in FIG. 5 represents a strobe irradiation region, and 2 in FIG. 5 represents a strobe non-irradiation region.

  The division of the block is associated with the one set when acquiring the RGB integrated value for calculating the AWB evaluation value and the AE evaluation value, and it is determined whether the block is a strobe irradiation area or a non-strobe irradiation area. Is determined by the ratio of pixels in the strobe irradiation area included in the.

  Next, white extraction is performed for each region (S9; white detection means). In the white extraction, a block equal to or lower than the threshold is stored as a white extraction block for the AWB evaluation value acquired in units of blocks for each region.

When the number of white extraction blocks is equal to or greater than a certain standard in both the strobe irradiation area and the strobe non-irradiation area, the AWB evaluation value in each white extraction block for each of the strobe irradiation area and the strobe non-irradiation area By calculating based on the AE evaluation value, the white balance correction coefficient can be calculated.
The white balance correction coefficient is preferably a correction coefficient that places importance on an area where the AE evaluation value is high, and can be calculated, for example, by weighting the AE evaluation value to the AWB evaluation value as shown in Equation 3 below. .

In addition, when the number of white extraction blocks is less than a certain reference in either one of the strobe irradiation area and the strobe non-irradiation area, the area where the number of white extraction blocks is less than a certain reference is the AWB evaluation value in each block. By calculating based on the AE evaluation value, the white balance correction coefficient can be calculated.
The white balance correction coefficient is preferably a correction coefficient that places importance on an area where the AE evaluation value is high, and can be calculated, for example, by weighting the AE evaluation value to the AWB evaluation value as shown in Equation 4 below. .

  Alternatively, when the number of white extraction blocks is less than a certain standard in either one of the strobe irradiation area and the strobe non-irradiation area, the area where the number of white extraction blocks is less than the certain standard has the other white extraction block number. You may use the white balance correction coefficient of the area | region beyond a fixed reference | standard.

When the number of white extraction blocks is less than a certain standard in both the strobe irradiation area and the strobe non-irradiation area, the white balance coefficient can be calculated by calculating based on the AWB evaluation value and the AE evaluation value in each block of the entire screen.
The white balance correction coefficient is preferably a correction coefficient that places importance on an area where the AE evaluation value is high, and can be calculated, for example, by weighting the AE evaluation value to the AWB evaluation value as shown in Equation 5 below. (S10: correction coefficient calculation means).

  As shown in FIG. 6, the pixel located at the center of the block is set as the block center pixel, and the WB correction coefficients Rgain and Bgain calculated for each region are set to the block center pixel corresponding to each region (S11; white balance calculating means). 6 in FIG. 6 is a block center pixel for setting white balance correction coefficients Rgain and Bgain calculated in the strobe irradiation area, and 4 in FIG. 6 is a block center pixel for setting Rgain and Bgain calculated in the strobe non-irradiation area.

  Thereafter, the method for setting the WB correction coefficient for each block (S12 to S13 in FIG. 12) is the same as that in the first embodiment (S8 to S9 in FIG. 3).

According to this embodiment, even when the color temperatures of the light sources irradiated in the strobe irradiation area and the strobe non-irradiation area are different, a good white balance can be applied to each.
In addition, since a low-pass filter is applied to the correction coefficient between pixels, it is possible to avoid an unnatural white balance being applied at the boundary between areas in an image in which there are areas having different WB correction coefficients.
Further, by using the monitoring image immediately before the flash photography, it is possible to divide the area without taking a plurality of pictures, and to shorten the time required for photography.
In addition, since the WB correction coefficient is calculated for each region, the amount of calculation required for calculating the WB correction coefficient can be reduced as compared with the method of calculating for each block or pixel.
Furthermore, even when a strobe irradiation area and a strobe non-irradiation area are mixed in a block in a block, it is possible to finely divide the area for each block according to the ratio of the area division result for each pixel in the block.

  In this embodiment, the captured image and the monitoring image are divided into 16 horizontal 16 × vertical 16 blocks, but may be divided by an arbitrary number of divisions. In that case, when dividing the area, it is necessary to match the number of divisions of the captured image and the monitoring image for each block of the captured image and the monitoring image in order to determine whether it is a stroboscopic irradiation area or a non-stroboscopic irradiation area.

[Fourth Embodiment]
In the third embodiment, a stroboscopic irradiation area and a non-stroboscopic irradiation area are divided on a pixel basis, and the area is further divided on a block basis in accordance with the ratio of the stroboscopic irradiation area in the block. In the present embodiment, the strobe light amount is calculated for each pixel, the area is further divided into blocks according to the strobe light amount in the block, and the WB corresponding to the strobe light amount ratio for each block is calculated. A correction coefficient is calculated.

FIG. 13 is a flowchart showing a series of processing for strobe WB correction according to this embodiment.
First, RAW-RGB and YUV at the time of photographing by strobe light emission are stored in the SDRAM (S1). In addition, the amount of stroboscopic light emission is adjusted by detecting reflected light from the object obtained by pre-emission before the main emission with the CCD.

  At the same time, the AWB evaluation value and the AE control value (shutter speed, aperture value, AGC gain value, ND filter position) at the time of shooting are stored in the internal RAM (S2).

  Next, the captured image is read out from the SDRAM, and subjected to resizing processing so as to have the same size as the monitoring image is stored in an SDRAM area different from the captured image before resizing (S3).

  De-gamma processing is performed on YUV during monitoring and immediately before shooting (S4). Since YUV for monitoring at the time of shooting and immediately before shooting is data after gamma correction, the output is linear with respect to the input and does not change. Usually, gamma has a characteristic that it is lifted from low luminance to medium luminance to make the gradation change abrupt and make the gradation on the high luminance side gentle. FIG. 11 shows the gamma characteristic. Because of this characteristic, the output changes greatly with a slight change on the low-brightness side, so when dividing the area by the luminance difference for each pixel of the captured image and the monitoring image immediately before shooting, obtain the exact degree of strobe impact I can't. Therefore, a process of returning to a linear value by degamma is performed. If the gamma table used for shooting is different from that for monitoring, de-gamma processing corresponding to the used gamma table is performed.

  Next, luminance information in units of pixels of the captured image and the monitoring image immediately before capturing is calculated (S5). The calculation method of luminance information is shown below.

      Luminance information = 0.299R + 0.587G + 0.114B

  Next, the exposure correction coefficient (k) is set to the luminance information of each pixel of the monitoring image and / or the captured image so that the exposure amount of the monitoring image (first imaging signal) immediately before capturing is the same as the exposure amount of the captured image. (S6; exposure correction coefficient calculation means). The method for calculating the exposure correction coefficient is the same as that in the first embodiment. The monitoring image is subjected to image processing through the CCD I / F and is overwritten and saved in the SDRAM as needed as YUV. Also, the AE control values (shutter speed, aperture value, AGC gain value, ND filter position, pixel addition amount) at the time of monitoring immediately before shooting are stored in the internal RAM.

  Also, when adjusting the exposure amount, there is no change even if the pixel with no light amount is multiplied by the exposure correction coefficient, and the division method that divides according to the size of the difference makes division difficult. As described above, it is preferable to multiply the image having a large exposure amount by the exposure correction coefficient (k).

  Further, when the shading correction parameters are different between the monitoring image and the captured image, it is preferable to add the difference between the shading correction parameters of the monitoring image and the captured image to the exposure correction coefficient of each pixel.

  Next, in the same pixel, the stroboscopic light amount in units of pixels is calculated by subtracting the luminance information of the captured image and the luminance information of the monitoring image (S7).

Further, the area is divided into blocks each composed of a plurality of pixels (S8; area dividing means). The division of the block is associated with what is set when the RGB integrated value for calculating the AWB evaluation value and the AE evaluation value is acquired.
Then, the strobe light amount for each block is calculated based on the strobe light amount for each pixel (S9; strobe light amount calculating means).

  The strobe contribution ratio of each block is calculated from the ratio between the external light amount and the strobe light amount in each block. The external light amount indicates the AE evaluation value of the monitoring image after exposure correction. The strobe contribution ratio of each block indicates the ratio of the strobe light amount when the sum of the external light amount and the strobe light amount is taken as 100%.

  Next, white extraction is performed (S10; white detection means). In the white extraction, for the AWB evaluation value acquired in units of blocks from the entire screen, a block equal to or lower than the threshold is stored as a white extraction block compared to the threshold, or a threshold is selected from among the blocks having a strobe contribution ratio equal to or less than a certain value. The block below the threshold value is stored as a white extraction block.

When the number of white extraction blocks is equal to or greater than a certain standard, the first white balance correction coefficient can be calculated by calculating based on the AWB evaluation value and the AE evaluation value in each white extraction block.
The first white balance correction coefficient is preferably a correction coefficient that places importance on an area where the AE evaluation value is high. For example, as shown in the following Equation 3, the AWB evaluation value is weighted with the AE evaluation value. It can be calculated.

Furthermore, by calculating based on the first white balance coefficient and the second white balance coefficient that can be corrected to a good white balance when the effect of the strobe is 100% according to the strobe contribution rate, A third white balance correction coefficient can be calculated.
The third white balance correction coefficient is preferably a correction coefficient such that a block having a high strobe contribution ratio places more importance on the second white balance coefficient than a block having a low strobe contribution ratio. In addition, a weighted average of the first white balance coefficient and the second white balance coefficient can be calculated for each block.

When the number of white extraction blocks is less than a certain standard, the first white balance correction coefficient can be calculated by calculating the entire screen based on the AWB evaluation value and the AE evaluation value of each block.
The first white balance correction coefficient is preferably a correction coefficient that places importance on a region with a high AE evaluation value. For example, as shown in the following Equation 4, the AWB evaluation value is weighted with the AE evaluation value. It can be calculated.

  Alternatively, the white balance correction coefficient set in the monitoring image immediately before shooting is set as the first white balance correction coefficient.

Furthermore, by calculating based on the first white balance coefficient and the second white balance coefficient that can be corrected to a good white balance when the effect of the strobe is 100% according to the strobe contribution rate, A third white balance correction coefficient can be calculated (S6).
The third white balance correction coefficient is preferably a correction coefficient such that a block having a high strobe contribution ratio places more importance on the second white balance coefficient than a block having a low strobe contribution ratio. In addition, the weighted average of the first white balance coefficient and the second white balance coefficient can be calculated for each block (S11; first to third white balance correction coefficient calculating means).

  Here, the second white balance correction coefficient may be set in advance as a predetermined value as a correction coefficient that can be corrected to a good white balance when the effect of the strobe is 100%, but the strobe is emitted. If the change in the color temperature of the strobe light is large due to the voltage at the time, the WB when emitting light at the lower limit voltage and the WB when emitting light at the upper limit voltage are held as internal data in advance, and the voltage at the time of strobe emission From this, it is preferable to calculate the second white balance correction coefficient by linear interpolation.

  Next, RAW-RGB is read from the SDRAM, and the third white balance correction coefficients Rgain and Bgain calculated for each block are set as the center pixel of the corresponding block, with the pixel located at the center of the block as the block center pixel as shown in FIG. (S12; white balance calculating means).

  Thereafter, the method for setting the WB correction coefficient for each block (S13 to S14 in FIG. 13) is the same as that in the first embodiment (S8 to S9 in FIG. 3).

According to the present embodiment, it is possible to set a fine WB correction coefficient corresponding to the influence of the strobe for each block.
Further, even when the color temperature of the light source irradiated in the strobe irradiation area and the strobe non-irradiation area is different, good white balance can be applied to each.
Furthermore, since a low-pass filter is applied to the correction coefficient between pixels, it is possible to avoid an unnatural white balance being applied at the boundary between areas in an image in which there are areas having different WB correction coefficients.
Furthermore, by using the monitoring image immediately before the flash photography, it is possible to divide the area without taking a plurality of pictures, and to shorten the time required for photography.

  In this embodiment, the captured image and the monitoring image are divided into 16 horizontal 16 × vertical 16 blocks, but may be divided by an arbitrary number of divisions.

[Fifth Embodiment]
FIG. 14 shows a flow of a series of processing for strobe WB correction according to the fifth embodiment.
First, a captured image (second imaging signal) by strobe light emission is stored in the SDRAM as YCC data (memory storage means). At this time, the WB gain, gamma table, and color correction parameters set for the monitoring image until immediately before are applied to the captured image (S1). In addition, the amount of stroboscopic light emission is adjusted by detecting reflected light from the object obtained by pre-emission before the main emission with the CCD.

  Next, the captured image is read from the SDRAM and subjected to resizing processing so as to have the same size as the monitoring image is stored in an SDRAM area different from the captured image before resizing (S2).

  The resized photographed image is divided into a horizontal 16 × vertical 16 block in a grid pattern (block dividing means), and an integrated value of Y signals is calculated for each block (luminance signal calculating means) (S3). ).

Further, an exposure correction coefficient (k) is set for the Y signal of each pixel of the monitoring image so that the same exposure condition as that of the captured image is set for the monitoring image (first imaging signal) immediately before shooting (S4). The monitoring image is subjected to image processing through the CCD I / F, and is overwritten and saved in the SDRAM as needed as YCC data (memory storage means).
The exposure correction coefficient (k) is calculated by the following formula. The shutter speed, aperture value, and ND filter position of the monitoring image at the time of shooting and immediately before shooting are the same.

  k = stillAGC / monAGC

  Here, the still AGC represents a captured image AGC, and the mon AGC represents a monitoring image AGC.

  Next, the monitoring image after exposure correction is read out from the SDRAM, and the entire monitoring image is divided into 16 horizontal 16 × vertical 16 blocks (block dividing means), and an integrated value of Y signals is calculated for each block (luminance signal) Integrating means) (S5).

  Furthermore, the Y signal integrated value of the monitoring image is subtracted from the Y signal integrated value of the captured image of the same block and compared with a preset reference value. Region division (strobe light amount calculation means, region division means) is performed with the block No. as a non-flash irradiation area (S6).

  FIG. 5 shows an area divided into a strobe irradiation area and a strobe non-irradiation area in block units. 5 in FIG. 5 represents a strobe irradiation region, and 2 in FIG. 5 represents a strobe non-irradiation region.

  Next, in order to obtain an AWB evaluation value for each area, RGB of a plurality of blocks corresponding to each area are integrated (S7).

  Further, white extraction (white detection means) is performed for each region (S8). In the white extraction, a block equal to or lower than the threshold is stored as a white extraction block for the AWB evaluation value acquired in units of blocks for each region.

  Here, when the number of white extraction blocks is greater than a certain standard for each region, the WB correction coefficient is calculated by weighting the AWB evaluation value of the white extraction block and the average luminance value of the block. In addition, if it is not above a certain standard, the WB correction coefficient is calculated by weighting the AWB evaluation value of all blocks in the area and the average luminance value of the block, or the WB correction coefficient calculated in an area other than the area is (S9). Here, S9 is a correction coefficient calculation means in the present invention.

  When there is no white extraction block that satisfies a certain standard in any area, the WB correction coefficient is calculated by weighting the AWB evaluation value of each block and the average luminance value of the block over the entire screen.

As shown in FIG. 6, the pixel located at the center of the block is set as the block center pixel, and the WB correction coefficients (first correction coefficients) R _gain and B _gain calculated for each area are set to the block center pixels corresponding to the respective areas. (S10). The white circle arranged at the center of the region in FIG. 6 is the WB correction coefficient R _gain calculated in the stroboscopic irradiation region, the central pixel 3 of the stroboscopic irradiation region block for setting B _gain , and the black circle arranged at the center of the region in FIG. This is the strobe non-irradiation area block center pixel 4 for setting R _gain and B _gain calculated for the area.

The WB correction coefficient (second correction coefficient) R _gain and B _gain of the non-block center pixel are the WB correction coefficient (first correction coefficient) set for the block center pixel and the peripheral block center pixel of the block, It is obtained by interpolation based on the distance from the pixel (S11). Here, S10 and S11 are white balance calculation means in the present invention.

  Based on the explanatory diagram of the calculation method of the WB correction coefficient by interpolation shown in FIG. 7, the WB correction coefficient calculation method of the non-block center pixel will be described. The calculation target of the WB correction coefficient for calculating the WB correction coefficient by interpolation is the non-block central pixel 11, and the block central pixels around the non-block central pixel 11 are R0, R1, R2, and R3, respectively. R3 is normalized by 1, and the position of the non-block center pixel for which the WB correction coefficient is to be calculated is represented by x and y, and the WB correction coefficient is calculated by the following equation.

R _gain = (1-x) (1-y) R0 + x (1-y) R1 + (1-x) yR2 + xyR3
B _gain = (1-x) (1-y) B0 + x (1-y) B1 + (1-x) yB2 + xyB3

  When there is only one peripheral block center pixel like the 12-part pixel in FIG. 8, the WB correction coefficient of the block center pixel is set as the 12-part WB correction coefficient (second correction coefficient).

  When there are only two peripheral block center pixels as in the pixels of the 13, 14, 15, and 16 portions in FIG. 9, the interpolation is based on the two correction coefficients of the two block center pixels and the distance from the pixel. A WB correction coefficient (second correction coefficient) is obtained. The calculation formula is shown in the following formula.

In the case of region 13, R _gain = (1-x) R2 + xR3
B _gain = (1-x) B2 + xB3

In the case of region 14, R _gain = (1-x) R0 + xR1
B _gain = (1-x) B0 + xB1

In the case of region 15, R _gain = (1-y) R1 + yR3
B _gain = (1-y) B1 + yB3

In the case of region 16, R _gain = (1-y) R0 + yR2
B _gain = (1-y) B0 + yB2

The R and B data of the entire screen are multiplied by WB correction coefficients R _gain and B _gain calculated for each pixel (S12).

(Periphery center pixel)
Further, the peripheral central pixel in the present invention refers to a central pixel arranged in the periphery. For example, a line segment connecting the peripheral central pixel and the pixel for obtaining the second correction coefficient is all center. This is the central pixel that does not intersect the line segment that connects the pixels in a grid pattern.
However, the line segment connecting the center pixel and the pixel for obtaining the second correction coefficient is not longer than the length of the diagonal line segment of one block.

  In addition, if the peripheral center pixel is located on any one of the line segments that connect all the central pixels in a lattice shape, the pixel for acquiring the second correction coefficient is the line segment and the central pixel. Of the quadrangles formed by the line segments that connect each other, it is preferable that the four central pixels form the smallest quadrilateral.

  On the other hand, if the peripheral center pixel is located on any one of the line segments that connect all the central pixels in a grid pattern, both ends of the line segment are obtained. It may be two central pixels arranged in the.

  According to this embodiment, even when the color temperatures of the light sources irradiated in the strobe irradiation area and the strobe non-irradiation area are different, a good white balance can be applied to each. In addition, since a low-pass filter is applied to the correction coefficient between pixels, it is possible to avoid an unnatural white balance being applied at the boundary between areas in an image in which there are areas having different WB correction coefficients.

  Further, by using the monitoring image immediately before the flash photography, it is possible to divide the area without taking a plurality of pictures, and to shorten the time required for photography.

  Furthermore, since the WB correction coefficient is calculated for each region, the amount of calculation required for calculating the WB correction coefficient can be reduced as compared with the method of calculating for each block or pixel.

  In the present embodiment, the captured image and the monitoring image are divided into horizontal 16 × vertical 16 blocks, but may be divided into any number of divisions. In such a case, the number of divisions of the captured image and the monitoring image is determined for each block of the captured image and the monitoring image at the time of dividing the region, in order to determine whether it is a strobe irradiation region or a strobe non-irradiation region from the integrated value of Y signals. It is necessary to match.

1 is a schematic diagram showing an appearance of a digital camera that is a first embodiment of an imaging apparatus according to the present invention. 1 is a block diagram illustrating a configuration in a first embodiment of an imaging apparatus according to the present invention. It is a block diagram which shows the flow of a series of processes of strobe WB correction | amendment which concerns on 1st Embodiment. It is a figure which shows an example of AE control value. FIG. 2 is a schematic diagram in which an imaging signal in the first embodiment of the imaging apparatus according to the present invention is divided into grid-like blocks, and an explanatory diagram illustrating a strobe irradiation region and a strobe non-irradiation region in the schematic diagram. It is explanatory drawing explaining the strobe irradiation area block center pixel and strobe non-irradiation area block center pixel in the schematic diagram of FIG. It is explanatory drawing of the calculation method of the WB correction coefficient by interpolation. It is explanatory drawing explaining the area | region 12 in the schematic diagram of FIG. It is explanatory drawing explaining the area | regions 13, 14, 15, and 16 in the schematic diagram of FIG. It is a flowchart of a series of processes of strobe WB correction in the second embodiment of the imaging apparatus according to the present invention. It is a graph which shows a gamma characteristic. It is a flowchart of a series of processes of strobe WB correction in the third embodiment of the imaging apparatus according to the present invention. It is a flowchart of a series of processes of strobe WB correction in the fourth embodiment of the imaging apparatus according to the present invention. It is a flowchart of a series of processes of strobe WB correction in the fifth embodiment of the imaging device according to the present invention.

Explanation of symbols

1 Strobe irradiation area 2 Strobe non-irradiation area 3 Strobe irradiation area block central pixel 4 Strobe non-irradiation area block central pixel 12, 13, 14, 15, 16 area

Claims (9)

An image sensor that converts light incident from the optical system into an electrical signal and outputs the signal as an image signal;
Imaging means for storing the imaging signal in a memory;
In an imaging apparatus comprising: a white balance evaluation value acquisition unit that extracts a white balance evaluation value from the imaging signal;
The imaging means stores a first imaging function for storing an imaging signal without flash emission in the memory as a first imaging signal;
A second imaging function for storing an imaging signal with strobe light emission in the memory as a second imaging signal;
Block dividing means for dividing the first imaging signal and the second imaging signal into a plurality of blocks;
A luminance signal integrating means for integrating the luminance signal for each of the divided blocks and calculating a luminance integration result;
In the luminance integration result of the block of the first imaging signal and the luminance integration result of the block of the second imaging signal, a calculated value calculated by calculating each of the blocks at the same position,
A region dividing means for dividing the region by comparing with a predetermined reference value;
Correction coefficient calculation means for calculating a correction coefficient for white balance for each area based on the white balance evaluation value for each of the divided areas;
White balance calculation means for setting a correction coefficient for white balance for each of the divided areas ,
The exposure correction coefficient so that the exposure amount is the same between the first image pickup signal and the second image pickup signal is set to a shutter speed, an aperture value, an AGC gain value, and an ND filter at the time of image pickup by the first image pickup function. Calculated from the position, pixel addition, shutter speed, aperture value, AGC gain value, and ND filter position at the time of imaging by the second imaging function, the luminance integration result for each block of the first imaging signal, and An image pickup apparatus comprising: an exposure correction coefficient calculating unit that sets the luminance integration result for each block of the second image pickup signal . White detection means for performing white detection based on the white balance evaluation value and a preset white detection range;
The correction coefficient calculation means calculates a white balance correction coefficient for each area based on the white detection result detected by the white detection means and the white balance evaluation value for each of the divided areas. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. The exposure correction coefficient calculation means is configured to calculate an exposure correction coefficient calculated in the block of the first imaging signal block and / or the second imaging signal block for which the luminance signal is a predetermined value or less. The imaging apparatus according to claim 1 , wherein weighting is performed on the image pickup apparatus. A photoelectric conversion step of converting light incident from the optical system into an electrical signal by an image sensor and outputting it as an image signal;
An imaging step of storing the imaging signal in a memory;
A white balance evaluation value acquisition step of extracting a white balance evaluation value from the imaging signal, and an imaging method comprising:
The imaging step includes a first imaging step of storing an imaging signal without strobe light emission in the memory as a first imaging signal;
A second imaging step of storing an imaging signal with strobe light emission in the memory as a second imaging signal;
A block dividing step of dividing the first imaging signal and the second imaging signal into a plurality of blocks;
A luminance signal integration step of calculating the luminance integration result by integrating the luminance signal for each of the divided blocks;
In the luminance integration result of the block of the first imaging signal and the luminance integration result of the block of the second imaging signal, a calculated value calculated by calculating each of the blocks at the same position,
A region dividing step of dividing the region by comparing with a predetermined reference value;
A correction coefficient calculating step of calculating a correction coefficient for white balance for each area based on the white balance evaluation value for each of the divided areas;
A white balance calculation step of setting a white balance correction coefficient for each of the divided areas ,
The exposure correction coefficient so that the exposure amount is the same between the first image pickup signal and the second image pickup signal is set to a shutter speed, an aperture value, an AGC gain value, and an ND filter at the time of image pickup by the first image pickup function. Calculated from the position, pixel addition, shutter speed, aperture value, AGC gain value, and ND filter position at the time of imaging by the second imaging function, the luminance integration result for each block of the first imaging signal, and An imaging method comprising: an exposure correction coefficient calculation step for setting a luminance integration result for each block of the second imaging signal . A white detection step of performing white detection based on the white balance evaluation value and a preset white detection range;
The correction coefficient calculating step calculates a white balance correction coefficient for each region based on the white detection result detected by the white detection step and the white balance evaluation value for each of the divided regions. The imaging method according to claim 4 , wherein: In the exposure correction coefficient calculation step, among the blocks of the first image pickup signal and / or the blocks of the second image pickup signal, a block whose luminance signal is a predetermined value or less is an exposure correction coefficient calculated in the block. The imaging method according to claim 4 , wherein weighting is performed on. In a computer provided with an apparatus having an image processing function,
Photoelectric conversion processing for converting the light incident from the optical system into an electrical signal by the imaging device and outputting it as an imaging signal;
Imaging processing for storing the imaging signal in a memory;
A white balance evaluation value acquisition process for extracting a white balance evaluation value from the imaging signal, and a computer-readable recording medium storing a program for executing the program,
The imaging process includes a first imaging process for storing an imaging signal without strobe light emission in the memory as a first imaging signal;
A second imaging process of storing an imaging signal with strobe light emission in the memory as a second imaging signal;
Block division processing for dividing the first imaging signal and the second imaging signal into a plurality of blocks;
Luminance signal integration processing for integrating the luminance signal for each of the divided blocks and calculating the luminance integration result;
In the luminance integration result of the block of the first imaging signal and the luminance integration result of the block of the second imaging signal, a calculated value calculated by calculating each of the blocks at the same position,
An area dividing process for dividing the area by comparing with a predetermined reference value;
A correction coefficient calculation process for calculating a correction coefficient for white balance for each area based on the white balance evaluation value for each of the divided areas;
A white balance calculation process for setting a correction coefficient for white balance for each of the divided areas ,
The exposure correction coefficient so that the exposure amount is the same between the first image pickup signal and the second image pickup signal is set to a shutter speed, an aperture value, an AGC gain value, and an ND filter at the time of image pickup by the first image pickup function. Calculated from the position, pixel addition, shutter speed, aperture value, AGC gain value, and ND filter position at the time of imaging by the second imaging function, the luminance integration result for each block of the first imaging signal, and A computer-readable recording medium storing a program for executing a white balance calculation process, comprising: an exposure correction coefficient calculation process set to a luminance integration result for each block of the second imaging signal . A white detection process for performing white detection based on the white balance evaluation value and a preset white detection range;
The correction coefficient calculation process calculates a white balance correction coefficient for each area based on the white detection result detected by the white detection process and the white balance evaluation value for each of the divided areas. A computer-readable recording medium having recorded thereon a program for executing the white balance calculation process according to claim 7 . In the exposure correction coefficient calculation process, among the blocks of the first image pickup signal and / or the blocks of the second image pickup signal, a block whose luminance signal is a predetermined value or less is an exposure correction coefficient calculated in the block. The computer-readable recording medium which recorded the program for performing the white balance calculating process of Claim 7 or 8 characterized by performing weighting with respect to.