JP6351255B2 - Imaging apparatus, imaging method, program, and storage medium - Google Patents

Description

  The present invention relates to an autofocus technique in an imaging apparatus such as an electronic still camera and a video camera.

  Conventionally, when performing autofocus (also referred to as AF) in an electronic still camera or a video camera, the lens position where the high frequency component of the luminance signal obtained from an image sensor using a CCD (charge coupled device) is maximized A method of setting the in-focus position is used. As one of these methods, the following scan method is known. That is, an evaluation value (also referred to as a focus evaluation value) based on the high frequency component of the luminance signal obtained from the image sensor is stored each time while driving the lens over the entire focus detection range. Here, an area for acquiring a focus evaluation value (hereinafter referred to as an AF frame) is set near the center or near the subject detection area in the screen, and the maximum value of the focus evaluation value in each set AF frame is set. A corresponding lens position (hereinafter also referred to as a peak position) is acquired, and a focusing position at the time of photographing is determined.

  By the way, the focusing result obtained in this way is used not only for determining the focusing position at the time of shooting, but also by setting a plurality of AF frames to acquire the distribution of distances in the screen and processing at the time of shooting. And may be used for judgment during image processing.

  For example, Japanese Patent Application Laid-Open No. 2004-151620 discloses the content of acquiring subject distance information of each subject, dividing the screen into several regions, determining a scene for each divided region, and performing predetermined image processing. .

  In Patent Document 2, a captured image is divided into a plurality of regions, and a region to be subjected to blurring processing is determined in each region based on a distribution of distances. At this time, it is disclosed that the setting of a noise removal filter to be applied to the distribution of distances is adjusted in accordance with the detection amount of image blur caused by camera shake.

JP2012-04716A JP2013-42371A

  However, if the distribution of the distance cannot be obtained with high accuracy, there is a possibility that an erroneous region division process is performed or the determination is erroneous. For example, when the AF ranging error becomes large due to the influence of camera shake, or when the variation amount of the AF result becomes large due to the depth of focus depending on the setting of the focal length and F value, The correct distance distribution cannot be obtained.

  In the prior art disclosed in the above-mentioned Patent Document 1, since the determination of the accuracy of the distance information is not performed as described above, there is a case where the region division based on the distance is erroneous. In addition, in the conventional technique disclosed in Patent Document 2 described above, although the influence of camera shake is taken into account, the original distance information may be lost by applying a filter, and the accuracy may deteriorate. Also, depending on the degree of camera shake, there may be a case where accuracy cannot be maintained even if a filter is applied.

  The present invention has been made in view of the above-described problems, and an object thereof is to obtain a distance distribution with high accuracy and to appropriately perform image processing or photographing processing based on the distance distribution based on the result. .

The imaging apparatus according to the present invention determines which bracket shooting is to be performed among a plurality of bracket shootings including at least one of continuous shooting, AF bracket shooting, AE bracket shooting, and strobe bracket shooting depending on the scene. Based on the bracket determination means, the distribution acquisition determination means for determining whether or not to acquire the distribution of the in-focus state in the captured image based on the focal length and the exposure condition, and the distribution acquisition determination means in the captured image. When it is determined to acquire the focus state distribution, the focus state is detected in a plurality of focus detection areas while moving the focus lens, and the focus state distribution in the captured image is acquired and based on the distribution. The captured image is divided into a plurality of regions, the focus of the focus lens is adjusted, and the distribution acquisition determination unit performs the adjustment. When it is determined that the distribution of the focus state in the image image is not acquired, the focus lens is moved while the focus lens is moved, and the focus state in the preset focus detection region is detected to adjust the focus of the focus lens. And a focusing means for performing .

  According to the present invention, it is possible to obtain a distance distribution with high accuracy and appropriately perform image processing or photographing processing based on the distance distribution based on the result.

It is a block diagram which shows the structure of the electronic camera which is one Embodiment of the imaging device to which this invention is applied. It is a flowchart explaining operation | movement of an electronic camera. It is a flowchart explaining the bracket determination in FIG. 3 is a flowchart for explaining scanning method determination processing in FIG. 2. 3 is a flowchart for explaining a main exposure AF scan in FIG. 2. It is a figure explaining AF frame setting in FIG. 3 is a flowchart for explaining a subroutine of image processing in FIG. 2.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

<Block diagram of electronic camera>
FIG. 1 is a block diagram showing a configuration of an electronic camera which is an embodiment of an imaging apparatus to which the present invention is applied.

  In FIG. 1, reference numeral 101 denotes a photographing lens including a zoom mechanism. Reference numeral 102 denotes an aperture and a shutter that control the amount of light. Reference numeral 103 denotes an AE processing unit. Reference numeral 104 denotes a focus lens for focusing (adjusting the focus) on an image sensor, which will be described later. Reference numeral 105 denotes an AF processing unit. Reference numeral 106 denotes a strobe. Reference numeral 107 denotes an EF processing unit. Reference numeral 108 denotes an image sensor as a light receiving means or a photoelectric conversion means for converting a subject image, which is reflected light from a subject formed through the focus lens 104, into an electrical signal. Reference numeral 109 denotes an A / D converter including a CDS circuit that removes output noise from the image sensor 108 and a non-linear amplifier circuit that is executed before A / D conversion. Reference numeral 110 denotes an image processing unit. Reference numeral 112 denotes a format conversion unit.

  Reference numeral 113 denotes a high-speed built-in memory (for example, a random access memory, hereinafter referred to as DRAM). The DRAM 113 is used as a high-speed buffer as temporary image storage means, or a working memory for image compression / decompression. An image recording unit 114 includes a recording medium such as a memory card and its interface. Reference numeral 115 denotes a system control unit (hereinafter referred to as CPU) that controls the system such as a photographing sequence. Reference numeral 116 denotes an image display memory (hereinafter referred to as VRAM), and 117 denotes an operation display unit that displays an image for assisting operation and a camera state in addition to an image display, and also displays a shooting screen and a distance measurement area at the time of shooting.

  Reference numeral 118 denotes an operation unit for operating the camera from the outside. The operation unit 118 includes the following, for example. There are a menu switch for performing various settings such as a shooting function of the image pickup apparatus and a setting at the time of image reproduction, a zoom lever for instructing a zoom operation of the shooting lens, an operation mode switching switch between a shooting mode and a playback mode, and the like.

  Reference numeral 121 denotes a switch (hereinafter referred to as SW1) for performing a shooting standby operation such as AF or AE. Reference numeral 122 denotes a photographing switch (hereinafter referred to as SW2) that performs photographing after the operation of SW1. Reference numeral 123 denotes a subject detection module that detects a subject using the image signal processed by the image processing unit 110 and sends information (position / size / reliability) of one or more detected subjects to the CPU 115. It should be noted that the subject detection method is not the main point of the present invention and will not be described in detail. An angular velocity sensor unit 125 detects the movement of the camera itself and sends camera movement information to the CPU 115.

<Operation of electronic camera>
Next, the operation of the electronic camera of this embodiment will be described with reference to the flowchart of FIG.

  First, in S201, the CPU 115 determines the state (ON / OFF) of SW1 instructing to prepare for photographing. If the state of SW1 is ON (on), the process proceeds to S203, and if it is OFF (off), the process proceeds to S202. move on.

  In S202, SW1 standby operation is performed, control is performed so as to keep the focus and exposure of the image displayed on the operation display unit 117 optimal, and information necessary for bracket determination in S203 described later is acquired. Specifically, the AF processing unit 105 performs focus tracking on the subject in the screen, and the AE processing unit 103 performs control so as to keep the exposure appropriate, and acquires exposure information necessary for bracket determination. The subject detection module 123 also acquires subject motion information necessary for bracket determination.

  In S203, bracket determination for determining the optimum bracket processing (AF bracket, AE bracket, strobe bracket, continuous shooting) at the time of shooting is performed according to the procedure described later, and the process proceeds to S204.

  In S204, it is determined whether or not to perform AF scan for acquiring the distribution of distances in the screen (hereinafter referred to as distance map) during AF scan after SW1, according to the procedure described later, and the process proceeds to S205. In S205, the AE processing unit 103 performs AE processing for main exposure, and the process proceeds to S206.

  In S206, AF for main exposure is performed according to the procedure described later, and the process proceeds to S207. Here, the in-focus position (in-focus state) at the time of shooting is determined, and if it is determined in S204 that the AF scan for acquiring the distance map is performed, the distance map is also acquired based on the AF scan result. .

  In S207, bracketing processing at the time of shooting is finally determined according to the AF scan result in S206. Here, as a result of the AF scan in S206, if it is determined that the scene has no distance difference in the screen and the effect of the AF bracket is not effective, the bracket processing is changed so that the AF bracket is not performed.

  In S208, the CPU 115 determines the state (ON / OFF) of the photographing switch SW2, and proceeds to S209 if the state of SW2 is ON, but returns to S208 if it is OFF.

  In S209, based on the bracket type determined in S207, bracket shooting is performed according to the procedure described later, and the process proceeds to S210.

  In S210, image processing based on the bracket type is performed on the plurality of captured images obtained by bracket shooting in S209 according to a procedure described later, and the present processing is terminated.

<Bracket judgment>
FIG. 3 is a flowchart illustrating the bracket determination process in S203 of FIG. Here, the optimum processing at the time of photographing is determined from information obtained before turning on SW1.

  Here, the accuracy of the distance map also deteriorates when the AF variation increases due to the effect of increasing the depth of focus or the effect of camera shake. Further, when the influence of the curvature of field due to the performance of the focus lens 104 is large, or when the focus lens 104 and the image sensor 108 are greatly inclined with respect to the optical axis, the distance map is greatly affected. Therefore, the accuracy of the distance map is guaranteed by measuring in advance the degree of influence of field curvature and tilt and correcting the distance map value for each individual based on the measured value (hereinafter referred to as image plane correction). I have to. However, as the depth of focus increases, the accuracy of measurement for image plane correction cannot be maintained, making it difficult to perform image plane correction.

  Based on the above situation, when a reliable distance map can be acquired, AF variation can be guaranteed by considering the influence of the depth of focus and camera shake from the focal length and aperture setting at the time of AF scanning. The case where image surface correction can be performed is targeted.

  In S301, the setting conditions at the time of performing the AF scan in S206 in FIG. 2 are predicted, it is determined whether or not the scene can obtain a reliable distance map, and the process proceeds to S302. For example, when an AF scan is performed, when the optical zoom setting of the photographing lens 101 is shorter than a predetermined focal length, or when the aperture 102 is wider than a predetermined value (predetermined range). If it is within, it is determined that a reliable distance map can be acquired.

  In S302, it is determined whether or not the camera or the subject is moving by referring to the output results from the angular velocity sensor unit 125 and the subject detection module 123 immediately before turning on SW1. If it is a scene where the camera or the subject is moving, the process proceeds to S309, and if it is not a scene, the process proceeds to S303.

  In S303, it is determined from the output result of the AE processing unit 103 immediately before turning on SW1 whether or not the scene is darker than a predetermined value and has a strobe light emission effect. If it is determined that the scene is darker than a predetermined value and has a strobe light emission effect, the process proceeds to S308. If it is not determined that the strobe light emission effect is effective, the process proceeds to S304.

  In S304, the exposure distribution in the screen in the AE processing unit 103 immediately before turning on SW1 is acquired. If it is determined from the result that the scene has an exposure difference in the screen or a backlight scene, the process proceeds to S307. . If there is no exposure difference in the screen and the scene is not a backlight scene, the process proceeds to S305.

  In S305, if it is determined in S301 that a reliable distance map can be acquired, the process proceeds to S306, and if it is determined that a reliable distance map cannot be acquired, the process proceeds to S307.

  In S306, it is determined that AF bracketing is performed as a process at the time of shooting, and this process ends. In step S307, it is determined that the AE bracketing is performed as a process at the time of shooting, and this process ends. In step S308, it is determined that strobe bracketing is to be performed as a process at the time of shooting, and this process ends. In step S309, it is determined that continuous shooting is performed as a process at the time of shooting, and this process ends.

<Determination of scanning method>
FIG. 4 is a flowchart for explaining the AF scan method determination process in S204 in FIG. Here, based on the bracket type and the setting conditions at the time of AF scanning, it is determined whether or not to perform AF scanning for distance map acquisition.

  In S401, it is referred to which bracket type is determined to be implemented in the bracket determination in S203 of FIG. If it is determined that AE bracketing and strobe bracketing are to be performed, the process proceeds to S402. If it is determined to perform AF bracketing, the process proceeds to S404. If it is determined to perform continuous shooting, the process proceeds to S405.

  Here, when performing continuous shooting processing, the focusing accuracy in each AF frame also deteriorates due to the influence of the movement of the subject or the camera. Therefore, it is assumed that AF scanning is performed only for determining the in-focus position without acquiring the distance map.

  Further, the case where the AF bracket process is performed is a case where it is determined that a reliable distance distribution can be acquired in the distance map reliability determination in S301 of FIG. 3, and therefore, an AF scan for acquiring a distance map is performed. Shall. Here, when the influence of camera shake during AF scanning is large and the accuracy of the distance map obtained by AF scanning is low, it is determined that the distance map cannot be acquired. In general, it is said that the influence of camera shake is determined by the relationship between the focal length and the exposure time. Therefore, whether the focal length and the exposure time at the optical zoom magnification of the photographing lens 101 at the time of AF scanning satisfy a predetermined condition. Thus, whether or not the distance map can be acquired is determined.

  In S402, based on the setting conditions at the time of performing the AF scan, it is determined whether or not the AF setting conditions capable of acquiring a correct distance map are satisfied, and the process proceeds to S403. In S403, when it is determined in S402 that the distance map can be acquired, if it is determined that the distance map can be acquired, the process proceeds to S404, and if it is determined that the distance map cannot be acquired, the process proceeds to S405. In S404, it is determined that an AF scan for obtaining a distance map is to be executed, and this process is terminated.

  In S405, it is determined that a normal AF scan for determining the in-focus position is to be performed, and the present process is terminated.

<AF scan>
FIG. 5 is a flowchart for explaining the AF process of S206 in FIG. Here, when it is determined that the distance map acquisition scan is performed in the AF scan method determination process of S204 in FIG. 2, a scan for acquiring the distance map and determining the in-focus position at the time of photographing is performed. When it is determined that the distance map acquisition scan is not performed, an AF scan for determining a focus position at the time of shooting is performed.

  In S501, it is checked whether the distance map acquisition scan is determined to be performed in the AF scan method determination process in S204 of FIG. 2, and when it is determined that the scan for acquiring the distance map is performed, the process proceeds to S502 and is not determined. In this case, the process proceeds to S508.

  In S502, N × M AF frames (focus detection areas) are set in the screen. For example, when N = 7 and M = 9, the AF frame setting is as shown in FIG. Alternatively, the AF frame setting range may be set in consideration of the subject detection position by the subject detection module 123 in the screen.

  In S503, the focus lens 104 is moved to the scan start position, and AF scanning is performed until the peak positions in all AF frames set in S502 are found for obtaining the distance map. Here, the scan start position is, for example, the position at the far end of the AF scanable range such as infinity.

  In S504, an in-focus position is determined based on the peak position of each AF frame obtained as a result of the AF scan in S503.

  In S505, the result of the reliability determination of the distance map in S301 in FIG. 3 is referred to, and if a reliable distance map can be acquired, the process proceeds to S506, and if a reliable distance map cannot be acquired, the process proceeds to S507.

  In S506, image plane correction is performed to cope with a case where there is an error in the distribution of peak positions due to curvature of field or inclination of the lens / image sensor, and the process proceeds to S511. Here, the distance map is acquired by correcting the peak position of each AF frame based on the error amount for each image height acquired in advance.

  In S507, since it is determined that the distance map is not reliable, it is determined that the scene cannot be subjected to image plane correction for field curvature or tilting, and the distance map is not corrected without correcting the peak position of each AF frame. Acquire and proceed to S511.

  FIG. 6A shows a case where the subject is a front flower and a background mountain, and FIG. 6B shows a distance map when the front flower and the background mountain are photographed. The front flowers have peak positions distributed in the range of 199 to 202, and the background mountains have peak positions distributed in the range of 100 to 103. By grouping adjacent frames with close peak positions, as shown in FIG. 6B, the captured image is surrounded by a focus area 1 (region surrounded by fine dotted lines) and 2 (coarse dotted lines). The area can be divided into three areas (area) and 3 (area surrounded by a solid line). Here, since the focus areas 1 and 2 have close peak positions, they can be divided into a background group by the focus areas 1 and 2 and a front subject group by the focus area 3 by integrating them as the same area. . Furthermore, the accuracy of grouping may be improved by combining the result of dividing the screen by color in the image processing unit 110 and the result of division by distance.

  The screen is divided into a plurality of groups based on the distance map obtained in S506 or S507 as described above.

  In S508, when the subject is detected by the subject detection module 123, an AF frame is set at the subject position, and when no subject is detected, a multi-frame AF frame is set, and the process proceeds to S509.

  In step S509, an AF scan for determining the in-focus position is performed. Here, when the subject is detected, scanning is performed until the peak position is found in the AF frame set as the subject detection position, and when the subject is not detected, the AF frame close to the peak position is obtained from the multi-frame AF frame. Perform a scan.

  In S510, the in-focus position is determined from the peak position in each AF frame obtained as a result of scanning in S509.

  In S511, if the in-focus position is determined in S504 or S510, the focus lens is driven to the in-focus position. If the in-focus position is not determined and the in-focus state is not determined, a subject existence probability called a predetermined fixed point is determined. The focus lens is driven to a higher position.

  In step S512, the focus / non-focus frame is displayed on the operation display unit 117. At the time of in-focus, the AF frame within the in-focus position and depth is displayed as the in-focus frame, and at the time of out-of-focus, the non-in-focus frame is displayed at a predetermined position such as the center, and this processing is terminated.

<Shooting process>
FIG. 7 is a flowchart for explaining the image processing operation for the captured image in S210 of FIG. Here, the result of the bracket determination in S203 and S207 in FIG. 2, the result of the AE process for main exposure in S205 in FIG. 2, and the AE result and distance map obtained by the AF process for main exposure in S206 are appropriate. Perform appropriate image processing.

  In S701, it is checked whether or not a distance map has been acquired in the AF processing for main exposure in S206 of FIG. 2. If the distance map is acquired, the process proceeds to S702, and if not, the process proceeds to S706.

  In S702, as a result of the bracket determination in S203 and S207 in FIG. 2, it is determined which bracket type is determined to be performed at the time of shooting. If AF bracketing is performed, the process proceeds to S703, and if AE bracketing and strobe bracketing are performed. The process proceeds to S704.

  In step S703, the main subject region in the screen is specified based on the grouping result based on the distance map and the color distribution in the screen for each captured image that has been subjected to focus bracket shooting (focus bracket processing), and other than the main subject region. An image with a blurring process according to the distance distribution is created. Here, in the reliability determination of the distance map in S301 of FIG. 3, since it is determined to perform the AF bracket process only when the distance map is determined to be a reliable scene, a reliable distance map can be acquired. It is possible to perform advanced blur processing with high accuracy.

  In S704, a simple blurring process is performed in which an area of a predetermined size determined in advance according to the distribution of the distance map is left, and the process proceeds to S705. Here, simple blurring processing, for example, when it is determined that the scene is a distant scene based on a distance map, blurring processing is performed on an area other than a predetermined size near the center of the screen, so that a diorama effect can be obtained. Can be obtained. Here, since the subject area is not strictly determined, a highly reliable distance map is not necessarily required.

  In step S <b> 705, an image is generated on which processing for cutting out a part of the screen is performed based on the subject detection area, the distance map, and the region division result based on the color distribution, and the process advances to step S <b> 707.

  In S <b> 706, an image subjected to a process of cutting out a part of the screen according to the detection result of the subject and the in-focus position in the screen is generated, and the process proceeds to S <b> 707.

  In step S707, a process for applying a predetermined color filter to the screen according to the detection result of the subject, the color distribution in the screen, and the exposure distribution is performed, and the process ends.

  As described above, it is possible to appropriately perform image processing using the distance map by determining whether or not the distance map is a scene that can be accurately acquired based on the setting conditions during AF scanning.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (11)

Bracket determination means for determining which bracket shooting is to be performed among a plurality of bracket shootings including at least one of continuous shooting, AF bracket shooting, AE bracket shooting, and strobe bracket shooting according to a scene ;
Distribution acquisition determination means for determining whether or not to acquire a distribution of in-focus state in the captured image based on the focal length and the exposure condition;
When it is determined by the distribution acquisition determination means to acquire the distribution of the in-focus state in the captured image, the in-focus state in the plurality of focus detection areas is detected while moving the focus lens, and the in-captured image is detected. A distribution of in-focus states is acquired, the captured image is divided into a plurality of regions based on the distribution, the focus of the focus lens is adjusted, and the distribution of in-focus states in the captured image is determined by the distribution acquisition determining unit. A focus adjusting unit that detects the in-focus state in a preset focus detection area while moving the focus lens and determines the focus of the focus lens when it is determined that the focus lens is not acquired. An imaging device that is characterized. When the focus adjustment unit determines that continuous shooting is performed by the bracket determination unit, the focus adjustment state in a preset focus detection region is determined while moving the focus lens regardless of the focal length and exposure conditions. The imaging device according to claim 1, wherein the focus adjustment of the focus lens is performed. Said bracket determining means, when the focal length and aperture value first condition is satisfied, performs the AF bracket shooting, if the focal length and the aperture value does not satisfy the first condition, perform the AF bracket shooting The imaging apparatus according to claim 1, wherein it is determined that there is no image. Wherein the first condition, wherein the focal length of the taking lens when performing the focal adjustment is shorter than a predetermined distance, or the aperture value of the photographing lens is characterized in that it is a smaller than a predetermined value Item 4. The imaging device according to Item 3 . Wherein the first condition, the imaging apparatus according to claim 3, the focal distance or the aperture value of the photographic lens when performing the focal adjustment, characterized in that the case is within a predetermined range. The image processing apparatus further includes a correction unit that performs a blurring process according to the distribution of the in-focus state in the captured image . When the first condition is satisfied, the correction unit performs correction more than when the first condition is not satisfied. The imaging apparatus according to claim 3 , wherein the imaging apparatus is strengthened. First processing means for performing a first process on a captured image using a result obtained by dividing the captured image into a plurality of regions;
A second processing means for performing a second process different from the first process using a result obtained by dividing the captured image into a plurality of areas;
4. The imaging according to claim 3 , wherein the first processing and the second processing are performed when the AF bracket photographing is performed, and the second processing is performed when the AF bracket photographing is not performed. apparatus. 2. The scene according to claim 1, wherein the scene is at least one of a scene in which a camera or a subject is moving, a scene having an effect of strobe light emission, a scene having an exposure difference in a screen, and a backlight scene. Imaging device. A bracket determination step for determining which bracket shooting is to be performed among a plurality of bracket shootings including at least one of continuous shooting, AF bracket shooting, AE bracket shooting, and strobe bracket shooting according to the scene ;
A distribution acquisition determination step of determining whether or not to acquire a distribution of in-focus state in the captured image based on the focal length and the exposure condition;
When it is determined in the distribution acquisition determination step that the distribution of the in-focus state in the captured image is acquired, the in-focus state in the plurality of focus detection areas is detected while moving the focus lens, and the in-captured image is detected. Obtain a distribution of in-focus states, divide the captured image into a plurality of regions based on the distribution, perform focus adjustment of the focus lens, and determine the distribution of in-focus states in the captured image by the distribution acquisition determination step. A focus adjustment step of adjusting the focus of the focus lens by detecting a focus state in a preset focus detection region while moving the focus lens when it is determined that the focus lens is not acquired. A characteristic imaging method. The program for making a computer perform each process of the imaging method of Claim 9 . A computer-readable storage medium storing a program for causing a computer to execute each step of the imaging method according to claim 9 .