JP2017098777A - Imaging apparatus and control method, and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately calculate a flicker waveform under a flicker light source.SOLUTION: An exclusion area is identified based on a photometric value in each divided area in a representative image, and a photometric value in each divided area excluding the divided areas in the image other than the representative image corresponding to the exclusion area is acquired. A flicker waveform is generated based on photometric values, from which photometric values to be excluded from the photometric values in each divided area of the representative image is excluded, and a photometric value in each divided area excluding divided area corresponding to the exclusion areas in the image other than the representative image.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an imaging apparatus including an imaging element driven by a rolling shutter system, a control method thereof, and a program.

  With the recent increase in sensitivity of an image sensor provided in an imaging apparatus such as a digital camera, it has become possible to take a blur-free photograph with a high-speed shutter even in a relatively dark environment such as a room. Further, under an artificial light source that is widely used as an indoor light source, for example, a fluorescent lamp, a flicker phenomenon in which illumination light periodically fluctuates due to a periodic voltage change of a commercial power source. In such a high-speed shutter shooting under a flicker light source such as a fluorescent lamp whose illumination light periodically fluctuates, the exposure of the image and the variation of the color temperature occur every frame due to the effect of flicker, or within one frame. Exposure unevenness and color unevenness may occur.

  To solve this problem, there is a flicker removal technology that detects the flicker period and phase, generates a flicker synchronization signal so that exposure can be performed in accordance with the peak timing of the flicker light quantity with little change in brightness, and performs exposure. Proposed. However, the flicker waveform varies depending on the artificial light source, and there are various light sources such as a light source having a long light amount and a long light source. Depending on the flicker waveform of the light source, the period and timing at which flicker-removed imaging can be performed vary.

  In the technique disclosed in Patent Document 1, a feature point that is a flicker peak is detected and photographed. Further, in the technique disclosed in Patent Document 2, in order to cope with both frequencies of 50 Hz and 60 Hz, shooting is performed at a frame rate in which the phase is shifted by about 180 degrees for each frame, and the flicker frequency is calculated from the ratio of the images of the two frames before and after. Seeking.

JP 2014-220763 A JP 2007-37103 A

  However, since Patent Document 1 does not consider a flicker waveform that varies depending on the artificial light source, even if there is a shooting timing that allows flicker-removed shooting other than the peak, it cannot be used for shooting. Further, in Patent Document 2, since the flicker waveform cannot be obtained and the flicker peak is not obtained, it is not possible to perform photographing without removing the flicker.

  When a flicker waveform is generated by exposing a period of one or more cycles of flicker at once and measuring a change in luminance, the influence of the subject reflected in the image data is greatly affected. FIG. 8 shows a flicker waveform obtained by acquiring image data in a period of one cycle or more and calculating from a luminance change. A change in luminance with time is observed by dividing the image data 200 into a plurality of pieces and acquiring the luminance. An area 201 is an area affected by the subject in the image data 200. According to the luminance value corresponding to the area 201, a waveform different from the original waveform is formed as a flicker waveform due to the influence of the subject. That is, there is a problem that it is difficult to generate an accurate flicker waveform under a flicker light source.

  By the way, in flickerless photography that captures images at the timing when the light source emits light, the quality of the captured image is greatly influenced by whether or not the image can be captured while the light source emits light. The situation where the light source emits light varies depending on the type of the light source (fluorescent lamp or LED illumination or the like). Therefore, in order to appropriately set the flickerless shooting period, it is important to obtain an accurate flicker waveform.

  An object of the present invention is to accurately obtain a flicker waveform under a flicker light source.

  In order to achieve the above object, the present invention is an image pickup apparatus including an image pickup device driven by a rolling shutter system, and obtains a photometric value for each of a plurality of images acquired by the image pickup device. Selection means for selecting a representative image from the plurality of images, first division means for dividing the representative image selected by the selection means into a plurality of regions having different exposure timings, and the first division A second acquisition unit that acquires a photometric value for each divided region divided by the unit, and a flicker waveform generated under a flicker light source based on the photometric value for each divided region acquired by the second acquisition unit. A specifying unit that specifies a divided region corresponding to the photometric value to be excluded; and the representative image of the plurality of images in correspondence with the divided region divided by the first dividing unit. Corresponding to the divided area specified by the specifying means among the divided areas divided by the second dividing means and a second dividing means for dividing each of the images other than the plurality of areas having different exposure timings A third acquisition means for acquiring a photometric value for each divided area excluding the object, a photometric value excluding the photometric value to be excluded from the photometric values for each divided area of the representative image, and the third acquiring means. Generating means for generating the flicker waveform based on the obtained photometric value for each divided region.

  According to the present invention, a flicker waveform can be accurately obtained under a flicker light source.

It is a schematic block diagram of an imaging device. It is a flowchart of a flicker waveform generation process. FIG. 6 is a diagram (FIGS. (A) and (b)) showing charge accumulation timing and image signal readout timing for flicker detection when the commercial power supply frequency is 50 Hz and 60 Hz. It is a figure which shows a some image and a rough waveform. It is a figure which shows the example which divided | segmented the representative image into five. It is a figure which shows the example which divided | segmented the image other than a representative image into five. It is a figure which shows the exact flicker waveform produced | generated from the photometry value for every division area. It is a figure which shows the flicker waveform calculated by the conventional method. It is a figure which shows the example of the flicker waveform produced | generated in the environment where the light emission time of a light source is long, and a short environment, respectively (a figure, (b)). It is a flowchart of a process of imaging | photography timing control. It is a figure which shows the relationship between a shutter driving | running | working period and a flickerless imaging | photography period. It is a figure which shows the relationship between a shutter driving | running | working period and a flickerless imaging | photography period. It is a figure which shows the relationship between a shutter driving | running | working period and a flickerless imaging | photography period.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of an imaging apparatus according to an embodiment of the present invention. This imaging apparatus is configured as a digital single-lens reflex camera, and includes a camera body 100 and a lens unit 120 that can be attached to and detached from the camera body 100.

  First, the configuration of the camera body 100 will be described. A camera microcomputer 101 which is a microcomputer CPU controls each part of the camera body 100. The memory 102 is a memory such as a RAM or a ROM connected to the camera microcomputer 101. The image sensor 103 is an image sensor such as a CCD or CMOS including an infrared cut filter, a low-pass filter, and the like, and photoelectrically converts a light beam incident through the lens unit 120 and outputs an image signal.

  The shutter 104 shields the image sensor 103 when not photographing, and opens to guide the light beam to the image sensor 103 when photographing. That is, the shutter 104 travels so as to be in a light shielding state in which the image sensor 103 is shielded from a light beam incident through the lens unit 120 and in a retracted state in which the light beam incident through the lens unit 120 is guided to the image sensor 103. . The half mirror 105 changes the optical path of the light beam incident through the lens unit 120 and guides the light beam incident through the lens unit 120 to the image sensor 103 during photographing. The half mirror 105 guides incident light to the photometric sensor 108 when not photographing, and reflects a part of the incident light to form an image on the focusing plate 106.

  The display element 107 is a display element using a PN liquid crystal or the like, displays a frame (AF frame) indicating a focus detection area used for automatic focus adjustment control (AF control), and the like when looking into the optical viewfinder. It indicates to the user whether AF is being performed at the position. The photometric sensor 108 is a charge storage type image sensor that accumulates electric charges according to the amount of incident light such as a CMOS, and performs not only photometry but also subject face detection, subject tracking, flicker detection, etc. based on the output image signal. It can be carried out. In the present embodiment, a photometric sensor is taken as an example of a device that detects flicker, but the image sensor 103 may be used as a device for detecting flicker.

  The pentaprism 109 guides the subject image on the focusing screen 106 to the photometric sensor 108 and an optical finder (not shown). The photometric sensor 108 is disposed at a position where the subject image formed on the focusing screen 106 via the pentaprism 109 is viewed obliquely. The focus detection circuit 110 performs focus detection for AF control, and a part of the light beam incident through the lens unit 120 and passing through the half mirror 105 by the AF mirror 111 is contained in the focus detection circuit 110. It is guided to an AF sensor (not shown).

  The ICPU 112 is a CPU for driving control of the photometric sensor 108 and image processing / calculation, and is related to photometry, subject face detection, subject tracking, flicker detection and the like based on an output signal (image signal) from the photometric sensor 108. Perform various calculations. The memory 113 is a memory such as a RAM or a ROM connected to the ICPU 112. In this embodiment, a configuration in which the ICPU 112 is provided separately from the camera microcomputer 101 is described. However, a configuration in which the camera microcomputer 101 executes processing executed by the ICPU 112 may be used.

  The operation unit 114 is a release button for the user to instruct the camera body 100 to start a shooting preparation operation (SW1 is turned on) or a shooting operation is started (SW2 is turned on), and the user can set various settings of the camera body 100. Includes setting buttons to do. The operation unit 114 includes a power switch for the user to turn on / off the power of the camera body 100, a mode dial for the user to select an operation mode of the camera body 100 from a plurality of modes, a touch panel, and the like. . The lens CPU 121 (hereinafter referred to as LPU) controls each unit of the lens unit 120, for example, a focus lens, a zoom lens, a diaphragm driving unit, and the like, and transmits information about the lens to the camera microcomputer 101.

  Incidentally, the photometric sensor 108 is an image sensor that can be driven by a rolling shutter system. In general, the ICPU 112 sequentially scans a plurality of pixels arranged two-dimensionally in the photometric sensor 108 for each predetermined line, resets the charge of the pixels, and sequentially scans for each predetermined line after a predetermined exposure time has elapsed. The accumulated charge is read out and a signal is output. The predetermined number of lines may be 1 or 2 or more.

  Next, generation of a flicker waveform under a flicker light source will be described. FIG. 2 is a flowchart of the flicker waveform generation process. In the process of FIG. 2, the ICPU 112 serves as a selection unit, first, second, and third division unit, first and second acquisition unit, identification unit, and generation unit in the present invention. This process is started, for example, when the power of the camera body 100 is turned on by turning on the power switch. First, in step S101, the ICPU 112 continuously drives the photometric sensor 108 to acquire a plurality of images. As shown in FIG. 3, this operation is an accumulation and readout operation for detecting flicker.

  FIGS. 3A and 3B are diagrams showing charge accumulation timing and image signal readout timing for flicker detection. The ICPU 112 performs control so that accumulation and reading are continuously performed 12 times at a cycle of 600 fps and about 1.667 ms. This 600 fps is a value equal to the least common multiple of the flicker frequencies (100 Hz and 120 Hz) assumed in advance. Further, by accumulating 12 times at 600 fps, accumulation is performed in a period of 20 ms as a whole, and the light quantity change of the flicker light source is equivalent to two cycles regardless of whether the commercial power supply frequency is 50 Hz or 60 Hz. Will be included. Regarding accumulation / reading, it is necessary to secure a period of at least one cycle.

  In recent single-lens reflex cameras, an image signal is acquired by a pentaprism photometric sensor immediately before shooting, and the image signal is processed to perform face detection and subject tracking, and then perform photometry based on the image signal. There is a system. In the case of a CMOS sensor, partial reading can be performed relatively easily, so that the total time of accumulation and reading is adjusted to about 1.667 ms by so-called thinning-out reading. In this way, the photometric sensor 108 is driven at 600 fps (cycle of 1.667 ms).

  Next, in step S102, the ICPU 112 acquires a photometric value (luminance value) for each of the plurality of images acquired in step S101 for luminance evaluation for flicker detection. FIGS. 3A and 3B show transitions of charge accumulation timing, image signal readout timing, and photometric value when the commercial power supply frequency is 50 Hz and 60 Hz, respectively. The nth accumulation is described as “accumulation n”, the readout of the accumulation n is described as “readout n”, and the photometric value obtained from the result of the readout n is described as “AE (n)”. In addition, regarding the acquisition time of each photometric value, accumulation | storage is performed in finite time. The ICPU 112 obtains the median value (median) of the photometric values for each of the plurality of lines in the photometric sensor 108 for each of the images, and acquires the obtained plural photometric values as the photometric values of each image. In addition, you may make it acquire the average value of the photometric value of a some line as a photometric value of each image.

  When the commercial power supply frequency is 50 Hz, the light emission period (light quantity change period) of the flicker light source is about 10 ms and 10 ÷ 1.667≈6. Therefore, as shown in FIG. In other words, the same photometric value is obtained in six cycles. That is, the relationship of AE (n) ≈AE (n + 6) is established. Similarly, since the light emission period of the flicker light source when the commercial power supply frequency is 60 Hz is about 8.33 ms and is 8.33 ÷ 1.667≈5, as shown in FIG. The same photometric value is obtained with AE (n) ≈AE (n + 5).

  On the other hand, under a light source having no flicker (no change in light amount), AE (n) is substantially constant regardless of n. Therefore, the ICPU 112 calculates the evaluation values F50 and F60 when the commercial power supply frequencies are 50 Hz and 60 Hz, using Equations 1 and 2.

  The ICPU 112 performs flicker detection by comparing the evaluation value F50 and the evaluation value F60 with a predetermined threshold value F_th. Specifically, when F50 <F_th and F60 <F_th are satisfied, it can be said that all of the plurality of photometric values are substantially equal, and therefore it is determined that no flicker occurs. When F50 <F_th and F60 ≧ F_th are satisfied, it can be said that the plurality of photometric values are substantially equal in six cycles and are not substantially equal in five cycles. Therefore, it is determined that a flicker with a light quantity change period of 10 ms occurs (under a flicker light source with a commercial power supply frequency of 50 Hz). When F50 ≧ F_th and F60 <F_th are satisfied, it can be said that the plurality of photometric values are substantially equal in the five-time cycle and are not substantially equal in the six-time cycle. For this reason, it is determined that flicker having a light quantity change period of 8.33 ms occurs (under a flicker light source having a commercial power supply frequency of 60 Hz).

  It is also conceivable that the photometric value changes greatly due to the movement of the imaging device such as panning or the movement of the subject, and F50 ≧ F_th and F60 ≧ F_th. In that case, the ICPU 112 compares the evaluation value F50 with the evaluation value F60 and performs flicker detection. Specifically, when F50 ≧ F_th, F60 ≧ F_th, and F50 ≦ F60 are satisfied, it is determined that a flicker with a light quantity change period of 10 ms occurs (under a flicker light source with a commercial power supply frequency of 50 Hz). The When F50 ≧ F_th and F60 ≧ F_th and F50> F60 are satisfied, it is determined that flicker with a light amount change period of 8.33 ms is generated (under a flicker light source with a commercial power supply frequency of 60 Hz). If F50 ≧ F_th, F60 ≧ F_th, and F50 = F60, the light emission cycle of the flicker light source cannot be determined, and therefore it may be determined that no flicker occurs or flicker cannot be detected. When F50 ≧ F_th and F60 ≧ F_th are satisfied, flicker detection may be performed again, assuming that the reliability of the flicker detection result is low.

  Next, in step S103, the ICPU 112 generates a coarse flicker waveform (hereinafter referred to as a coarse waveform) using the photometric value for each image calculated in step S102. FIG. 4 is a diagram illustrating a plurality of images and a rough waveform. Images 301 to 306 in FIG. 4 are six images taken at a period of 1.667 ms. Photometric values Y301 to Y306 of the images 301 to 306 correspond to AE (1) to AE (6). As described in step S102, the ICPU 112 obtains the photometric values Y301 to Y306 by representing the photometric value for each line by the median value during the accumulation period. Then, the ICPU 112 generates a rough waveform by connecting adjacent photometric values and graphing them, such as the photometric value Y301 and the photometric value Y302, and the photometric value Y302 and the photometric value Y303. In FIG. 4, the number of images is six, but a rough waveform may be generated using the photometric values Y of all acquired images.

  Next, in step S <b> 104, the ICPU 112 selects an image corresponding to an area having the smallest degree of change in the coarse waveform from the images 301 to 306 as a representative image. Here, the determination of an image with a small waveform change can be made by, for example, comparing the flicker cycle with the photometric values Y301 to Y306. For example, the ICPU 112 determines a photometric value Y that minimizes the difference between two photometric values Y adjacent to the target photometric value Y, and selects an image having the determined photometric value Y as a representative image. Alternatively, when it is difficult to select an image with little change in luminance, an image whose luminance changes monotonously may be selected. In step S105, which will be described later, since the representative image is further divided into regions and the luminance value is calculated, it is desirable to select an image that can predict the newly obtained luminance value as the representative image. In the present embodiment, it is assumed that the image 301 is selected as the representative image. Note that it is preferable to use an image with a small waveform change as a representative image because the accuracy is high, but it is not essential that the representative image is an image with a minimum waveform change.

  Next, in step S105, the ICPU 112 divides the representative image in the horizontal direction and obtains a photometric value (luminance value) for each divided area. That is, the ICPU 112 first divides the selected representative image. Here, the representative image is divided so as to be one divided region for every predetermined number of lines in the photometric sensor 108. In the CMOS sensor, when the image is taken by the rolling shutter method, even in the case of image data within one screen, the image data has different accumulation timing depending on the line. Therefore, the image is divided into a plurality of divided regions so that a time difference occurs in the accumulation start / end times with respect to the sensor reading order. Further, by obtaining a photometric value for each of the divided areas, it is possible to obtain a photometric value with different exposure timing, that is, acquisition timing, within one screen.

  FIG. 5 shows an example in which the image 301 is divided into five divided areas 301-1 to 301-5. Note that the number of divisions of the area is not limited to five. If you divide it finely, you can measure the luminance change at different timings more finely, but the number of pixels for calculating the luminance decreases, so it is more susceptible to the image and noise that you shoot. End up. Therefore, the number of divisions is preferably determined from the number of pixels and characteristics of the image sensor. The photometric values obtained in each of the divided regions 301-1 to 301-5 are photometric values Y301-1 to Y301-5. When a plurality of lines correspond to one divided region, the median value (median) of the photometric values of each line corresponding to the divided region is set as a photometric value for each divided region. The average value of the photometric values of each line corresponding to the divided area may be used as the photometric value for each divided area.

  Next, in step S106, the ICPU 112 identifies a divided area corresponding to a photometric value to be excluded when generating a flicker waveform as an excluded area. First, the ICPU 112 sorts the photometric values Y301-1 to Y301-5 of each divided area into usable data and unusable data. Since the subject is shown in the captured image, depending on the region, the subject is greatly influenced, and an inappropriate photometric value may be calculated from the viewpoint of flicker detection. As shown in FIG. 5, the ICPU 112 arranges the photometric values Y301-1 to Y301-5 on the rough waveform.

  From FIG. 5, it can be seen that when the photometric values Y301-1 to Y301-5 are drawn on the coarse waveform, the photometric value Y301-5 is particularly deviated (displaced) from the coarse waveform. This indicates that the divided area 301-5 is an area having a photometric value greatly influenced by the subject. Therefore, from the viewpoint of generating a flicker waveform with high accuracy, it is desirable not to use the photometric value Y301-5 and to use the photometric value obtained from the area corresponding to the divided area 301-5 in the images 302 to 306. . Therefore, the ICPU 112 stores the divided area 301-5 as an excluded area when generating the flicker waveform. In addition, in order to obtain the photometric value Y that is most deviated from the rough waveform, it may be determined using a threshold value or the like. Further, depending on the state of the subject, there are cases where all of the photometric values Y301-1 to Y301-5 of each divided area can be used. In that case, it is not necessary to exclude any photometric value Y.

  The luminance value at the position corresponding to the divided area 301-5 is not used for generating the flicker waveform, but the divided area 301-5 is a part of an image acquired in a short period of 1.667 ms. FIG. 8 is a diagram showing a flicker waveform calculated by a conventional method. By capturing a plurality of images and arranging the luminance values, it is possible to avoid that the luminance values cannot be used in all the partial areas of the flicker waveform as in the area 201 in FIG. 8 and to obtain an accurate flicker waveform. Can be scattered in one flicker cycle.

  Next, in step S107, the ICPU 112 selects the next image from which to obtain the photometric value of the divided area. For example, assuming that the images are selected in ascending order of acquisition timing, an image acquired immediately after the currently selected image (for example, the image 302 after the image 301) is selected. In step S108, the ICPU 112 divides the selected image into a plurality of pieces as in step S105. At that time, the image is divided in the same manner as the representative image (the same number of divisions and the same division position). For example, in the example shown in FIG. 6, the image 302 is divided into five divided areas 302-1 to 302-5. In step S107, the ICPU 112 further selects one of the divided areas. For example, assuming that selection is performed in ascending order of acquisition timing, the divided region 302-1 is initially selected.

  Next, in step S109, the ICPU 112 determines whether or not the currently selected divided area is an area corresponding to the excluded area. As a result of the determination, if the currently selected divided area corresponds to the excluded area, the ICPU 112 advances the processing to step S112 because it is affected by the subject and is not suitable for use in flicker generation. . On the other hand, if the currently selected divided area does not correspond to the excluded area, the ICPU 112 calculates a photometric value (luminance value) for the currently selected divided area in step S110.

  Next, in step S111, the ICPU 112 arranges the calculated luminance value so as to be superimposed on the rough waveform, and in step S112, the ICPU 112 sets the photometric values for all the divided areas that do not correspond to the excluded area in the currently selected image. It is determined whether or not the calculation is completed. As a result of the determination, if there is a divided area for which the photometric value has not been calculated, the ICPU 112 returns the process to step S108 and advances the process target to the next divided area. On the other hand, if the calculation of the photometric value is completed for all the divided areas that do not correspond to the excluded area, the ICPU 112 advances the process to step S113. In the example shown in FIG. 6, photometric values Y302-1 to Y302-4 are calculated for the divided areas 302-1 to 302-4 of the image 302. The divided area 302-5 is a divided area that is an excluded area. Since it corresponds to 301-5, a photometric value is not calculated. Thus, the photometric values Y302-1 to Y302-4 are arranged so as to overlap the rough waveform.

  Next, in step S113, the ICPU 112 determines whether or not processing for all images has been completed. If there is an unprocessed image, the processing returns to step S107. By repeating steps S107 to S113, as shown in FIG. 6, photometric values Y303-1 to Y303-4 are similarly calculated for the image 303, and the photometric values are similarly applied to the remaining images 304 to 306. Is calculated and overlaid on the coarse waveform. On the other hand, if the processing for all the images is completed, in step S114, the ICPU 112 generates a fine flicker waveform that is not coarse. That is, as shown in FIG. 7, the ICPU 112 connects the photometric values Y301-1 to Y306-4 for each divided region of all images to generate an accurate flicker waveform. Note that a flicker waveform can be drawn by obtaining an approximate curve that connects all photometric values superimposed on the coarse waveform, and interpolation processing may be used as appropriate.

  According to the processing in FIG. 2, an excluded area is specified based on a photometric value for each divided area of the representative image, and for each divided area excluding those corresponding to the excluded area among the divided areas of images other than the representative image. A photometric value is obtained. Then, based on the photometric value excluding the photometric value to be excluded among the photometric values for each divided area of the representative image, and the photometric value for each divided area excluding those corresponding to the excluded area of the image other than the representative image, A flicker waveform is generated. Thereby, the flicker waveform can be obtained with high accuracy under the flicker light source.

  Next, a description will be given of a method for realizing flicker-less shooting that suppresses variations in shooting timing by using the flicker waveform appropriately obtained as described above.

  FIGS. 9A and 9B are diagrams illustrating examples of flicker waveforms generated under different light sources. FIG. 9A shows a flicker waveform in an environment where the light emission time of the light source is long, and FIG. 9B shows a flicker waveform in an environment where the light emission time of the light source is short.

  In order to realize appropriate flicker-less shooting, it is necessary to perform shooting at a timing when the luminance near the maximum luminance of the flicker waveform is obtained. In the waveform of FIG. 9B, compared to the waveform of FIG. 9A, the period in which the luminance close to the maximum luminance (peak) can be measured is short, and the flickerless shooting period (shootable period) in which flickerless shooting can be realized. ) Is shorter. In order to realize appropriate flickerless shooting, a period in which luminance close to the maximum luminance can be measured is determined as the flickerless shooting period, and shooting is performed in accordance with the period. For this purpose, a period during which the maximum luminance to a predetermined luminance decrease can be measured may be determined as the flickerless shooting period. For example, in the flicker waveform, a period higher than a value lower than the peak by a predetermined value X is determined as the flickerless imaging period. In particular, an appropriate flickerless imaging period can be determined by using an accurate flicker waveform generated by the processing of FIG. If an inaccurate flicker waveform is used, an accurate flickerless shooting period cannot be determined, and the flickerless shooting is not performed even though it is intended to have shot during the flickerless shooting period, or the timing when flickerless shooting can be performed is missed. Therefore, it is important to calculate an accurate flicker waveform using the method described above.

  FIG. 10 is a flowchart of the photographing timing control process. First, in step S201, the ICPU 112 generates and acquires a flicker waveform by the flicker waveform generation processing of FIG. The processing after step S202 is started after the flicker waveform is acquired.

  In step S202, the camera microcomputer 101 determines a flickerless shooting period. As described above, the camera microcomputer 101 determines, as the flickerless imaging period, a period higher than the peak by a predetermined value X in the flicker waveform obtained in step S201 (see FIG. 9A). In the processing after step S203, the camera microcomputer 101 compares the shutter travel period (that is, the shutter open period or shutter time) with the flickerless shooting period, and further considers the release stabilization timer to determine the actual shooting timing. decide.

  By the way, in normal shooting, the timing at which the front curtain of the shutter 104 travels from turning on SW2 instructing the start of the shooting operation by the release button of the operation unit 114 is measured by a release stabilization timer. This time measurement is performed in order to secure a time from the photographing instruction to the permission of exposure so that photographing can be performed in a stable time so as not to cause a variation in photographing timing by the image sensor 103 after SW2 is turned on. When the release stabilization timer expires, the front curtain of the shutter 104 travels, and the rear curtain of the shutter 104 travels according to the shutter speed, so that shooting by the image sensor 103 is realized. However, in flickerless shooting, the influence of flicker cannot be suppressed appropriately unless shooting is performed during the flickerless shooting period. For this reason, it is desirable to prevent the actual shooting start timing and the expiration of the release stabilization timer as much as possible.

  In step S203, the camera microcomputer 101 compares the shutter travel period (shutter opening period) with the flickerless shooting period, and determines whether or not the shutter travel period ≦ flickerless shooting period is satisfied. The shutter travel period (shutter opening period) is a period from the start of the front curtain travel to the completion of the rear curtain travel shown in FIG. As a result of the determination, if the shutter running period ≦ flickerless shooting period does not hold, the shutter running period is longer than the flickerless shooting period, and thus does not fall within the flickerless shooting period (shootable period). Therefore, the camera microcomputer 101 advances the process to step S208 and sets the start timing of the shutter travel period so that the time center of the shutter travel period matches the time center of the flickerless shooting period as shown in FIG. Control. As shown in FIG. 11, the time center from the start of the first curtain to the completion of the second curtain coincides with the time center of the flickerless shooting period. Thereafter, the process of FIG. 10 ends.

  On the other hand, if the result of determination in step S203 is that the shutter running period ≦ flickerless shooting period is satisfied, the shutter running period is not longer than the flickerless shooting period, and therefore can be within the flickerless shooting period. Therefore, it is necessary to determine whether the travel completion timing of the rear curtain of the shutter 104 falls within the flickerless shooting period based on the travel start timing of the front curtain of the shutter 104. Therefore, the camera microcomputer 101 advances the process to step S204.

  In step S204, the camera microcomputer 101 determines whether or not the expiration timing of the release stabilization timer, that is, the expiration timing of the release stabilization period is within the flickerless shooting period. As a result of the determination, if the expiration timing of the release stable period is not included in the flickerless-less shooting period, the camera microcomputer 101 advances the process to step S207. That is, if the front curtain of the shutter 104 travels at the expiration timing of the release stabilization timer, the travel start timing of the front curtain of the shutter 104 deviates forward in the flickerless shooting period. Therefore, in step S207, the camera microcomputer 101 matches the start timing of the shutter travel period with the start timing of the flickerless-less shooting period, as shown in FIG. By controlling in this way, the shutter running period can be included in the flickerless-less shooting period, and the deviation from the expiration timing of the release stabilization timer to the actual shooting start timing can be minimized. Thereby, it is possible to suppress the deviation of the photographing timing to be smaller than when photographing according to the maximum luminance of the flicker waveform. Thereafter, the process of FIG. 10 ends.

  As a result of the determination in step S204, when the expiration timing of the release stabilization period is included in the flickerless less shooting period, there is a possibility that flickerless shooting can be realized at the shooting timing by the original release stabilization timer. This also depends on the travel completion timing of the rear curtain of the shutter 104. Therefore, the camera microcomputer 101 advances the process to step S205. In step S205, the camera microcomputer 101 determines whether or not the shutter travel period from the expiration timing of the release stable period is within the flickerless shooting period. In other words, the camera microcomputer 101 determines whether or not the end timing of the shutter travel period assumed when the shutter travel period starts from the expiration timing of the release stabilization period is included in the flickerless shooting period.

  As a result of the determination in step S205, when the expected shutter travel period end timing is not included in the flickerless shooting period, the timing of completion of the rear curtain of the shutter 104 is the time behind the flickerless shooting period. It will come off. Therefore, the camera microcomputer 101 shifts the process to step S207. In step S207 in this case, the camera microcomputer 101 matches the start timing of the shutter travel period with the start timing of the flickerless-less shooting period. By controlling in this way, it is possible to set the shutter travel period within the flickerless shooting period.

  On the other hand, if the end timing of the assumed shutter travel period is included in the flickerless shooting period as a result of the determination in step S205, flickerless shooting can be realized even if shooting is started at the completion timing of the release stabilization timer. Is possible. Therefore, the camera microcomputer 101 advances the process to step S206, and matches the start timing of the shutter travel period with the expiration timing of the release stable period, as shown in FIG. Thus, actual shooting is started with the original release time lag. More specifically, as shown in FIG. 13, the camera microcomputer 101 causes the front curtain of the shutter 104 to travel at the expiration timing of the release stabilization timer, and causes the rear curtain of the shutter 104 to travel according to the shutter time. Then, both the shutter front curtain travel start timing and the shutter rear curtain travel completion timing are in the flickerless shooting period, and flickerless shooting can be realized without causing variations in shooting timing. Thereafter, the process of FIG. 10 ends.

  According to this embodiment, a flicker waveform can be obtained with high accuracy under a flicker light source. In addition, since the flickerless shooting period (shootable period) is determined based on the accurate flicker waveform obtained by the processing of FIG. 2 and the peak in the flicker waveform, the shooting possible period is determined based on the accurate waveform, The quality of the captured image can be improved. Further, the start timing of the shutter travel period is controlled based on the comparison between the shutter travel period (shutter open period) and the flickerless shooting period by the processing of FIG. Therefore, it is possible to ensure a long actual shooting period in which the shutter running period and the flickerless shooting period overlap. In addition, since the actual shooting timing is determined in consideration of the release stabilization timer, it is possible to secure a long actual shooting period and to suppress variations in shooting timing.

  Note that the number of divided areas identified as excluded areas in step S106 of FIG. 2 is not limited to one, and may be two or more.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included.

101 Camera microcomputer 108 Photometric sensor 112 ICPU

Claims (16)

An imaging apparatus including an imaging element driven by a rolling shutter system,
First acquisition means for acquiring a photometric value for each of a plurality of images acquired by the imaging device;
Selecting means for selecting a representative image from the plurality of images;
First division means for dividing the representative image selected by the selection means into a plurality of regions having different exposure timings;
Second acquisition means for acquiring a photometric value for each divided area divided by the first dividing means;
A specifying unit for specifying a divided region corresponding to a photometric value to be excluded when generating a flicker waveform under a flicker light source based on a photometric value for each divided region acquired by the second acquiring unit;
A second dividing unit that divides each of the plurality of images other than the representative image into a plurality of regions having different exposure timings in correspondence with the divided regions divided by the first dividing unit;
Third acquisition means for acquiring a photometric value for each divided area excluding one corresponding to the divided area specified by the specifying means among the divided areas divided by the second dividing means;
The flicker waveform is generated based on the photometric value excluding the photometric value to be excluded from the photometric value for each divided area of the representative image and the photometric value for each divided area acquired by the third acquisition unit. And an image generating apparatus.   The imaging apparatus according to claim 1, wherein the selection unit selects the representative image based on a photometric value for each of the plurality of images acquired by the first acquisition unit. The generating means generates a rough waveform based on a photometric value for each of the plurality of images,
The imaging apparatus according to claim 2, wherein the selection unit selects, as a representative image, an image corresponding to a region having the smallest degree of change in the coarse waveform generated by the generation unit. The generating means generates a rough waveform based on a photometric value for each of the plurality of images,
The determining means determines a photometric value having a maximum deviation from the rough waveform generated by the generating means among the photometric values for each of the divided regions as the excluded photometric value. The imaging device according to any one of 1 to 3.   5. The imaging apparatus according to claim 1, wherein the plurality of images are acquired at a period higher than a light emission period of the flicker light source.   The imaging apparatus according to claim 1, wherein the plurality of images are acquired in a period corresponding to at least one period of a light emission period assumed by the flicker light source.   2. The first acquisition unit acquires a median value of photometric values of a plurality of lines in the imaging device in each of the plurality of images as a photometric value for each of the plurality of images. The imaging apparatus of any one of -6.   The imaging according to any one of claims 1 to 7, wherein the first dividing unit divides the representative image so as to form one divided region for every predetermined number of lines in the imaging device. apparatus.   The imaging according to any one of claims 1 to 8, further comprising a determination unit that determines a shootable period based on the flicker waveform generated by the generation unit and a peak in the flicker waveform. apparatus.   The imaging apparatus according to claim 9, wherein the determination unit determines a period higher than a value lower than the peak by a predetermined value in the flicker waveform as the imageable period.   11. The imaging apparatus according to claim 9, further comprising a control unit configured to control a start timing of the shutter opening period based on a comparison between the shutter opening period and the photographing enabled period.   When the shutter opening period is longer than the shootable period, the control means sets the start timing of the shutter opening period so that the time center of the shutter open period matches the time center of the shootable period. The imaging apparatus according to claim 11, wherein the imaging apparatus is controlled.   When the shutter opening period is not longer than the shootable period, the control means does not include an expiration timing of a release stable period for securing a time from a shooting instruction to permission of exposure within the shootable period. The image pickup apparatus according to claim 11, wherein a start timing of the shutter opening period is matched with a start timing of the photographing possible period.   In the case where the shutter opening period is not longer than the shootable period, the control means may include an expiration timing of a release stable period for securing a time from a shooting instruction to permission of exposure within the shootable period. Determining whether or not the shutter opening period end timing assumed when the shutter opening period starts from the expiration timing of the release stable period is included in the shootable period, and ending the shutter opening period When the timing is not included in the shootable period, the start timing of the shutter opening period is matched with the start timing of the shootable period, while the end timing of the shutter open period is included in the shootable period. The start timing of the shutter opening period is matched with the expiration timing of the release stable period. The imaging apparatus according to any one of claims 11 to 13, characterized in Rukoto. A method for controlling an image pickup apparatus including an image pickup element driven by a rolling shutter method,
A first acquisition step of acquiring a photometric value for each of a plurality of images acquired by the imaging device;
A selection step of selecting a representative image from the plurality of images;
A first division step of dividing the representative image selected in the selection step into a plurality of regions having different exposure timings;
A second acquisition step of acquiring a photometric value for each of the divided areas divided by the first division step;
A specifying step for specifying a divided region corresponding to a photometric value to be excluded when generating a flicker waveform under a flicker light source based on the photometric value for each divided region acquired in the second acquiring step;
A second dividing step of dividing each of the plurality of images other than the representative image into a plurality of regions having different exposure timings in correspondence with the divided regions divided by the first dividing step;
A third acquisition step of acquiring a photometric value for each divided region excluding one corresponding to the divided region specified by the specifying step among the divided regions divided by the second dividing step;
The flicker waveform is generated based on the photometric value excluding the excluded photometric value among the photometric values for each divided area of the representative image and the photometric value for each divided area acquired by the third acquiring step. A method for controlling the imaging apparatus.   A program for causing a computer to execute the control method for an imaging apparatus according to claim 15.