JP5471004B2 - Focus adjustment apparatus, focus adjustment method, and program - Google Patents

Description

  The present invention relates to a focus adjustment apparatus, a focus adjustment method, and a program.

  Conventionally, in a digital camera that adjusts the in-focus position of a lens using the contrast AF method, since it takes time to switch the focus, the focus is switched for each screen. Specifically, images in which the focus is switched from near focus to far focus are sequentially acquired for each screen, and the in-focus position is specified by calculating the contrast of each image (see, for example, Patent Document 1).

JP-A-2005-352459

  However, in the method of Patent Document 1, since the focus is changed from the near focus to the far focus, the image displayed on the display screen is blurred for a moment. For this reason, the focus adjustment cannot always be performed, and the focus adjustment is performed only when the user performs the adjustment while consciously pressing a predetermined adjustment button. As a result, it takes time to take an in-focus image, and there is a risk of missing the imaging timing.

  Therefore, an object of the present invention is to provide a focus adjustment device, a focus adjustment method, and a program that can display an image in focus even when focus adjustment is performed.

In order to solve the above-described problem, a focus adjustment apparatus according to a first aspect of the present invention includes a first acquisition unit that sequentially acquires a first image captured with a lens focal length in a first state, and the first acquisition. Display means for sequentially displaying the first images sequentially acquired by the means, and during the acquisition of the first image by the first acquisition means, the focal length of the lens is set to a second state different from the first state. Based on the second image acquired by the second acquisition unit and the second acquisition unit for acquiring the second image captured by the second acquisition unit, the lens at the time of imaging the first image by the first acquisition unit A changing means for changing the state of the focal length from the first state to another state based on the second state, and a comparison determination by calculating the contrast of the first image and the second image, respectively. Means for changing When the determination unit determines that the contrast of the second image is better than the contrast of the first image, the state of the focal length of the lens when the first image is captured by the first acquisition unit is The first state is changed to another state based on the second state .

According to a second aspect of the present invention, when the determining means determines that the contrast of the first image is better than the contrast of the second image, the changing means is based on the first acquiring means. It is characterized in that the state of the focal length of the lens at the time of capturing the first image is maintained in the first state.
According to a third aspect of the present invention, the changing unit further includes the first obtaining unit configured to determine that the contrast of the second image is better than the contrast of the first image by the determining unit. The focal length state of the lens at the time of capturing the first image is changed from the first state to the second state.
According to a fourth aspect of the present invention, the display means sequentially displays the first images sequentially acquired by the first acquisition means without displaying the second images acquired by the second acquisition means. It is characterized by displaying.
According to a fifth aspect of the present invention, the first acquisition unit sequentially acquires a first image captured in a state where the focal length of the lens is fixed to the first focal length, and the second acquisition unit , During the acquisition of the first image by the first acquisition means, to acquire a second image that is imaged temporarily with a second focal length different from the first focal length of the lens, Thereafter, the focal length of the lens is returned to the first focal length, and the imaging operation of the first acquisition unit is continued.
According to a sixth aspect of the present invention, the lens is a variable focus lens that changes a focal point according to an applied voltage, and the second acquisition unit is acquiring the first image by the first acquisition unit. In addition, the focal length of the variable focal length lens is changed to temporarily change the focal length of the lens to a second focal length different from the first focal length.
According to a seventh aspect of the present invention, the second acquisition unit acquires a plurality of second images for focus adjustment while changing a focal length of the lens, and the display unit acquires the second focus adjustment second image. The plurality of first images are updated and displayed at predetermined update intervals without displaying images.
The invention described in claim 8 is further characterized by further comprising a focus adjusting means for performing focus adjustment by comparing contrasts of the plurality of second images acquired by the second acquiring means.
The invention according to claim 9 further includes imaging means for sequentially generating image frames by imaging, wherein the first acquisition means and the second acquisition means constitute each image frame generated by the imaging means. Each line of a plurality of pixels in a predetermined direction is sequentially acquired as the first image and the second image according to a predetermined acquisition rule.
The invention according to claim 10 is characterized in that the determination means compares and determines a contrast in a predetermined direction of a line as the first image and a contrast in a predetermined direction of the line as the second image. Yes.
The invention described in claim 11 further includes imaging means for sequentially generating image frames by imaging, wherein the first acquisition means and the second acquisition means are configured to predetermine each image frame generated by the imaging means. According to the acquisition rule, the first image and the second image are sequentially acquired.
According to a twelfth aspect of the present invention, the image displayed on the display means at the display timing of the second image picked up by the image pickup means is generated by interpolation from the first images before and after the second image. It further has a generation means.
According to a thirteenth aspect of the present invention, the changing unit is configured to capture the first image while the plurality of first images are sequentially updated and displayed on the display unit in the still image capturing mode. The focal length of the lens is changed to the focal length when the second image is captured, and the imaging means captures the main image with the focal length of the lens fixed.
The invention described in claim 14 further includes a moving image recording unit that records a moving image including the plurality of first images, and the changing unit is configured to perform the moving image recording unit in the moving image capturing mode. During recording of a moving image, the focal length of the lens at the time of capturing the first image is changed to the focal length at the time of capturing the second image.
The invention described in claim 15 is further characterized in that the acquisition of the first image by the first acquisition unit and the acquisition of the second image by the second acquisition unit are performed alternately and sequentially.
According to a sixteenth aspect of the present invention, the lens is a variable focus lens that changes a focus according to an applied voltage, and the first image and the second image are changed by changing a focus of the variable focus lens. It is characterized by changing the focal length when taking an image.
The invention described in claim 17 further includes management means for storing first management information and second management information relating to a focal length when the first image and the second image are captured, and the management means If the determination means determines that the contrast of the second image is better than the contrast of the first image, the content of the first management information is changed to the focal length when the second image is captured. It is characterized by doing.

In the focus adjustment method according to an aspect of the present invention, the focus adjustment device sequentially acquires a first image captured with a lens focal length in a first state, and the first acquisition process. During display processing for sequentially displaying first images acquired sequentially and acquisition of the first image by the first acquisition processing, imaging is performed with the focal length of the lens in a second state different from the first state. Based on the second acquisition process for acquiring the second image acquired and the second image acquired by the second acquisition process, the focal length of the lens at the time of imaging the first image by the first acquisition process A change process for changing the state from the first state to another state based on the second state, and a determination process for comparing and determining the contrast of the first image and the second image, respectively And executing the change process When it is determined that the contrast of the second image is better than the contrast of the first image by constant processing, the focal length state of the lens at the time of imaging the first image by the first acquisition processing is The first state is changed to another state based on the second state .

According to a nineteenth aspect of the present invention, there is provided a program according to a nineteenth aspect, in which a computer of a focus adjusting apparatus sequentially acquires a first image picked up with a lens focal length in a first state, and the first acquisition unit. Display means for sequentially displaying the first image to be acquired, and during the acquisition of the first image by the first acquisition means, the focal length of the lens was taken in a second state different from the first state Based on the second acquisition unit that acquires the second image and the second image acquired by the second acquisition unit, the state of the focal length of the lens when the first image is captured by the first acquisition unit is determined. , Changing means for changing from the first state to another state based on the second state, calculating contrasts of the first image and the second image, respectively, and functioning as determination means for comparison and determination, Change means When the determination unit determines that the contrast of the second image is better than the contrast of the first image, the state of the focal length of the lens when the first image is captured by the first acquisition unit is The first state is changed to another state based on the second state .

  According to the present invention, even when focus adjustment is performed, it is possible to display a focused image that is captured with the focal length of the lens fixed.

It is a block diagram which shows schematic structure of the imaging device of one Embodiment to which this invention is applied. It is a figure for demonstrating the imaging of the image by the imaging part of the imaging device of FIG. 3 is a flowchart illustrating an example of an operation related to a still image imaging process performed by the imaging apparatus of FIG. 1. 3 is a flowchart illustrating an example of an operation related to a moving image imaging process performed by the imaging apparatus of FIG. 1. 3 is a flowchart illustrating an example of an operation related to a live view display process by the imaging apparatus of FIG. 1. It is a figure for demonstrating an example of the focus adjustment by the imaging device of FIG. It is a figure for demonstrating an example of the focus adjustment by the imaging device of FIG. 14 is a flowchart illustrating an example of an operation related to a live view display process performed by an imaging apparatus according to a modified example.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.
FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment to which the present invention is applied.
The imaging apparatus 100 according to the present embodiment captures a first image (a first image that is one of a plurality of images captured at the same focal length) captured with a lens focal length fixed at a predetermined update interval. A second image captured while updating the live view display and changing the focal length of the lens (a second image that is one of a plurality of images captured differently from the focal length when the first image is captured) ) Is used to adjust the focal length of the lens at the time of capturing the first image when it is determined that the contrast of the second image is better than the contrast of the first image. Change the focal length.
Specifically, as illustrated in FIG. 1, the imaging apparatus 100 includes an imaging unit 1, an imaging control unit 2, an image data generation unit 3, a memory 4, an image processing unit 5, and an information management unit 6. , A display processing unit 7, a display unit 8, a recording medium 9, an operation input unit 10, and a CPU 11.

  The imaging unit 1 captures a subject and generates an image frame as an imaging unit. Specifically, the imaging unit 1 includes a lens unit 1a and an electronic imaging unit 1b.

  The lens unit 1a includes, for example, a plurality of lenses. Specifically, the lens unit 1a includes a variable focus lens (not shown) that changes the focus according to the applied voltage. An example of the variable focus lens is a liquid lens.

The liquid lens has a structure in which two liquids (for example, a predetermined aqueous solution and oil) having different refractive indexes are sealed in a predetermined container without being mixed with each other. In addition, an electrode is attached to a predetermined position of the container, and when a voltage is applied to the electrode from a voltage control unit 2a (described later) of the imaging control unit 2, the interface between the two liquids is curved like a lens. To realize the lens function. Further, the voltage level applied from the voltage controller 2a changes, so that the curvature of the interface between the two liquids changes, and as a result, the focal length of the liquid lens changes.
That is, the characteristics of the liquid lens are that the lens curvature changes according to the applied voltage level, the lens curvature change speed is high, the durability is high because there is no physical moving part, and the voltage level Is a high voltage, but no current flows, so there is little power consumption.

  The electronic imaging unit 1b includes an image sensor such as a CMOS (Complementary Metal-oxide Semiconductor), for example, and converts a subject image that has passed through various lenses of the lens unit 1a into a two-dimensional image signal.

  Although not shown, the imaging control unit 2 includes a timing generator, a driver, and the like. Then, the imaging control unit 2 scans and drives the electronic imaging unit 1b with a timing generator and a driver, converts the optical image into a two-dimensional image signal with the electronic imaging unit 1b at a predetermined period, and the electronic imaging unit 1b. Image frames are read out from the imaging area for each screen and output.

Here, imaging of an image by the CMOS image sensor of the electronic imaging unit 1b will be described.
Unlike a CCD image sensor, a CMOS image sensor performs exposure and transfer for each line in a predetermined direction (for example, horizontal direction) of an image frame. That is, the CMOS image sensor sequentially transfers image signals line by line from the uppermost part of the sensor one line at a time, and completes the lowermost line transfer to transfer one frame, and then the uppermost line again. Transfer sequentially. Specifically, the CMOS image sensor includes a frame synchronization signal that synchronizes data of each image frame, a line synchronization signal that is synchronized with the output timing of the line data of each image frame, output data and a clock for synchronizing with the output data (illustrated). (Omitted) is output to the imaging controller 2 (see FIG. 2A).
Then, the imaging control unit 2 receives the data of each line output from the electronic imaging unit 1 b in synchronization with the frame synchronization signal and the line synchronization signal, and outputs the data to the image data generation unit 3.

In addition, the imaging control unit 2 includes a voltage control unit 2a that controls the voltage applied to the electrode of the liquid lens.
When the voltage control unit 2a receives the line synchronization signal output from the electronic imaging unit 1b, the focal length of the first management information and the second management information defined in the focus adjustment table 6b (described later) is synchronized therewith. The drive voltage applied to the liquid lens is changed by changing the PWM (Pulse Width Modulation) ratio. For example, as shown in FIG. 2A, the voltage control unit 2a switches the drive voltage between “High” corresponding to the focal position A and “Low” corresponding to the focal position B for each line. Thereby, the curvature of the interface between the two liquids of the liquid lens is changed to change the focal length (focal position) of the liquid lens. As a result, the data output from the electronic imaging unit 1b has a different focal length for each line. For example, as shown in FIG. 2B, the capture data of the odd lines constitutes an image (for example, a display image) in which the focus position of the liquid lens is acquired at the focus position A, while capturing the even lines. The data constitutes an image (for example, a focus adjustment image) in which the focus position of the liquid lens is acquired at the focus position B.
In FIG. 2B, an image that is in focus is illustrated as an image acquired at the focal position A, and an image that is not in focus is illustrated as an image acquired at the focal position B. However, it is not limited to these.

  In addition, the imaging control unit 2 performs adjustment control of conditions when imaging a subject such as AE (automatic exposure processing) and AWB (automatic white balance).

The image data generation unit 3 appropriately adjusts the gain for each RGB color component with respect to the analog value signal of the image data transferred from the electronic imaging unit 1b, and then performs sample holding by a sample hold circuit (not shown). The digital signal is converted into digital data by a / D converter (not shown), color processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y and color difference signal Cb , Cr (YUV data).
The luminance signal Y and the color difference signals Cb and Cr output from the color process circuit are DMA-transferred to a memory 4 used as a buffer memory via a DMA controller (not shown).

  The memory 4 is composed of, for example, a DRAM or the like, and temporarily stores data processed by the image processing unit 5 or the CPU 11.

The image processing unit 5 includes an image acquisition unit 5 a that acquires the image data (YUV data) generated by the image data generation unit 3 by imaging the subject by the imaging unit 1 and the imaging control unit 2.
In the live view display process (described later), the image acquisition unit 5a sequentially acquires odd-numbered line image data and even-numbered line image data in each image frame alternately (in accordance with a predetermined acquisition rule).
That is, the image acquisition unit 5a captures an image of an odd-numbered line in each image frame generated by the image data generation unit 3 by imaging a subject in a state where the focus position of the liquid lens is the focus position A (for example, display image Image data) is obtained sequentially. Specifically, for example, the voltage control unit 2a controls the driving voltage of the liquid lens based on the first management information, and displays the focal length of the liquid lens as the display focal length (the focal position of the liquid lens is the focal position). A). Then, the subject image that has passed through the liquid lens is photoelectrically converted by the electronic imaging unit 1b, and the image data generation unit 3 displays the image for display based on the odd-numbered line data in each image frame output from the electronic imaging unit 1b. Image data (an image acquired at the focal position A; see FIG. 2B) is generated. The image acquisition unit 5 a acquires the image data of the display image generated by the image data generation unit 3.
The image acquisition unit 5a also captures image data of even lines in each image frame generated by the image data generation unit 3 by imaging the subject in a state where the focal length of the liquid lens is the focal position B (for example, for focus adjustment). Image data of the image) is acquired sequentially. Specifically, for example, the voltage control unit 2a controls the driving voltage of the liquid lens based on the second management information, and sets the focal length of the liquid lens to the AF focal length (the focal position of the liquid lens is the focal position). B). The subject image that has passed through the liquid lens is photoelectrically converted by the electronic imaging unit 1b, and the image data generation unit 3 performs focus adjustment based on the data of the even lines in each image frame output from the electronic imaging unit 1b. Image data of an image (an image acquired at the focal position B; see FIG. 2B) is generated. The image acquisition unit 5 a acquires the image data of the focus adjustment image generated by the image data generation unit 3.

Note that since the AF focal length of the liquid lens for obtaining the focus adjustment image is not fixed to a predetermined focal length, the image acquisition unit 5a and the image data of the focus adjustment image that is not in focus. The focus adjustment images that are in focus are acquired sequentially.
In other words, the focus position A of the liquid lens related to the acquisition of the display image is fixed after focusing on the subject, so that the focus is always maintained as long as the distance of the subject does not change. When the distance changes, a state where the focus is temporarily out of focus (transient) may also occur. On the other hand, since the focal position B of the liquid lens related to the acquisition of the focus adjustment image constantly changes in order to focus on the subject whose distance changes, the state of being out of focus and the state of being in focus Fluctuates.
Here, the image acquisition unit 5a acquires a plurality of first images (for example, display images) captured in a state where the focal length of the liquid lens is fixed to a predetermined focal length (for example, a display focal length). The 1st acquisition means is comprised. The image acquisition unit 5a also captures a second image (for example, a focus adjustment image) captured at a focal length (for example, an AF focal length) that is different from the focal length when the first image is captured. ) Is acquired at intervals shorter than the display frame rate (update interval). In addition, the image acquisition unit 5a acquires a plurality of second images for focus adjustment (for example, focus adjustment images) while changing the focal length of the liquid lens.

In addition, the image processing unit 5 includes a determination unit 5b that calculates and compares the contrast of a first image (for example, a display image) and a second image (for example, a focus adjustment image).
Based on the image data of each of the first image and the second image, the determination unit 5b specifies the maximum and minimum luminance values of a plurality of pixels constituting the image, and determines the difference as the contrast value of the image. Calculate as Then, the determination unit 5b compares and determines the calculated contrast values of the first image and the second image. For example, the determination unit 5b calculates the contrast value of the odd-line image data as the display image and the contrast value of the even-line image data as the focus adjustment image, and compares and determines these contrast values. To do.

The image processing unit 5 includes an interpolation image generation unit 5c that generates an interpolation image.
The interpolated image generation unit 5c generates an interpolated image from the first images before and after the second image as an image displayed on the display unit 8 at the display timing of the second image. Specifically, the interpolated image generation unit 5c performs an averaging operation on the image data of the odd lines above and below the even line image instead of the even line image (for example, the focus adjustment image) of each image frame. Thus, the image data of the interpolation image is generated.
Here, the interpolation image generation unit 5c constitutes an image generation unit that generates an image to be displayed on the display unit 8 by interpolation from the first image before and after the second image at the display timing of the second image.

The information management unit 6 includes a change control unit 6a that changes the content of the focus adjustment table 6b that manages the focal length of the lens.
The focus adjustment table 6b stores, as management means, first management information and second management information related to the focal length when the display image (first image) and the focus adjustment image (second image) are captured. Yes. For example, the first management information includes a display focal length with the focal position as the focal position A, and the second management information includes an AF focal distance with the focal position as the focal position B.
Then, the change control unit 6a manages the display focal length when the determination unit 5b of the image processing unit 5 determines that the contrast value of the focus adjustment image is better than the contrast value of the display image. The content of the management information is changed to the focal length when the focus adjustment image is captured. As a result, the focal length of the liquid lens when the display image (first image) is captured is changed to the focal length when the focus adjustment image (second image) is captured.
Here, when the determination unit 5b determines that the contrast of the focus adjustment image (second image) is better than the contrast of the display image (first image), the change control unit 6a captures the display image. It functions as a changing means for changing the focal length of the liquid lens at the time to the focal length when the focus adjustment image is captured.

The display processing unit 7 performs control to read out the display image data temporarily stored in the memory 4 and display it on the display unit 8.
Specifically, the display processing unit 7 includes a VRAM, a VRAM controller, a digital video encoder, and the like. The digital video encoder periodically reads the luminance signal Y and the color difference signals Cb and Cr read from the memory 4 and stored in the VRAM (not shown) under the control of the CPU 11 from the VRAM via the VRAM controller. And a video signal is generated based on these data and output to the display unit 8.

  The display unit 8 is, for example, a liquid crystal display device, and displays an image captured by the electronic imaging unit 1b based on a video signal from the display processing unit 7 on a display screen. Specifically, the display unit 8 sequentially updates a plurality of image frames generated by imaging the subject by the imaging unit 1 and the imaging control unit 2 at a predetermined frame rate in the still image capturing mode and the moving image capturing mode. While displaying the live view image. The display unit 8 displays an image (rec view image) recorded as a still image or an image being recorded as a moving image.

  The recording medium 9 is composed of, for example, a non-volatile memory (flash memory) or the like, and recording still image data encoded by a coding unit (not shown) of the image processing unit 5 in a predetermined compression format, or a plurality of recording still images The moving image data composed of image frames is stored.

  The operation input unit 10 is for performing a predetermined operation of the imaging apparatus 100. Specifically, the operation input unit 10 includes a shutter button related to a subject shooting instruction, a selection determination button related to a selection instruction for an imaging mode, a function, and the like, a zoom button related to a zoom amount adjustment instruction, and the like (all illustrated). Abbreviation), a predetermined operation signal is output to the CPU 11 in accordance with the operation of these buttons.

  The CPU 11 controls each part of the imaging device 100. Specifically, the CPU 11 performs various control operations according to various processing programs (not shown) for the imaging apparatus 100.

Next, still image imaging processing by the imaging apparatus 100 will be described with reference to FIG.
FIG. 3 is a flowchart illustrating an example of an operation related to the still image capturing process.
The still image capturing process is executed when a still image capturing mode is selected from a plurality of image capturing modes displayed on the menu screen based on a predetermined operation of the selection determination button of the operation input unit 10 by the user. It is processing. The live view display process in the still image capturing process will be described later with reference to FIG.

First, the CPU 11 causes the information management unit 6 to initialize variables for managing the focal lengths of the first management information and the second management information in the focus adjustment table 6b (step S1A). Specifically, the information management unit 6 sets a predetermined focal length A as an initial value of the first management information in the focus adjustment table 6b, and sets a predetermined focal length B as an initial value of the second management information. .
Next, the CPU 11 initializes the line position related to the position of the line acquired as the first image and the second image, thereby setting the line position acquired as the first image as the line position A (for example, an odd line). At the same time, the line position acquired as the second image is set as a line position B (for example, an even line) (step S2A).

Next, the CPU 11 performs a live view display process for displaying a live view image on the display screen of the display unit 8 based on a plurality of image frames generated by imaging the subject by the imaging unit 1 and the imaging control unit 2 (step S1). S3A; see FIG. 5, details will be described later).
Then, during the live view display process, the CPU 11 determines whether a still image capturing instruction has been input based on a predetermined operation of the shutter button of the operation input unit 10 by the user (step S4A).

If it is determined that the still image capturing instruction is not input (step S4A; NO), until it is determined in step S4A that the still image capturing instruction is input (step S4A; YES). The CPU 11 shifts the process to step S3A and repeatedly performs the live view display process.
When it is determined that a still image imaging instruction has been input (step S4A; YES), the CPU 11 causes the imaging control unit 2 to display the focal length of the liquid lens based on the focus adjustment table 6b (focal length for display). A) is changed (step S5A), and exposure conditions (shutter speed, aperture, amplification factor, etc.) and conditions such as white balance are adjusted, and an optical image of the subject is captured by the electronic imaging unit 1b under predetermined conditions.
Thereafter, the CPU 11 causes the image data generation unit 3 to generate YUV data of an image frame for recording a still image transferred from the electronic imaging unit 1 b, and stores the YUV data of the image in a predetermined storage area of the recording medium 9. (Step S6A).
Thus, the still image capturing process is terminated.

Next, moving image imaging processing by the imaging apparatus 100 will be described with reference to FIG.
FIG. 4 is a flowchart illustrating an example of an operation related to the moving image capturing process.
The moving image capturing process is a process executed when a moving image capturing mode is instructed from among a plurality of image capturing modes displayed on the menu screen based on a predetermined operation of the selection determination button of the operation input unit 10 by the user. is there. The live view display process in the moving image capturing process will be described later with reference to FIG.

First, the CPU 11 causes the information management unit 6 to initialize variables for managing the focal lengths of the first management information and the second management information in the focus adjustment table 6b (step S1B). Specifically, the information management unit 6 sets a predetermined focal length A as an initial value of the first management information in the focus adjustment table 6b, and sets a predetermined focal length B as an initial value of the second management information. .
Next, the CPU 11 initializes the line position related to the position of the line acquired as the first image and the second image, thereby setting the line position acquired as the first image as the line position A (for example, an odd line). At the same time, the line position acquired as the second image is set as the line position B (for example, even line) (step S2B).

Next, the CPU 11 performs a live view display process for displaying a live view image on the display screen of the display unit 8 based on a plurality of image frames generated by imaging the subject by the imaging unit 1 and the imaging control unit 2 (step S1). S3B; see FIG. 5, details will be described later).
Then, during the live view display process, the CPU 11 determines whether a moving image imaging start instruction is input based on a predetermined operation of the shutter button of the operation input unit 10 by the user (step S4B).

If it is determined that the moving image capturing start instruction is not input (step S4B; NO), until it is determined in step S4B that the moving image capturing start instruction is input (step S4B; YES). The CPU 11 shifts the process to step S3B and repeatedly performs the live view display process.
When it is determined that a moving image imaging start instruction has been input (step S4B; YES), the CPU 11 causes the imaging control unit 2 to adjust conditions such as exposure conditions (shutter speed, aperture, amplification factor, etc.) and white balance. Then, the image data generation unit 3 generates YUV data of an image frame related to the first image of the subject imaged by the electronic imaging unit 1b under a predetermined condition, and is additionally recorded in the memory 4 as an image frame constituting a moving image. (Step S5B).

Next, the CPU 11 determines whether or not a moving image capturing end instruction is input based on a predetermined operation of the shutter button of the operation input unit 10 by the user (step S6B).
If it is determined that the moving image capturing end instruction has not been input (step S6B; NO), until it is determined in step S6B that the moving image capturing end instruction has been input (step S6B; YES). The CPU 11 shifts the process to step S3B and repeatedly performs the processes after the live view display process.
If it is determined that a moving image capturing end instruction has been input (step S6B; YES), the CPU 11 stores a moving image composed of a plurality of image frames recorded in the memory 4 in a predetermined storage area of the recording medium 9. (Step S7B).
Thereby, the moving image capturing process is completed.

Next, live view display processing in still image capturing processing and moving image capturing processing will be described with reference to FIG.
FIG. 5 is a flowchart illustrating an example of an operation related to the live view display process.
The live view display processing is substantially the same between the processing content executed in the still image capturing processing and the processing content executed in the moving image capturing processing, and will be described below.

  As shown in FIG. 5, first, the CPU 11 causes the voltage control unit 2a of the imaging control unit 2 to change the PWM ratio according to the focal length A of the first management information in the focus adjustment table 6b, and apply it to the liquid lens. A drive voltage (for example, “High”) is set. Then, the voltage control unit 2a applies the set drive voltage to the electrodes, changes the curvature of the interface between the two liquids of the liquid lens, and changes the focal length of the liquid lens to the focal length A (step S11). ). When the subject image that has passed through the liquid lens is photoelectrically converted by the electronic imaging unit 1b and the odd-numbered line data corresponding to the line position A output from the electronic imaging unit 1b is input to the image data generation unit 3, The data generation unit 3 generates YUV data from the odd line data. Then, the image acquisition unit 5a acquires the image data of the odd lines generated by the image data generation unit 3 (Step S12).

  Next, the CPU 11 causes the voltage control unit 2a of the imaging control unit 2 to change the PWM ratio in accordance with the focal length B of the second management information in the focus adjustment table 6b, so that the drive voltage (for example, “ Low ”). Then, the voltage control unit 2a applies the set drive voltage to the electrodes, changes the curvature of the interface between the two liquids of the liquid lens, and changes the focal length of the liquid lens to the focal length B (step S13). ). When the subject image that has passed through the liquid lens is photoelectrically converted by the electronic imaging unit 1b, and even line data corresponding to the line position B output from the electronic imaging unit 1b is input to the image data generation unit 3, an image is generated. The data generation unit 3 generates YUV data from the even line data. And the image acquisition part 5a acquires the image data of the even line produced | generated by the image data generation part 3 (step S14).

Thereafter, the CPU 11 causes the imaging control unit 2 to change the line position A related to the acquisition of the first image to the next line and change the line position B related to the acquisition of the second image to the next line (step S15). ).
Next, the CPU 11 causes the image acquisition unit 5a to determine whether or not one frame of the first image and the second image have been acquired (step S16). If it is determined that one frame has not been acquired (step S15; NO), the CPU 11 shifts the process to step S11.
By repeating the above processing until it is determined in step S16 that one frame has been acquired (step S16; YES), the image acquisition unit 5a acquires the first image and the second image for one frame. .

Next, the CPU 11 branches the process according to the contrast calculation mode (step S17). Specifically, when the contrast calculation mode is the normal mode, the CPU 11 causes the determination unit 5b of the image processing unit 5 to contrast all vertical and horizontal pixels in the AF area (focused position) of the second image. While the value is calculated (step S18), if the contrast calculation mode is the horizontal mode, the contrast value of the pixel in the horizontal line in the AF area of the second image is calculated (step S19).
Thereafter, the CPU 11 causes the determination unit 5b to compare the calculated contrast value with the contrast value of the second image in the immediately preceding image frame. Then, based on the comparison result, the information management unit 6 updates the focal length B of the second management information by adopting the focal length with the better contrast value so that the contrast is improved (step S20). Thus, when the next second image is captured, the voltage control unit 2a sets the drive voltage applied to the liquid lens according to the focal length B of the second management information in the focus adjustment table 6b, and the liquid lens By applying to the electrode, the focus of the liquid lens can be adjusted.
As described above, the CPU 11, the determination unit 5b of the image processing unit 5, the information management unit 6, and the voltage control unit 2a perform focus adjustment by comparing the contrasts of the plurality of second images acquired by the image acquisition unit 5a. The focus adjustment unit is configured.

Subsequently, the CPU 11 causes the interpolation image generation unit 5c to generate an interpolation image from the odd lines before and after the even line, and then causes the display processing unit 7 to perform display based on the image data of the first image and the interpolation image. Is displayed on the display screen of the display unit 8 (step S21).
During the live view display process, the display unit 8 repeatedly displays an image for display. That is, the display unit 8 displays a display image made up of a plurality of first images and interpolated images while updating them at a predetermined update interval without displaying the second image for focus adjustment.

Next, the CPU 11 causes the determination unit 5b to calculate the contrast value of the AF area of the first image according to the set contrast calculation mode, and obtains the calculated contrast value and the contrast value of the second image. Let them compare. Then, based on the comparison result, the information management unit 6 updates the focal length A of the first management information by adopting the focal length having the better contrast value so that the contrast of the first image is improved (Step S1). S22).
Thereby, the live view display process is terminated.

  In the above-described still image capturing process and moving image capturing process, for example, as illustrated in FIGS. 6A to 6C, the object A is captured while the object A and the object B are captured as subjects with the center-weighted AF. When the object A is in the substantially central portion within the angle of view, the object A is selected as an AF target and the object A is in focus. At this time, the focal length A (display focal length) and the focal length B (AF focal length) of the first management information and the second management information defined in the focus adjustment table 6b are equal (FIG. 6). (See (a)).

  Here, if the orientation of the imaging apparatus 100 is changed and the object B is put in a substantially central portion within the angle of view, the object B enters the AF area, and therefore the object B is selected as an AF target. Then, while the focal length A (display focal length) of the liquid lens is fixed, the focal length B (AF focal length) of the second management information defined in the focus adjustment table 6b is changed, so that the liquid An AF operation for focusing on the object B by changing the focal length B of the lens is performed (see FIG. 6B).

  The contrast of the first image captured at the focal length A (display focal length) is compared with the contrast of the second image captured at the focal length B (AF focal length). As a result of the determination, if the contrast of the second image captured at the focal length B is higher than the contrast of the first image captured at the focal length A, the first specified in the focus adjustment table 6b. The focal length A of the management information is changed to the focal length B of the second management information so that the focal length A (display focal length) and the focal length B (AF focal length) are equal (see FIG. 6C). ).

  Further, in the above-described still image capturing process and moving image capturing process, for example, as shown in FIGS. 7A to 7C, the focus adjustment table is used when a predetermined object is being captured as a subject in multi-area AF. The focal length A (display focal length) and the focal length B (AF focal length) of the first management information and the second management information defined in 6b are equal (see FIG. 7A).

  Here, the focal length A (display focal length) of the liquid lens is fixed when a person with higher priority than the background object (regardless of whether or not a person is registered) enters the angle of view. In this state, by changing the focal length B (AF focal length) of the second management information defined in the focus adjustment table 6b, the AF operation for changing the focal length B of the liquid lens to focus on the person (See FIG. 7B).

The contrast of the first image captured at the focal length A (display focal length) is compared with the contrast of the second image captured at the focal length B (AF focal length). As a result of the determination, if the contrast of the second image captured at the focal length B is higher than the contrast of the first image captured at the focal length A, the first specified in the focus adjustment table 6b. The focal length A of the management information is changed to the focal length B of the second management information so that the focal length A (display focal length) is equal to the focal length B (AF focal length) (see FIG. 7C). ).
In this example, the case where a person with a higher priority than the background object has entered the angle of view with the camera fixed, is provided at the center of the angle of view by panning the camera. The same applies to the case where the subject in the selected AF area changes (that is, a change from an object to a person, a change from an object to an object, a change from a person to a person, etc.).

As described above, according to the imaging apparatus 100 of the present embodiment, it is possible to update a display image captured in a state where the focal length of the liquid lens is fixed at a predetermined update interval and perform live view display. The focus adjustment can be performed using the second image for focus adjustment that is captured while changing the focal length of the liquid lens (different from the focal length when the first image is captured). When the determination unit 5b determines that the contrast of the first image is better than the contrast of the first image, the change control unit 6a determines the focal length of the liquid lens when the first image is captured when the second image is captured. The focal length can be changed.
That is, in the conventional focus adjustment lens, since the moving speed is slow, it is necessary to change the focus for each image frame. Therefore, when the focus adjustment sequence is executed by half-pressing the shutter button or the like, the number of focus confirmation points is increased. It takes time to acquire the image frame. In addition, during the focus adjustment process, the image display changes accordingly, and the image is blurred. On the other hand, in the imaging apparatus 100 according to the present embodiment, the focal length at the time of capturing the first image and the second image is quickly changed by using a liquid lens as a variable focus lens and changing its focus. Can do. As a result, the first image with a fixed focus is always displayed, and the second image for focus adjustment is acquired behind the first image (in parallel in time division), so that the focus can be adjusted without changing the image display. Adjustment can be performed, and the focal length of the first image for display is changed only when a focal length that is in focus is found. Furthermore, since the focus adjustment process can be continuously performed, a focus adjustment action such as half-pressing the shutter button is not necessary, and the first image that is always in focus can be displayed when viewed from the user.
Accordingly, only an image captured with the liquid lens having a fixed focal length can be displayed. Therefore, even when focus adjustment is performed, a temporary image during adjustment of the focus position by the contrast method is displayed. Blurring can be reduced, and a phenomenonally focused image can always be displayed.

Note that the operation of acquiring the second image for focus adjustment performed behind the display of the first image with the fixed focus may be repeatedly performed at regular intervals (in a short time) or intermittently. It may be performed (non-periodically).
Here, when the acquisition of the second image for focus adjustment is repeated at regular intervals, continuous automatic focus adjustment is executed by comparing the contrast between the acquired second images. You may make it do. In addition, when acquiring the second image for focus adjustment intermittently, the focus is directly adjusted by comparing the contrast between the first image and the second image without executing continuous automatic focus adjustment on the back side. You may make it perform adjustment.

Further, in the still image capturing mode, while the plurality of first images are sequentially updated and displayed on the display unit 8, the focal length of the liquid lens when the first image is captured is the focal point when the second image is captured. The main image can be picked up in a state where the distance is changed and the focal length of the liquid lens is fixed, and the first focal length is adjusted on the display unit 8 without displaying the second image for focus adjustment. Only an image can be displayed, and a focused image can be taken as a still image.
Further, in the moving image capturing mode, during the recording of a moving image composed of a plurality of image frames, the focal length of the liquid lens at the time of capturing the first image may be changed to the focal length at the time of capturing the second image. It is possible to capture an image with a suitable focal length as a moving image.
Therefore, it is possible to always perform focus adjustment and display of an in-focus image by reducing temporary blurring of the image during adjustment of the focus position by the contrast method.

Further, the electronic imaging unit 1b is composed of a CMOS image sensor, the image acquisition unit 5a acquires each line of each image frame alternately as a first image and a second image, respectively, and the determination unit 5b Since the contrast in the predetermined direction of the line as one image is compared with the contrast in the predetermined direction of the line as the second image, the focal length of the liquid lens at the time of imaging the first image is determined according to the determination result. It is possible to change the focal length when the image is captured, and it is possible to obtain a line related to the second image for focus adjustment at an interval shorter than the update display interval of the image frame, so that the focus adjustment is performed. It can be done more quickly.
At this time, since the image displayed on the display unit 8 at the display timing of the line related to the second image is generated by interpolation from the lines related to the first image before and after the second image, the line corresponding to the second image is also displayed. It is possible to prevent the resolution of the display image composed of a plurality of lines related to the first image from being lowered.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, the contents of the live view display process can be arbitrarily changed as appropriate. As shown in FIG. 8, when the electronic image pickup unit 1b is composed of a CMOS image sensor, each line is used as a first image and a second image. Each time image data is acquired, contrast comparison determination and focal length change may be performed.
That is, in the live view display process shown in FIG. 8, the image data of odd lines and even lines is acquired and the line positions A and B are changed as in the live view display process shown in FIG. After (steps S11 to S15), the determination unit 5b calculates the contrast value of the pixel at the line position B of the second image (step S51). Thereafter, the determination unit 5b compares the calculated contrast value with the contrast value at the same line position in the immediately preceding image frame, and based on the comparison result, the information management unit 6 contrasts so that the contrast is improved. The focal length B with the better value is adopted to update the focal length B of the second management information (step S52).

Next, the determination unit 5b calculates the contrast value of the pixel at the line position A of the first image, compares the calculated contrast value with the contrast value of the second image, and based on the comparison result, The information management unit 6 updates the focal length A of the first management information by adopting the focal length with the better contrast value so that the contrast of the first image is improved (step S53).
Subsequently, the CPU 11 determines whether it is an image display update timing (step S54). If it is determined that it is not the display update timing (step S54; NO), the CPU 11 shifts the process to step S11.
By repeating the above processing until it is determined in step S54 that the display update timing is reached (step S54; YES), the information management unit 6 makes the first management information so that the contrast of the first image is improved. The update of the focal length A is continued.
If it is determined that it is the display update timing (step S54; YES), the CPU 11 causes the interpolation image generation unit 5c to generate an interpolation image from the odd lines before and after the even lines, and then causes the display processing unit 7 to Based on the image data of the first image and the interpolated image, a display image is displayed on the display screen of the display unit 8 (step S55).
Thereby, the live view display process is terminated.

  Therefore, when the electronic image pickup unit 1b is composed of a CMOS image sensor, the focus adjustment can be appropriately performed without acquiring all the lines of the image frame, and the focus adjustment process can be reduced.

Furthermore, in the above embodiment, the electronic imaging unit 1b is configured from a CMOS image sensor, but may be configured from a CCD image sensor. In this case as well, the electronic imaging unit 1b is displayed on the display unit 8 at the display timing of the second image. The generated image is interpolated from the first image before and after the second image, so that the frame rate of the display image can be prevented from decreasing.
If it is determined by contrast comparison between the first image and the second image that the second image has better contrast, the second image (an image captured while changing the focal length of the liquid lens) is selected. You may make it display. That is, when displaying an in-focus image, it is not always necessary to display the first image captured with the liquid lens having a fixed focal length, and the in-focus second image with high contrast is displayed. You may make it display.

  In addition, the configuration of the imaging apparatus 100 is merely an example illustrated in the above embodiment, and is not limited thereto. In other words, the imaging apparatus 100 is exemplified as the focus adjustment apparatus, but is not limited thereto. For example, the first image and the second image are picked up by an image pickup device different from the image pickup device 100, and only the image data transferred from the image pickup device is recorded and only the focus adjustment processing is executed. It may be a device.

In addition, in the above-described embodiment, the functions of the first acquisition unit, the display unit, the second acquisition unit, the determination unit, and the change unit are controlled by the CPU 11 in the image acquisition unit 5a of the image processing unit 5. The determination unit 5b, the change control unit 6a of the information management unit 6 and the display unit 8 are driven. However, the present invention is not limited to this, and a predetermined program or the like is executed by the CPU 11. It is good also as a structure implement | achieved by.
That is, a program including a first acquisition processing routine, a display control processing routine, a second acquisition processing routine, a determination processing routine, and a change processing routine is stored in a program memory (not shown) that stores the program. Then, the first acquisition processing routine may cause the CPU 11 to acquire a plurality of first images captured in a state where the focal length of the lens is fixed. Further, a plurality of first images may be updated and displayed at a predetermined update interval on the CPU 11 by a display control processing routine. In addition, the second acquisition process routine may cause the CPU 11 to generate a synthesized moving image in which the second image captured with the lens focal length different from the focal length when the first image is captured is acquired. Further, the contrast of the first image and the second image may be calculated by the CPU 11 by the determination processing routine, and the comparison determination may be made. Further, when the CPU 11 determines that the contrast of the second image is better than the contrast of the first image by the change processing routine, the focal length of the lens at the time of capturing the first image is captured. The focal length may be changed.

DESCRIPTION OF SYMBOLS 100 Imaging device 1 Imaging part 1a Lens part 1b Electronic imaging part 5 Image processing part 5a Image acquisition part 5b Determination part 5c Complementary image generation part 6 Information management part 6a Change control part 6b Focus adjustment table 8 Display part 11 CPU

Claims (19)

First acquisition means for sequentially acquiring first images picked up with a focal length of the lens in a first state;
Display means for sequentially displaying the first images sequentially obtained by the first obtaining means;
A second acquisition unit configured to acquire a second image captured by setting a focal length of a lens in a second state different from the first state during acquisition of the first image by the first acquisition unit;
Based on the second image acquired by the second acquisition unit, the focal length of the lens when the first image is captured by the first acquisition unit is changed from the first state to the second state. Change means for changing to another state based on the state;
Determining means for calculating and comparing the contrast of the first image and the second image, respectively;
With
The changing unit is configured such that when the determination unit determines that the contrast of the second image is better than the contrast of the first image, the focus of the lens when the first acquisition unit captures the first image. A focus adjusting apparatus that changes a distance state from the first state to another state based on the second state . When the determining unit determines that the contrast of the first image is better than the contrast of the second image, the changing unit is configured to change the lens at the time of capturing the first image by the first acquiring unit. The focus adjustment apparatus according to claim 1 , wherein a focal length state is maintained in the first state. The changing unit is configured such that when the determination unit determines that the contrast of the second image is better than the contrast of the first image, the focus of the lens when the first acquisition unit captures the first image. The focus adjustment apparatus according to claim 2 , wherein the distance state is changed from the first state to the second state. The display means sequentially displays the first images sequentially acquired by the first acquisition means without displaying the second images acquired by the second acquisition means . 4. The focus adjustment device according to any one of 3 . The first acquisition means sequentially acquires a first image captured in a state where the focal length of the lens is fixed to the first focal length,
The second acquisition unit is configured to temporarily capture a first focal length of a lens that is different from the first focal length during acquisition of the first image by the first acquisition unit. the two images acquired, then, according to the focal length of the lens to any one of claims 1 to 4, characterized in that to continue the imaging operation of the first acquisition means is returned to said first focal length Focus adjustment device. The lens is a variable focus lens that changes a focus according to an applied voltage,
The second acquisition unit temporarily changes the focal length of the lens by changing the focal point of the variable focus lens during the acquisition of the first image by the first acquisition unit. The focus adjusting apparatus according to claim 5 , wherein the second focal lengths are different from each other. The second acquisition means includes
Acquire multiple second images for focus adjustment while changing the focal length of the lens,
The display means includes
The focus according to any one of claims 1 to 5 , wherein the plurality of first images are updated and displayed at a predetermined update interval without displaying the second image for focus adjustment. Adjustment device. The focus adjustment apparatus according to claim 7 , further comprising a focus adjustment unit that performs focus adjustment by comparing contrasts of the plurality of second images acquired by the second acquisition unit. It further comprises imaging means for sequentially generating image frames by imaging,
The first acquisition means and the second acquisition means are:
Claims, characterized in that sequentially acquires each as the first image and the second image according to a predetermined acquisition rule in a predetermined direction of each line comprising a plurality of pixels constituting each image frame generated by said image pickup means The focus adjustment apparatus according to any one of 1 to 3 . The determination means includes
The focus adjustment apparatus according to claim 9 , wherein the contrast in the predetermined direction of the line as the first image is compared with the contrast in the predetermined direction of the line as the second image. It further comprises imaging means for sequentially generating image frames by imaging,
The first acquisition means and the second acquisition means are:
The focus adjustment according to any one of claims 1 to 9 , wherein each image frame generated by the imaging unit is sequentially acquired as the first image and the second image according to a predetermined acquisition rule. apparatus. Claims and further comprising an image generating means for interpolating generates an image displayed on the display unit at a display timing of the imaged the second image from the first image of the front and rear of the second image on the imaging device Item 12. The focus adjustment apparatus according to any one of Items 9 to 11 . The changing means is
In still image capturing mode, while the plurality of first images are sequentially updated and displayed on the display means, the focal length of the lens when capturing the first image is the focus when capturing the second image. Change to distance,
The imaging means includes
The focus adjustment apparatus according to claim 9 , wherein the main image is captured in a state where the focal length of the lens is fixed. A moving image recording means for recording a moving image composed of the plurality of first images;
The changing means is
In the moving image capturing mode, during the recording of the moving image by the moving image recording means, the focal length of the lens when capturing the first image is changed to the focal length when capturing the second image. The focus adjustment apparatus according to any one of claims 9 to 13 , wherein the focus adjustment apparatus is characterized by the following. The focus adjustment according to claim 1 , wherein acquisition of the first image by the first acquisition unit and acquisition of the second image by the second acquisition unit are alternately performed sequentially. apparatus. The lens is
It is a variable focus lens that changes the focus according to the applied voltage,
The focal point according to any one of claims 1 to 15 , wherein a focal length at the time of capturing the first image and the second image is changed by changing a focal point of the variable focus lens. Adjustment device. A management unit for storing first management information and second management information related to a focal length when the first image and the second image are captured;
The management means includes
When the determination unit determines that the contrast of the second image is better than the contrast of the first image, the content of the first management information is changed to a focal length when the second image is captured. The focus adjusting apparatus according to any one of claims 1 to 16 , wherein Focus adjustment device
A first acquisition process for sequentially acquiring a first image picked up with a lens focal length in a first state;
Display processing for sequentially displaying the first images sequentially acquired by the first acquisition processing;
A second acquisition process for acquiring a second image picked up in a second state different from the first state during the acquisition of the first image by the first acquisition process;
Based on the second image acquired by the second acquisition process, the focal length state of the lens at the time of capturing the first image by the first acquisition process is changed from the first state to the second state. Change process to change to other state based on state,
A determination process for calculating and comparing the contrast of the first image and the second image, respectively
And execute
In the changing process, when the determination process determines that the contrast of the second image is better than the contrast of the first image, the focus of the lens when the first image is captured by the first acquisition process is determined. A focus adjustment method , wherein the distance state is changed from the first state to another state based on the second state . The computer of the focus adjustment device,
First acquisition means for sequentially acquiring first images picked up with a focal length of the lens in a first state;
Display means for sequentially displaying the first images sequentially obtained by the first obtaining means;
A second acquisition means for acquiring a second image picked up with a focal length of a lens in a second state different from the first state during the acquisition of the first image by the first acquisition means;
Based on the second image acquired by the second acquisition unit, the focal length of the lens when the first image is captured by the first acquisition unit is changed from the first state to the second state. Change means to change to another state based on the state,
Determination means for comparing and determining the contrast of the first image and the second image, respectively.
Function as
The changing unit is configured such that when the determination unit determines that the contrast of the second image is better than the contrast of the first image, the focus of the lens when the first acquisition unit captures the first image. A program for changing a distance state from the first state to another state based on the second state .

Images