JP5610889B2 - Focus adjustment apparatus and method - Google Patents

Description

The present invention relates to a focus adjustment apparatus and method, and more particularly to a focus adjustment apparatus and method for detecting a subject such as a person, an animal, or an object included in an image or a part of the subject and automatically adjusting the focus.

  2. Description of the Related Art Conventionally, there are imaging devices such as a digital camera and a digital video camera that detect a specific subject such as a face from a photographed image and optimize the focus and exposure on the detected subject. For example, Patent Document 1 describes a photographing apparatus that detects the position of a face in an image and focuses the detected face (hereinafter referred to as “face AF”).

JP 2009-20190 A

  However, in Patent Document 1, when a face is detected, the face is focused, and when a face is not detected, the AF area set at a predetermined position on the screen is focused. An example of a moving image obtained by this control is shown in FIG. FIG. 8 shows a focus state when a subject enters and exits the screen during shooting.

  In FIG. 8A, since the face is not detected, the building in the normal frame is focused. FIGS. 8B and 8C show a state where a subject has entered the screen and a face has been detected. While the face is being detected, the face within the face frame is kept focused. To do. FIGS. 8D and 8E show an image after the subject has left the screen, and gradually focus on the building in the normal frame again.

  When editing a moving image, it is preferable that the edit cut point is a timing at which there is no movement in the image. However, even if an attempt is made to cut and edit a moving image as shown in FIG. 8, an image having a focus shift (for example, between FIGS. 8 (a)-(b), FIG. 8 (c)- (Between (d) and (e)).

  The present invention has been made in view of the above problems, and in the AF processing for detecting and focusing on a desired subject, the focus is not automatically moved before and after shooting of the desired subject. With the goal.

In order to achieve the above object, the focus adjustment apparatus of the present invention includes a detection unit that detects a subject having a predetermined characteristic from an image obtained during shooting of a moving image, and a focus based on the image. In a state in which the subject is detected by the detection means, the autofocus means for moving the lens to adjust the focus, the manual focus means for moving the focus lens based on the operation of the focus operation unit by the photographer, and the detection means In a state where the autofocus means is selected and the subject is not detected by the detection means, the detection is performed even in the first AF mode in which the manual focus means is selected and in the state where the subject is detected by the detection means. Even when the subject is not detected by the means, the autofocus means is selected. Setting means for selectively setting the second AF mode, and in the first AF mode, the autofocus means applies the detected subject to the detected subject when the subject is detected by the detection means. A corresponding AF area is set, and the focus lens is moved based on an image in the AF area to perform focus adjustment. When the object is not detected by the detection means, the manual focus means If the focus operation unit is not operated, the focus lens is not moved, and in the second AF mode, the autofocus unit moves the detected subject to the detected subject when the detection unit detects the subject. A corresponding AF area is set, and the focus lens is moved based on the image in the AF area. When the adjustment is performed and the subject is not detected by the detection means, the autofocus means sets an AF area in the image and moves the focus lens based on the image in the AF area. The focus adjustment is performed .

  According to the present invention, it is possible to prevent the focal point from automatically moving before and after the photographing of the desired subject in the AF processing for detecting and focusing the desired subject.

The block diagram which shows schematic structure of the video camera concerning the 1st and 2nd embodiment of this invention. 5 is a flowchart illustrating a focus adjustment control process according to the first embodiment of the present invention. The figure which shows an example of the image obtained by the "face only AF" of this invention. The flowchart for demonstrating the main face determination process in the 2nd Embodiment of this invention. The figure explaining an example of the determination result of the main face determination in the 2nd Embodiment of this invention. The block diagram which shows schematic structure of the video camera concerning the 3rd Embodiment of this invention. 10 is a flowchart illustrating focus adjustment control processing according to the third embodiment of the present invention. The figure which shows an example of the image obtained by the conventional "face AF".

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

<First Embodiment>
FIG. 1 is a block diagram illustrating a schematic configuration of a video camera as an example of an imaging apparatus according to an embodiment of the present invention. In the first embodiment, a video camera will be described. However, the present invention can also be applied to other imaging apparatuses that can capture moving images, such as a digital still camera.

  In FIG. 1, 101 is a focus ring, 102 is a first fixed lens, 103 is a variable power lens that moves in the direction of the optical axis and performs variable power, and 104 is a stop. Reference numeral 105 denotes a second fixed lens, and reference numeral 106 denotes a focus compensator lens (hereinafter referred to as “focus lens”) that has both a function of correcting the movement of the focal plane due to zooming and a focusing function. The first fixed lens 102, the variable magnification lens 103, the diaphragm 104, the second fixed lens 105, and the focus lens 106 constitute an imaging optical system.

  Reference numeral 107 denotes an image sensor as a photoelectric conversion element constituted by a CCD sensor or a CMOS sensor. Reference numeral 108 denotes a CDS / AGC circuit that samples the output of the image sensor 107 and adjusts the gain.

  Reference numeral 109 denotes a camera signal processing circuit that performs various types of image processing on the output signal from the CDS / AGC circuit 108 to generate a video signal. The camera signal processing circuit 109 includes a character generator that superimposes characters and marks on the screen in accordance with the control of a camera / AF microcomputer (hereinafter referred to as “camera / AF microcomputer”) 118 described later. Yes. Reference numeral 110 denotes a monitor constituted by an LCD or the like, which displays a video signal from the camera signal processing circuit 109 and characters / marks. A recording unit 111 records the video signal from the camera signal processing circuit 109 on a recording medium such as a magnetic tape, an optical disk, or a semiconductor memory.

  Reference numeral 112 denotes a focus ring encoder for detecting the rotation of the focus ring 101, and reference numeral 113 denotes a zoom drive source for moving the variable magnification lens 103. Reference numeral 114 denotes a focusing drive source for moving the focus lens 106. The zoom drive source 113 and the focusing drive source 114 are composed of actuators such as a stepping motor, a DC motor, a vibration motor, and a voice coil motor.

  Reference numeral 115 denotes an AF gate which passes only signals in an area within an AF frame used for focus detection, which is set by the camera / AF microcomputer 118 as will be described later, among image signals of all pixels from the CDS / AGC circuit 108. is there. The AF signal processing circuit 116 extracts a high frequency component, a luminance difference component (difference between the maximum value and the minimum value of the luminance level of the signal that has passed through the AF gate 115), and the like from the image signal that has passed through the AF gate 115, and the AF evaluation value. A signal is generated and output to the camera / AF microcomputer 118. The AF evaluation value signal represents the sharpness (contrast state) of the image generated based on the output signal from the image sensor 107, but the sharpness changes depending on the focus state of the image pickup optical system. The signal represents the focus state of the imaging optical system.

  The face detection unit 117 performs a known face detection process on the output signal from the CDS / AGC circuit 108 to detect a human face area in the shooting screen. The detection result is transmitted to the camera / AF microcomputer 118. As face detection processing, for example, there is a method of extracting a skin color region from the gradation color of each pixel represented by image data and detecting the face with a matching degree with a face contour plate prepared in advance. In addition, a method of performing face detection by extracting facial feature points such as eyes, nose and mouth using a known pattern recognition technique is disclosed. Note that the present invention is not limited to the above-described method for the face detection processing method, and any method may be used.

  The AF mode switch 119 is a switch for the photographer to select one of “normal AF”, “face AF”, and “face AF” as the AF mode. For example, in the case of a push switch, each time it is pressed, it can be switched in the order of “normal AF” → “face AF” → “only face AF” → “normal AF”. The focus adjustment processing in each of the “normal AF”, “face AF”, and “face only AF” modes will be described later.

  The camera / AF microcomputer 118 controls the operation of the entire video camera. As one of the controls, an AF frame is set in the AF gate 115 according to the AF mode set by the AF mode switch 119 and the face detection result by the face detection unit 117. Further, based on the AF evaluation value signal output from the AF signal processing circuit 116, the focusing drive source 114 is controlled to move the focus lens 106 so that the AF evaluation value becomes large. In this way, the camera / AF microcomputer 118 performs focus control in the TV-AF system (hereinafter referred to as “TV-AF control”). Since TV-AF control is a known technique, detailed description thereof is omitted here. In addition, manual focus control (hereinafter referred to as “MF control”) is also performed in which the focus lens 106 is moved according to the operated direction and the amount of rotation in accordance with the operation of the focus ring 101 of the photographer.

Next, carried out by a video camera of a camera / AF microcomputer 118 having the above configuration, the A F control that put the first embodiment will be described with reference to FIG. This AF control is executed according to a computer program stored in the camera / AF microcomputer 118.

  When the process starts in S201, it is determined in S202 whether or not “Face AF” is selected as the AF mode by the AF mode switch 119. If it is “Face AF” in S202, the process proceeds to S203, and if it is not “Face AF”, the process proceeds to S209.

  In S203, it is determined whether a face is currently detected. If a face is detected in S203, the process proceeds to S204, and if a face is not detected, the process proceeds to S206. In S204, when one face is detected, the face is set as a main face (hereinafter referred to as “main face”), and when a plurality of faces are detected, the position of each face or Based on the size, a face that satisfies a predetermined condition is determined as a main face. Various methods have been proposed for determining the main face, and any method may be used. In S205, an AF frame is set at the main face detection position for the AF gate 115, and the process proceeds to S207. On the other hand, in S206, an AF frame is set at a predetermined position on the screen with respect to the AF gate 115, and the process proceeds to S207.

  In S207, it is determined whether the focus ring 101 is operated. If not operated, the process proceeds to S208, and if operated, the process proceeds to S216 to perform MF control. In S208, based on the AF evaluation value signal obtained from the image signal in the AF frame set in S205 or S206, the focus lens 106 is moved and focused so as to increase the AF evaluation value, so-called TV- AF control is performed.

  If it is determined in S202 that it is not “Face AF”, it is determined in S209 whether the AF mode is “Face AF only”. If it is determined that “Face AF only” is determined in S209, the process proceeds to S210. If not, “normal AF” is determined, and the process proceeds to the above-described processes after S206.

  In S210, it is determined whether a face is currently detected. If a face is detected, the process proceeds to S211. If a face is not detected, the process proceeds to S215. In S211, similarly to S204 described above, when one face is detected, that face is set as the main face, and when a plurality of faces are detected, based on the position and size of each face, Determine the main face. In S212, an AF frame is set at the detection position of the main face with respect to the AF gate 115, and the process proceeds to S213.

  In S213, it is determined whether or not the focus ring 101 is operated. If it is not operated, the process proceeds to S214, and if it is operated, the process proceeds to S215. In S214, “display for AF on face” described later is performed, and the process proceeds to S208. In S208, based on the AF evaluation value signal obtained from the image signal in the AF frame set in S212, the focus lens 106 is moved and focused so as to increase the AF evaluation value. AF control is performed.

  If no face is detected in S210, and if the focus lens is operated in S213, “display without AF on face” described later is performed in S215, and the process proceeds to S216 to perform MF control. .

  FIG. 3 is a diagram illustrating an example of an image obtained when “AF only for face” is selected in S209. FIG. 3 shows a focus state when a subject enters and exits the screen during shooting.

  FIG. 3A shows a state after the subject has been lost last time, and the background is blurred because the focus state when the subject has been lost is maintained. At this time, “display without AF for face” 301 is displayed. FIGS. 3B and 3C show a state in which a subject has entered the screen and a face is detected. While the face is detected, the face within the face frame is kept focused. To do. At this time, “display for AF on face” 302 is displayed. 3D and 3E show images after the subject has left the screen, and the focus state (the focus state in FIG. 3C) when the subject has left is maintained. Also at this time, “display without AF for face” 301 is displayed.

  Note that “display without AF for face” and “display with AF for face” are not limited to the example shown in FIG. 3, and can be recognized by the photographer, such as text display or voice notification. It doesn't matter what. Further, these displays may not be displayed.

  As described above, according to the first embodiment, “Face Only AF” performs AF only while a face is detected. If no face is detected, the focus ring 101 must be operated. The AF state when the face is detected immediately before is held. As a result, when the subject is removed from the screen, the focus state when the subject is removed can be maintained, so that an edit cut point can be obtained at a timing when there is no movement in the image.

  In addition, by performing “display without AF for face” and “display with AF for face”, it is possible to know at a glance the state where AF is not performed because face detection is not performed in “AF only for face”. Can do.

  Since “Face AF” is a function that is used in limited situations, “Normal AF” → “Face AF” → “Face AF” → “Normal AF” can be switched on the normal screen of the main unit SW or touch panel. It is preferable to allow the photographer to access immediately, such as a display. In addition, during “face only AF”, if the face is not detected, the MF occurs. Therefore, the focus lens 106 is moved immediately after the focus ring 101 is operated.

<Second Embodiment>
Next, a second embodiment of the present invention will be described.

  In the first embodiment described above, the main face determination performed in S204 at the time of “Face AF” and the main face determination performed in S211 at the time of “Face AF” are the same processing. In the second embodiment, different processes are performed for the main face determination at the time of “Face AF” and the main face determination at the time of “Face AF”. Since other than that is the same as that of 1st Embodiment mentioned above, description is abbreviate | omitted.

  FIG. 4A shows a main face determination process when a plurality of faces are detected during the “face AF” performed in S204 of FIG. 2, and FIG. 4B shows the main face determination process in S211 of FIG. A main face determination process in the case where a plurality of faces are detected at the time of “only face AF” is shown.

  First, the process of FIG. 4A will be described. When the process is started, a main face candidate on the screen is determined from the positions and sizes of the detected faces in S401. In S402, it is determined whether there is a main face on the previous screen. If there is a main face, the process proceeds to S403, and if there is not, the process proceeds to S404, the current main face candidate is set as the main face as it is, and the process ends.

On the other hand, in S403, it is determined from the position and size whether the main face on the previous screen and the current main face candidate are the same face. If it is determined in S403 that the face is the same, the current main face candidate is used as it is as the main face, and the process ends. If it is determined in S403 that the faces are not the same, the process proceeds to S404, and it is determined whether the main face candidates are the same for a predetermined time. If it is determined in S404 that the main face candidates are the same for a predetermined time, the process proceeds to S405 and the current main face candidate is newly set as the main face. If it is determined in S404 that it is not continuous for a predetermined time, the main face of the previous screen is maintained and the process is terminated.

  Thus, at the time of “face AF”, the face most suitable for the main face is always selected from the position and size in the screen, and AF is performed on that face.

  Next, the process of FIG. 4B will be described. When the process is started, it is determined in S410 whether the main face of the previous screen still exists. If the main face still exists, the process proceeds to S411, the current main face is maintained, and the process ends. On the other hand, if the main face does not exist, the process proceeds to S412 to determine the main face from the detected positions and sizes of the plurality of faces, and the process is terminated.

  The above process will be further described with reference to FIG. FIG. 5A shows a state of “Face AF”. First, B is detected as a main face, but when A appears thereafter, a state determined as a main face candidate from the position and size is predetermined. If time continues, A is reset to the main face. Thus, in “Face AF”, the face most suitable for the main face at that time is selected as the main face.

  On the other hand, FIG. 5B shows a state when “only the face AF” is performed, and shows a case where B is first detected as the main face and A appears thereafter, as in FIG. 5A. When “Face Only AF” is selected, the main face is not changed as long as there is a face once detected as the main face.

  As described above, according to the second embodiment, in addition to the same effects as those of the first embodiment described above, when “only the face AF” is performed, the face once detected is surely kept focused. Can do.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. FIG. 6 is a block diagram illustrating a schematic configuration of a video camera as an example of an imaging apparatus according to the third embodiment of the present invention. The difference from the configuration shown in FIG. 1 is that the touch panel 120 is further provided in the configuration shown in FIG. 1, and the photographer can specify the position in the screen. The output of the touch panel 120 is input to the camera / AF microcomputer 118.

  Next, AF control performed by the camera / AF microcomputer 118 in the third embodiment will be described with reference to FIG. This AF control is executed according to a computer program stored in the camera / AF microcomputer 118. The processing shown in FIG. 7 is different in that the processing of S701 and S702 is performed instead of the main face determination performed in S211 of FIG. 2 described in the first embodiment. Therefore, the same processes as those in FIG.

  When “Face Only AF” is selected (YES in S209) and it is determined that a face is detected (YES in S210), the process proceeds to S701, where the main face has already been selected and the main face is selected. Determine if is still present. If it is determined in S701 that the main face exists, the process proceeds to S212. On the other hand, if it is determined in S701 that there is no main face, the process proceeds to S702 to check whether the photographer has specified the main face. Here, when position information is input by the touch panel 120 and a face exists at the position, the photographer designates the main face, and the process proceeds to S212.

  As described above, according to the third embodiment, even if a face is detected, if there is no main face so far, AF is not performed until the photographer designates the main face, and the main face is detected. Is designated, AF is started for the designated main face. Thereby, in addition to the effect similar to 1st Embodiment mentioned above, it can continue focusing on the main face which a photographer desires more reliably.

  In the third embodiment described above, the case where the main face is designated by input to the touch panel 120 has been described, but it goes without saying that input means other than the touch panel may be used.

  Further, in the first to third embodiments described above, the case where the face is detected and focused on the detected face has been described, but the present invention is not limited to the face, such as a pet or a car. A subject having a predetermined characteristic may be detected and focused.

  Moreover, the present invention may be applied to a system constituted by a plurality of devices or an apparatus constituted by a single device.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (10)

Detecting means for detecting a subject having a predetermined characteristic from an image obtained during shooting of a moving image;
Auto focus means for adjusting the focus by moving the focus lens based on the image;
Manual focus means for moving the focus lens based on the operation of the focus operation unit by the photographer;
A first AF mode that selects the autofocus means when the subject is detected by the detection means, and selects the manual focus means when the subject is not detected by the detection means; Setting means for selectively setting the second AF mode for selecting the auto-focusing means regardless of whether the object is detected by the detecting means or the object is not detected by the detecting means. ,
In the first AF mode, when the subject is detected by the detection means, the autofocus means sets an AF area corresponding to the detected subject and based on the image in the AF area. The focus lens is moved to adjust the focus, and when the subject is not detected by the detection means, the manual focus means does not move the focus lens unless the focus operation unit is operated.
In the second AF mode, when the subject is detected by the detecting means, the autofocus means sets an AF area corresponding to the detected subject and based on the image in the AF area. The focus lens is moved to adjust the focus, and when the object is not detected by the detection means, the autofocus means sets an AF area in the image and applies the image in the AF area to the image in the AF area. A focus adjustment apparatus that performs focus adjustment by moving the focus lens based on the focus adjustment apparatus.   The main subject determination method is different between the first AF mode and the second AF mode when a plurality of subjects having the predetermined characteristics are detected by the detection unit. The focus adjustment device described in 1. In the first AF mode, when a plurality of subjects having the predetermined characteristics are detected, the autofocus unit focuses on the detected plurality of subjects by the previous control of the focus lens. If there is a subject combined with the subject, an AF area corresponding to the subject is set, the focus lens is moved based on an image in the AF area, and focus adjustment is performed.
In the second AF mode, when a plurality of subjects having the predetermined characteristics are detected, the detected positions and sizes of the plurality of subjects, and the autofocus unit performs the previous focus lens. based on the information of a subject focused by the control of, by setting the AF area, in claim 2 in which by moving the focus lens based on an image of the AF area and performing focus adjustment The focusing device as described. It further has an input means,
In the first AF mode, if there is an input for selecting any one of the plurality of subjects by the input means , an AF area corresponding to the subject selected by the input means is set, and within the AF area The focus adjustment apparatus according to claim 2 , wherein focus adjustment is performed by moving the focus lens based on an image . When a plurality of subjects having the predetermined characteristics are detected in the first AF mode, the autofocus means has no subject focused by the previous control of the focus lens, and the input 5. The focus adjustment apparatus according to claim 4 , wherein when there is no input for selecting any of the plurality of subjects by means, the manual focus means is selected . In the first AF mode, when the autofocus means is selected , the photographer is notified that the subject is focused by autofocus, and when the manual focus means is selected , focusing device according to any one of claims 1 to 5, characterized in that to inform the photographer that not focused on the subject with auto focus.   Even if the subject is detected by the detection means in the first AF mode, the manual focus means is selected when the focus operation unit is operated by a photographer. The focus adjustment apparatus according to claim 1, wherein the focus adjustment apparatus is characterized in that: From an image obtained during dynamic imaging, a detection step of detecting an object having a predetermined characteristic,
An autofocus step of adjusting the focus by moving the focus lens based on the image;
A manual focus step for moving the focus lens based on the operation of the focus operation unit by the photographer;
A first AF mode that selects the autofocus step when the subject is detected by the detection step, and selects the manual focus step when the subject is not detected by the detection step; A setting step for selectively setting the second AF mode for selecting the autofocus step regardless of whether the subject is detected by the detection step or the subject is not detected by the detection step. ,
In the first AF mode, when the subject is detected by the detection step, an AF area corresponding to the subject detected by the autofocus step is set, and based on an image in the AF area. The focus lens is moved to adjust the focus, and when the subject is not detected in the detection step, the focus lens is not moved unless the focus operation unit is operated in the manual focus step.
In the second AF mode, when the subject is detected by the detection step, an AF area corresponding to the detected subject is set by the autofocus step, and an image in the AF area is set. Based on this, the focus lens is moved to adjust the focus. When the subject is not detected by the detection step, an AF area is set in the image by the autofocus step. A focus adjustment method, wherein focus adjustment is performed by moving the focus lens based on an image . The program for making a computer perform each process of the focus adjustment method of Claim 8 . A computer-readable storage medium storing the program according to claim 9 .