JP2011004219A - Image capturing apparatus and tracking method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image capturing apparatus which tracks a main subject so as not to miss it, even when an acquisition interval of images for use in comparison is prolonged.SOLUTION: The image capturing apparatus includes: an imaging unit 15 (17) which acquires image information for recognizing a position of a tracking object in a screen; a first operation means 36 for calculating an absolute amount by which the tracking object moves as movement information while the imaging unit 15 (17) interrupts acquisition of the image information; a second operation means 33 for calculating an absolute amount of posture change of the image capturing apparatus as posture change information based on a detection signal from a posture detection means 24 for detecting a posture of the image capturing apparatus with the imaging unit 15 (17); and a recognition means 36 for performing normal recognition for recognizing the position of the tracking object in the screen from the image information when the imaging unit 15 (17) acquires the image information, and performing first interruption recognition for recognizing the position of the tracking object using a result by estimating the position of the tracking object in the screen from the image information acquired by the imaging unit 15 (17) after interruption using the position of the tracking object in the screen recognized after the interruption, the movement information, and the posture change information when the imaging unit 15 (17) interrupts the acquisition of the image information.

Description

  The present invention relates to an imaging apparatus and a tracking method.

  A technique for analyzing a through image and determining a moving direction of a main subject is known (see Patent Document 1).

JP 2008-287797 A

  In the prior art, the moving direction of the main subject is determined based on the change in the positional relationship between the main subject and the reference point between the through images. However, there is a possibility that the main subject may be lost while acquisition of image information such as a through image is interrupted.

  An imaging apparatus according to the present invention includes an imaging unit that acquires image information for recognizing the position of a tracking target in a screen, and an absolute amount that the tracking target has moved while the imaging unit interrupts acquisition of the image information. Based on a detection signal from a first calculation unit that calculates as information and a posture detection unit that detects the posture of the imaging device including the imaging unit, a second calculation that calculates the absolute amount of the posture change of the imaging device as posture change information When the image capturing unit acquires image information, normal recognition for recognizing the position of the tracking target in the screen is performed from the image information. When the image capturing unit interrupts the acquisition of the image information, Using the result of estimating the position of the tracking target in the screen from the image information acquired by the imaging unit after the interruption, using the position of the tracking target in the screen recognized by the camera, movement information, and posture change information Recognition Characterized in that it comprises a first recognition means for performing interrupt recognition to be.

  According to the present invention, even if acquisition of image information used for comparison is interrupted, tracking can be performed so as not to lose sight of the main subject.

It is a figure explaining the principal part structure of the single-lens reflex electronic camera by one embodiment of this invention. It is a figure which illustrates AF area display. It is a figure explaining the principal part structure of a single-lens reflex electronic camera when a live view changeover switch is turned on. It is a block diagram which illustrates the control circuit of a single lens reflex electronic camera. 6 is a timing chart for explaining continuous shooting operation when a live view switch is turned off. (a) is a diagram illustrating a change in angular velocity dθ / dt per predetermined time, (b) is a diagram illustrating a camera movement amount F (n), and (c) is a camera movement vector G (n). (D) is a diagram illustrating the amount of movement H (n) of the main subject. It is a figure explaining the relationship of Formula (2) between the frame images P1 and P2. (a) is a diagram illustrating a history of a movement vector G (n), and (b) is a diagram illustrating an absolute value | ΔG (n) | of a change amount of a motion vector G (n). 6 is a timing chart for explaining a continuous shooting operation when a live view switching switch is turned on. It is a figure which illustrates a computer apparatus.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a diagram for explaining a main configuration of a single-lens reflex electronic camera according to an embodiment of the present invention. In FIG. 1, a lens barrel 2 configured to be detachable from a camera body 1 is attached. The live view changeover switch 19 is turned off. The live view changeover switch 19 is a switch for performing a live view display operation described later.

<Observation with optical viewfinder>
Light from the subject enters the camera body 1 through the photographing optical systems 3 and 5 of the lens barrel 2 and the diaphragm 6. When the live view changeover switch 19 is turned off, the subject light incident on the camera body 1 is guided to the upper optical finder by a quick return mirror (hereinafter referred to as a main mirror) 7 before being released, and is then diffused on the screen 10. To form an image. Part of the subject light incident on the camera body 1 is reflected downward by the sub mirror 8 and also enters the AF sensor 9. The AF sensor 9 constitutes a known phase difference detection AF (autofocus) device using pupil division.

  The AF sensor 9 is used in a focus detection process in which the AF device detects a focus adjustment state by the lens barrel 2. When the lens drive information calculated by the lens drive amount calculation unit 38 (FIG. 4) based on the focus adjustment state is transmitted from the camera body 1 to the lens barrel 2, the AF motor 4 in the lens barrel 2 moves to the focusing lens. 3 is moved back and forth in the optical axis direction. Thereby, focus adjustment is performed on a predetermined subject.

  The subject light imaged on the diffusion screen 10 further enters the pentaprism 11 via the AF area display unit 25. The pentaprism 11 guides incident subject light to the optical viewfinder lens 12, and also guides part of it to the photometric sensor 15. The photographer observes the subject image through the optical viewfinder from the eyepiece 13.

  The light beam incident on the photometric sensor 15 via the photometric lens 14 forms a subject image on its imaging surface. The photometric sensor 15 is configured by a CCD image sensor or the like provided with a plurality of photoelectric conversion elements (for example, the number of pixels corresponding to VGA) corresponding to the pixels. The photometric sensor 15 outputs a photoelectric conversion signal for photometry according to the brightness of the subject image.

  The photometric sensor 15 is provided with a color filter (not shown), and the photometric sensor 15 outputs a photoelectric conversion signal received through the color filter. For example, an R, G, B color filter is disposed at the pixel position of the photometric sensor 15, so that a photoelectric conversion signal corresponding to light of the R color component is output from the pixel received through the R color filter, A photoelectric conversion signal corresponding to light of the G color component is output from the pixel received through the G color filter, and a photoelectric conversion signal corresponding to light of the B color component is output from the pixel received through the B color filter. .

  The AF area display unit 25 arranged side by side with the diffusion screen 10 is constituted by, for example, a transmissive liquid crystal display panel. The AF area display unit 25 displays the AF area so as to overlap the subject image formed on the diffusion screen 10.

  FIG. 2 is a diagram illustrating an AF area display. The AF apparatus according to the present embodiment is configured to detect phase differences at a plurality of points (31 points in the example of FIG. 2) on the shooting screen. The AF area display unit 25 emits the display segment corresponding to the AF area in which the phase difference detection information is adopted for focus adjustment among the 31 display segments 25a indicating the AF area, so that the position of the shooting screen is changed. Displays the point where the phase difference is being detected.

  After the release, the main mirror 7 is rotated to a position similar to that at the time of live view display (FIG. 3) described later, that is, the main mirror 7 is retracted from the optical path. The subject light in this case is guided to the image sensor 17 via the focal plane shutter 18 and forms a subject image on the image pickup surface of the image sensor 17. The image sensor 17 is configured by, for example, a CMOS image sensor including a plurality of charge storage photoelectric conversion elements corresponding to pixels. The image sensor 17 captures a subject image on the imaging surface and outputs a photoelectric conversion signal corresponding to the brightness of the subject image.

  The image sensor 17 outputs a photoelectric conversion signal received through a color filter (not shown). For example, by providing R, G, B color filters at the pixel position of the image sensor 17, a photoelectric conversion signal corresponding to light of the R color component is output from the pixels received through the R color filter, A photoelectric conversion signal corresponding to light of the G color component is output from the pixel received through the G color filter, and a photoelectric conversion signal corresponding to light of the B color component is output from the pixel received through the B color filter. . The electronic camera of the present embodiment uses the photoelectric conversion signal from the image sensor 17 for image recording, for live view display when live view is turned on, and for detecting contrast information. The contrast information is used at the time of focus adjustment by a contrast detection method when live view is turned on, which will be described later.

  The main monitor 16 is disposed on the back surface of the camera body 1. The main monitor 16 reproduces and displays a captured image and a live view image described later.

  The focal plane shutter 18 includes a front curtain 18a, a rear curtain 18b, and a shutter drive unit 18c. The shutter drive unit 18c performs traveling control of the front curtain 18a and the rear curtain 18b so that a predetermined exposure time is obtained.

  When the exposure operation is completed, the main mirror 7 is driven down to the position illustrated in FIG. 1, the focal plane shutter 18 is charged and the captured image is recorded on the recording medium 42, and the series of imaging operations ends.

<Live view observation>
FIG. 3 is a diagram for explaining a main configuration of a single-lens reflex electronic camera when the live view changeover switch 19 is turned on. The live view display means that a monitor image repeatedly captured at a predetermined time interval (for example, 30 frames / second) by the image sensor 17 is sequentially reproduced and displayed on the main monitor 16 before a shooting instruction (release). As illustrated in FIG. 3, when the main mirror 7 is up, the subject light flux is not guided to the finder portion, so that the photographer cannot confirm the subject image from the eyepiece portion 13. Therefore, the electronic camera displays a monitor image on the main monitor 16 so that the subject image can be observed on the main monitor 16.

  When the CPU 100 (FIG. 4) receives an ON operation signal from the live view switching switch 19, it instructs to switch to the live view mode. Thereby, the mirror 7 is retracted (up state) from the optical path, and the focal plane shutter 18 opens the front curtain 18a and the rear curtain 18b. The image processing unit 21 generates display data using the frame image acquired by the image sensor 17, and the main monitor 16 sequentially displays a reproduction image based on the display data (live view display).

  Since the display data for live view display only needs to have data having a size corresponding to the display resolution of the main monitor 16, the resolution may be smaller than the image data for recording. For this reason, the readout of the photoelectric conversion signal from the image sensor 17 for live view display reduces the data size compared to the readout data at the time of shooting by performing pixel thinning readout.

  FIG. 4 is a block diagram illustrating the control circuit of the single-lens reflex electronic camera described above. The CPU 100 of the camera body 1 includes a sequence control unit 31, a contrast AF calculation unit 32, a camera movement amount calculation unit 33, a feature extraction unit 35, a correlation calculation unit 36, a photometry calculation unit 37, and a lens driving amount calculation. Part 38.

  The release button 20 sends an operation signal corresponding to the release operation (full press operation) to the CPU 100. When the release button 20 is pressed down to about half of the normal stroke, a half-press operation signal is output, and when the release button 20 is pressed down to the normal stroke, a full-press operation signal is output.

  The live view changeover switch 19 sends an on or off signal to the CPU 100 in accordance with the live view changeover operation. The angular velocity sensor 24 detects a motion generated in the camera body and sends a detection signal (angular velocity signal) to the CPU 100.

  The AF motor 4 moves the focusing lens 3 (FIG. 1) forward and backward in the optical axis direction. The aperture drive unit 6a drives the aperture 6 (FIG. 1) to a predetermined aperture. The shutter drive unit 18c performs traveling control of the front curtain 18a and the rear curtain 18b described above. The image processing unit 21 performs predetermined signal processing on the photoelectric conversion signals from the image sensor 17 and the photometric sensor 15.

  The AF area display unit 25 performs the AF area display described above. As described above, the main monitor 16 reproduces and displays a live view image and a captured image. The memory 23 temporarily stores the photoelectric conversion signal and various data related to the camera operation. The recording interface (I / F) 41 has an interface function for recording a recording image signal on the recording medium 42. The recording medium 42 is constituted by a memory card, for example, and records a captured image signal.

  The sequence control unit 31 in the CPU 100 controls the charge and travel of the focal plane shutter 18 (FIG. 1) according to the release operation signal, and drives a mirror drive mechanism (not shown) to drive the main mirror 7 to move up ( 3) and mirror down driving (returning to the position shown in FIG. 1) is controlled.

  In the live view mode where the subject luminous flux is not guided to the AF sensor 9 (FIG. 3) (that is, when the main mirror 7 is up), the focus adjustment state cannot be detected by the phase difference detection method described above. Therefore, the contrast AF calculation unit 32 performs AF calculation using a contrast detection method. Specifically, a known focus evaluation value calculation is performed on the basis of the imaging signal output from the imaging element 17 while the AF motor 4 is driven to move the focusing lens 3, whereby the high frequency component of the image is sharpened. The focus evaluation value (hereinafter referred to as contrast information) replaced with is calculated. The contrast information is maximized when the photographing optical systems 3 and 5 are in a focused state where a sharp image is formed on the imaging surface of the image sensor 17, that is, when the blur of the edge of the subject image captured by the image sensor 17 is minimal. .

  When the focus evaluation value increases, the contrast AF calculation unit 32 continues to move the focusing lens 3 in the same direction, and when the focus evaluation value decreases, the contrast AF calculation unit 32 reverses the moving direction of the focusing lens 3 to obtain the focus evaluation value. The AF motor 4 is driven so as to move the focusing lens 3 to a position where the maximum value is obtained, that is, a position corresponding to the in-focus state.

  The camera movement amount calculation unit 33 calculates the movement amount of the camera body 1 based on the detection signal from the angular velocity sensor 24. Then, the calculated movement amount is used as posture change information indicating the absolute amount of the posture change of the camera. The feature extraction unit 35 uses the photoelectric conversion signal from the photometric sensor 15 or the image sensor 17 to extract the feature of the main subject. The main subject is, for example, a subject present at a position corresponding to the AF area specified by the photographer when the release button 20 is pressed halfway. Alternatively, when the release button 20 is pressed halfway, a known face detection process is performed based on the image signal captured by the image sensor 17, and the detected “face” is set as the main subject.

  The correlation calculation unit 36 calculates the amount of movement of the main subject in the screen based on a plurality of frame images having different acquisition times. For example, by using a known template matching technique, a region similar to the main subject in the previously acquired frame image is detected (tracked) from the frame image acquired later, and the position of the main subject in the screen is determined. Then, a movement vector indicating the amount of change in the position is calculated. The calculated movement vector is used as movement information of the main subject.

  The photometric calculator 37 calculates the subject brightness using the photoelectric conversion signal from the photometric sensor 15 or the image sensor 17. The photometry calculating unit 37 further performs a predetermined exposure calculation using the set imaging sensitivity and the calculated subject luminance, and determines the control aperture AV and the control shutter time TV.

  The lens driving amount calculation unit 38 detects the focus adjustment state (defocus amount) by the photographing optical systems 3 and 5 using the detection signal from the AF sensor 9, and calculates the driving amount of the focusing lens 3 according to the detection result. . When performing the focus adjustment by the phase difference detection method, the CPU 100 drives the AF motor 4 according to the drive amount calculated by the lens drive amount calculation unit 38. On the other hand, when performing the focus adjustment by the contrast detection method, the CPU 100 drives the AF motor 4 according to the increase / decrease of the focus evaluation value calculated by the contrast AF calculation unit 32.

<Sequential shooting when using optical viewfinder>
Since this embodiment has a feature in the operation when performing continuous shooting using the electronic camera described above, the following description will focus on the continuous shooting operation. FIG. 5 is a timing chart for explaining the continuous shooting operation when the live view changeover switch 19 is turned off.

  At time T <b> 1 in FIG. 5, a half-press operation signal from the release button 20 is input to the CPU 100. At this time, the main mirror 7 (sub mirror 8) is in the down position (in the optical path), the front curtain 18a of the focal plane shutter 18 is closed, the rear curtain 18b is open, and the aperture 6 is open. The angular velocity sensor 24 starts a detection operation in response to an instruction from the CPU 100 and performs detection every predetermined time. FIG. 6A is a diagram illustrating a change in angular velocity dθ / dt every predetermined time.

  The photometric sensor 15 (referred to as a second image sensor in FIG. 5) starts an accumulation operation in response to an instruction from the CPU 100. The accumulated photoelectric conversion signal is read out to the image processing unit 21. The image processing unit 21 generates an image P <b> 1 (see FIG. 7) from the photoelectric conversion signal from the second image sensor 15 and sends it to the CPU 100. The CPU 100 stores the image P1 in the memory 23, thereby repeating accumulation → reading → storage.

  The photometric calculation unit 37 performs photometric calculation based on the latest image P1 stored in the memory 23 to obtain an appropriate exposure. The photometric calculation is repeated every time the image P1 is acquired. The feature extraction unit 35 extracts the feature portion of the main subject from the latest image P1 and stores information indicating the region TM1 in the memory 23. The main subject is, for example, a subject that exists at a position corresponding to the AF area designated by the photographer at time T1.

  The AF sensor 9 starts an accumulation operation in response to an instruction from the CPU 100. The accumulated photoelectric conversion signal is read to the lens driving amount calculation unit 38. The lens drive amount calculation unit 38 detects the defocus amount based on the photoelectric conversion signal from the AF sensor 9, and calculates the drive amount of the focusing lens 3 according to the defocus amount. The CPU 100 drives the AF motor 4 in accordance with the calculated drive amount and moves the focusing lens 3 to repeat accumulation → reading → drive amount calculation → focusing lens (AF lens) drive.

  At time T2 in FIG. 5, reading of the second photoelectric conversion signal from the second image sensor 15 is completed. Based on the detection signal of the angular velocity sensor 24, the camera movement amount calculation unit 33 calculates the posture change of the camera body 1 from the previous accumulation end time by the second image sensor 15 to the current accumulation end time by the second image sensor 15. Find the absolute amount. Specifically, the detection signal at every predetermined time is integrated to obtain information indicating the focal length and the photographing distance of the photographing optical systems 3 and 5 from the lens barrel 2, and the integration result, the focal length information, and the photographing information are obtained. Using the distance information and the magnification information of the photometric lens 14, calculation is made as posture change information F (2) indicating the absolute amount of the posture change of the camera on the imaging surface of the second image sensor 15.

  The photoelectric conversion signal after the second accumulation by the second image sensor 15 is read to the image processing unit 21. The image processing unit 21 generates an image P2 from the second photoelectric conversion signal and sends it to the CPU 100. The CPU 100 stores the image P2 in the memory 23.

  The photometric calculation unit 37 performs photometric calculation based on the image P2 stored in the memory 23 to obtain an appropriate exposure. The correlation calculation unit 36 detects an area TM2 similar to the characteristic area TM1 of the main subject in the previously acquired frame image P1 from the frame image P2 acquired later. Specifically, the region TM2 similar to the region TM1 is obtained on the frame image P2 by performing correlation calculation while gradually shifting the position of interest from the position corresponding to the region TM1 on the frame image P2. Then, the distance from the position corresponding to the area TM1 to the position of the area TM2 on the frame image P2 is stored in the memory 23 as a movement vector G (2) indicating the relative movement amount of the main subject on the screen. .

  When the feature extraction unit 35 detects the region TM2 from the image P2, the feature extraction unit 35 makes the region TM2 a new region TM1. That is, in order to detect a new area TM2 by performing a correlation operation on the newly acquired frame image P2, the area of the main subject in the previous frame image P1 that was previously TM2 is changed to TM1.

The CPU 100 calculates the sum H (2) of the above-described camera posture change information F (2) and the movement vector G (2) obtained by the correlation calculation unit 36 as shown in the following equation (1), and stores the memory 23. To store. The sum H (2) represents the absolute amount that the main subject has moved (hereinafter referred to as movement information).
H (2) = F (2) + G (2) (1)

  The AF area used by the lens drive amount calculation unit 38 to detect the defocus amount (that is, the AF area that employs the phase difference detection information for focus adjustment) corresponds to the area TM2 detected in the latest frame image P2. Let it be an area.

  In order to quickly correspond the area TM2 detected in the latest frame image P2 to the AF area used for the defocus amount detection, the correlation driving unit 36 starts the next driving amount calculation by the lens driving amount calculating unit 38. A time T3 immediately after the latest area TM2 is detected from the frame image P2.

After time T2, the above-described operation is repeated until a full-press operation signal from the release button 20 is input to the CPU 100 at time T8. In this case, when the number of image acquisitions (accumulation number) by the second image sensor 15 is represented by n, the movement information H (n) of the main subject is represented by the following equation (2).
H (n) = F (n) + G (n) (2)
However, n is an integer of 2 or more.

  FIG. 6A is a diagram illustrating the output of the angular velocity sensor 24. FIG. 6B is a diagram illustrating camera posture change information F (n). FIG. 6C is a diagram illustrating the movement vector G (n). FIG. 6D is a diagram illustrating movement information H (n) of the main subject. FIG. 7 is a diagram for explaining the relationship of the above equation (2) between the frame images P1 and P2.

  The CPU 100 determines the history of the time t (n) from the (n−1) th accumulation end time to the nth accumulation end time (n is an integer of 2 or more) and the history of the movement vector G (n) (n is 2). The above integers), that is, t (2) to t (n) and G (2) to G (n) are stored in the memory 23. The stored history is used for the determination of Expression (4) and Expression (5) described later. Further, the CPU 100 stores posture change information F (n−1), F (n) and movement information H (n−1), H (n) of the main subject in the memory 23. The stored information is used for calculation using Expression (6), Expression (7), and Expression (8) described later.

  At time T <b> 8 in FIG. 5, a full press operation signal from the release button 20 is input to the CPU 100. The CPU 100 stops accumulation → reading → storage using the second image sensor 15, and stops accumulation → reading → drive amount calculation → focusing lens driving using the AF sensor 9. However, when the accumulation has already been completed, the process using the photoelectric conversion signals accumulated by the second image sensor 15 and the AF sensor 9 is performed, respectively, and control is performed so that no new accumulation is performed.

  The CPU 100 performs control so that the detection operation by the angular velocity sensor 24 and the calculation of the camera posture change information F by the camera movement amount calculation unit 33 are continued.

  When the CPU 100 starts up driving of the main mirror 7 and the sub mirror 8, the main mirror 7 and the sub mirror 8 are in the up state at time T10, and the out-of-light path is finished. The CPU 100 starts accumulation in the image sensor 17 (referred to as the first image sensor in FIG. 5) at time T10, and sends an instruction to the shutter control unit 18c to start traveling the front curtain 18a (open drive).

  At time T9 after the start of mirror-up driving, the CPU 100 sends an instruction to the diaphragm driving unit 6a to narrow the diaphragm 6 to a predetermined opening. The predetermined opening corresponds to, for example, the control aperture AV obtained based on the last photometric calculation result before the full pressing operation.

  The CPU 100 sends an instruction to the shutter control unit 18c, and when the predetermined shutter time elapses from the start of travel of the front curtain 18a (T10), the travel of the rear curtain 18b is started (closed drive). The predetermined shutter time corresponds to, for example, the control shutter time TV obtained based on the last photometric calculation result before the full-press operation.

  At time T <b> 12 when the travel of the trailing curtain 18 b is completed, the CPU 100 ends the accumulation of the first image sensor 17. The accumulated photoelectric conversion signal is read out to the image processing unit 21. The image processing unit 21 generates a recording image from the photoelectric conversion signal from the first image sensor 17 and sends it to the CPU 100. The CPU 100 sends an instruction to the recording interface 41 to record the captured image on the recording medium 42 and ends the imaging process for one frame.

  On the other hand, the CPU 100 starts the down drive of the main mirror 7 and the sub mirror 8 at the time T12 and instructs the shutter control unit 18c to perform a shutter charge. At time T13, the CPU 100 sends an instruction to the aperture drive unit 6a to return the aperture 6 to the open state.

  At time T14, the main mirror 7 and the sub mirror 8 are in the down state, and the return to the optical path is completed. By this time, the shutter charge (the front curtain 18a is closed and the rear curtain 18b is opened) is completed.

  When the full press operation signal from the release button 20 is continuously input to the CPU 100 after the time T14, the CPU 100 repeats the same processing as the photographing performed from the time T8 to the time T14 described above after the time T15. Thus, the continuous shooting operation for the second and subsequent frames is performed.

  From time T14 to time T15, processing similar to that from time T1 to time T8 described above is performed. Note that the number of times of image acquisition (accumulation count) by the second image sensor 15 performed between time T14 and time T15 is determined by the time required for the camera to prepare for continuous shooting after the second frame. In other words, the number of image acquisitions is not limited to the three times illustrated in FIG. 5, and the number of image acquisitions decreases when the preparation for shooting is made earlier, and the number of image acquisitions increases when a long time is required for the preparation for shooting.

<Estimation during blackout>
During the time from the time T8 when the main mirror 7 and the sub mirror 8 start up driving to the time T14 when the main mirror 7 and sub mirror 8 finish down driving, the luminous flux is not guided to the viewfinder, so the viewfinder field is dark. Become. Such a state is called a so-called blackout. During blackout, accumulation (image acquisition) by the second image sensor 15 is not performed. For this reason, when the main subject moves greatly in the screen during the blackout, the correlation calculation described above may take a long time after the blackout. Specifically, when blackout occurs during the acquisition of P1 and P2 in FIG. 7, the main subject moves greatly in the screen, and the position corresponding to the area TM1 in the frame image P2 and the main subject after blackout When the position of the existing area TM2 is significantly different from the position corresponding to the area T1 in the frame image P2, if the correlation calculation is performed while gradually shifting the position of interest, the calculation time is increased.

Therefore, the time required for the correlation calculation is shortened by estimating the position of the region TM2 on the frame image P2 after the blackout and performing the correlation calculation while gradually shifting the position of interest from the estimated position. For this purpose, the correlation calculation unit 36 acquires the correlation calculation from time T1 before blackout to time T8 and stores the history of the movement vector G (n) stored in the memory 23 (in this embodiment, n is 2). 5), the absolute value | ΔG (n) | of the amount of change between the movement vector G (n) and the movement vector G (n−1) according to the following equation (3) (ΔG (3), ΔG (4) and ΔG (5)) are obtained respectively.
| ΔG (n) | = | G (n) −G (n−1) | (3)
However, n is an integer of 3 or more. FIG. 8A illustrates the history of the movement vector G (n), and FIG. 8B illustrates the absolute value | ΔG (n) | of the change amount of the motion vector G (n). FIG.

The correlation calculation unit 36 determines that it is not necessary to perform the above estimation when the following expression (4) is satisfied (condition 1), and the above estimation is necessary when the following expression (5) is satisfied. (Condition 2).
Σ | ΔG (n) | / Σt (n) ≦ β (4)
Σ | ΔG (n) | / Σt (n)> β (5)
However, Σ | ΔG (n) | is the sum of absolute values of the amount of change of the motion vector G (n) from time T1 to time T8. Σt (n) is the sum of times calculated using the history of time t (n) stored in the memory 23. β is a predetermined determination threshold value.

  When the CPU 100 determines “Condition 1”, it is assumed that the user is chasing the subject, and the correlation calculation unit 36 first performs the correlation calculation after the blackout after time T14. By performing correlation calculation while gradually shifting the position of interest from the position corresponding to the region TM1 on the obtained frame image, a region TM2 similar to the region TM1 on the first frame image after blackout is obtained.

  When the CPU 100 determines “Condition 2”, it is assumed that the user is not following the subject, and the correlation calculation unit 36 first performs the correlation calculation after the blackout after time T14. A region TM2 similar to the region TM1 on the first frame image after blackout is obtained by performing correlation calculation while gradually shifting the target position from the following estimated position on the obtained frame image.

When the movement information of the main subject during blackout is Hb, the posture change information of the camera during blackout is Fb, and the movement vector during blackout is Gb, the following equation (6) is established.
Hb = Fb + Gb (6)
Here, Fb uses the calculation result of the camera posture change information F continuously performed during blackout.

The correlation calculation unit 36 assumes that Hb per unit time is the same as that immediately before blackout, and extrapolates during blackout using the following equation (7).
Hb = H (k) / t (k) × tb (7)
However, the number of times of image acquisition (accumulation count) before blackout by the second image sensor 15 is k, the movement information of the main subject obtained immediately before the blackout is H (k), and the (k−1) -th The time from the accumulation end time of the second image sensor 15 to the accumulation end time of the k-th second image sensor 15 is t (k), and the blackout time is tb.

Gb = Hb−Fb = H (k) / t (k) × tb−Fb (8)
The correlation calculation unit 36 calculates Gb of the movement vector during blackout according to Expression (8). The correlation calculation unit 36 estimates that the main subject exists at a position separated from the position of the region TM1 in the frame image immediately before the blackout by the movement vector Gb, and performs correlation calculation while gradually shifting the position of interest from this estimated position. To obtain a region TM2 similar to the region TM1 on the first frame image after blackout.

<Sequential shooting when live view is on>
FIG. 9 is a timing chart for explaining the continuous shooting operation when the live view changeover switch 19 is turned on. At time T <b> 31 in FIG. 9, a half-press operation signal from the release button 20 is input to the CPU 100. At this time, the main mirror 7 and the sub mirror 8 are in the up position (retracted out of the optical path), the front curtain 18a and the rear curtain 18b of the focal plane shutter 18 are both open, and the aperture 6 is open. The angular velocity sensor 24 starts a detection operation in response to an instruction from the CPU 100 and performs detection every predetermined time.

  The image sensor 17 (referred to as the first image sensor in FIG. 9) starts an accumulation operation in response to an instruction from the CPU 100. This accumulation operation is not for shooting and recording, but for live view display that performs pixel thinning readout. The accumulated photoelectric conversion signal is thinned and read out to the image processing unit 21. The image processing unit 21 generates an image P <b> 1 from the photoelectric conversion signal from the first image sensor 17 and sends it to the CPU 100. The CPU 100 stores the image P1 in the memory 23, thereby repeating accumulation → reading → storage.

  The photometric calculation unit 37 performs photometric calculation based on the latest image P1 stored in the memory 23 to obtain an appropriate exposure. The photometric calculation is repeated every time the image P1 is acquired. The feature extraction unit 35 extracts the feature portion of the main subject from the latest image P1 and stores information indicating the region TM1 in the memory 23. The main subject is, for example, a subject that exists at a position corresponding to the AF area designated by the photographer at time T31.

  The contrast AF calculation unit 32 calculates contrast information using a photoelectric conversion signal from a pixel corresponding to the AF area in the latest image P1. The CPU 100 repeats accumulation → reading → contrast information calculation → focusing lens driving by driving the AF motor 4 and moving the focusing lens 3 according to the increase / decrease in the calculated contrast information.

  At time T32 in FIG. 9, the second photoelectric conversion signal readout from the first image sensor 17 is completed. Based on the detection signal of the angular velocity sensor 24, the camera movement amount calculation unit 33 moves the camera body 1 movement amount F from the previous accumulation end time by the first image sensor 17 to the current accumulation end time by the first image sensor 17. Find (2). The calculation method is the same as when live view is off.

  The photoelectric conversion signal after the second accumulation by the first image sensor 17 is read out to the image processing unit 21. The image processing unit 21 generates an image P2 from the second photoelectric conversion signal and sends it to the CPU 100. The CPU 100 stores the image P2 in the memory 23.

  The photometric calculation unit 37 performs photometric calculation based on the image P2 stored in the memory 23 to obtain an appropriate exposure. The correlation calculation unit 36 detects an area TM2 similar to the characteristic area TM1 of the main subject in the previously acquired frame image P1 from the frame image P2 acquired later. Specifically, the region TM2 similar to the region TM1 is obtained on the frame image P2 by performing correlation calculation while gradually shifting the position of interest from the position corresponding to the region TM1 on the frame image P2. Then, the distance from the position corresponding to the area TM1 to the position of the area TM2 on the frame image P2 is stored in the memory 23 as a movement vector G (2) indicating the relative movement amount of the main subject on the screen. .

  When the feature extraction unit 35 detects the region TM2 from the image P2, the feature extraction unit 35 makes the region TM2 a new region TM1. That is, in order to detect a new area TM2 by performing a correlation operation on the newly acquired frame image P2, the area of the main subject in the previous frame image P1 that was previously TM2 is changed to TM1.

  The CPU 100 calculates the sum H (2) of the above-described camera posture change information F (2) and the movement vector G (2) obtained by the correlation calculation unit 36 as shown in the above equation (1), and stores the memory 23. To store. The sum H (2) represents the absolute amount (movement information) that the main subject has moved.

  The AF area used by the contrast AF calculation unit 32 for calculating contrast information (that is, the AF area corresponding to the pixel used for calculating contrast information) is the AF area corresponding to the area TM2 detected in the latest frame image P2. .

  In order to quickly correspond the area TM2 detected in the latest frame image P2 to the AF area used for calculating contrast information, the start of the next calculation by the contrast AF calculation unit 32 is time T34 after driving the focusing lens, and the correlation It is preferable that the calculation unit 36 detects the latest area TM2 from the latest frame image P2.

  After time T32, the above-described operation is repeated until the full-press operation signal from the release button 20 is input to the CPU 100 at time T38. In this case, when the number of times of image acquisition (accumulation number) by the first image sensor 17 is represented by n, the movement information H (n) of the main subject is represented by the above equation (2).

  The CPU 100 determines the history of the time t (n) from the (n−1) th accumulation end time to the nth accumulation end time (n is an integer of 2 or more) and the history of the movement vector G (n) (n is 2). The above integers), that is, t (2) to t (n) and G (2) to G (n) are stored in the memory 23. The stored history is used for the determination of Expression (4) and Expression (5) described later. Further, the CPU 100 stores posture change information F (n−1), F (n) and movement information H (n−1), H (n) of the main subject in the memory 23. The stored information is used for calculation using Expression (6), Expression (7), and Expression (8).

  At time T38 in FIG. 9, a full-press operation signal from the release button 20 is input to the CPU 100. The CPU 100 immediately stops the accumulation → reading → storage for live view display using the first image sensor 17. Further, the CPU 100 sends an instruction to the shutter control unit 18c to charge the front curtain 18a (closed drive).

  The CPU 100 performs control so that the detection operation by the angular velocity sensor 24 and the calculation of the camera movement amount F by the camera movement amount calculation unit 33 are continued.

  At time T39 after driving the focusing lens after time T38, the CPU 100 sends an instruction to the aperture driving unit 6a to narrow the aperture 6 to a predetermined opening. The predetermined opening corresponds to, for example, the control aperture AV obtained based on the last photometric calculation result before the full pressing operation.

  At time T40 after the front curtain charge, the CPU 100 causes the first image pickup device 17 to start accumulation for photographing and also sends an instruction to the shutter control unit 18c to start the travel of the front curtain 18a. When a predetermined shutter time elapses from time T40, the CPU 100 starts running of the trailing curtain 18b (closed drive). The predetermined shutter time corresponds to, for example, the control shutter time TV obtained based on the last photometric calculation result before the full-press operation.

  At time T42 when the trailing curtain 18b travels, the CPU 100 ends the shooting accumulation of the first image sensor 17. The accumulated photoelectric conversion signal is read out to the image processing unit 21. The image processing unit 21 generates a recording image from the photoelectric conversion signal from the first image sensor 17 and sends it to the CPU 100. The CPU 100 sends an instruction to the recording interface 41 to record the captured image on the recording medium 42 and ends the imaging process for one frame.

  On the other hand, the CPU 100 instructs the shutter controller 18c to charge the trailing curtain 18b at the time T42. At time T43, the CPU 100 sends an instruction to the aperture driving unit 6a to return the aperture 6 to the open state. At time T44, charging (opening) of the shutter rear curtain 18b ends.

  When the full press operation signal from the release button 20 is continuously input to the CPU 100 after the time T44, the CPU 100 repeats the same processing as the photographing performed from the time T38 to the time T44 described above after the time T46. Thus, the continuous shooting operation for the second and subsequent frames is performed.

  From time T44 to time T46, processing similar to that from time T31 to time T38 described above is performed. Note that the number of times of image acquisition (accumulation count) by the first image sensor 17 performed between time T44 and time T46 is determined by the time required for the camera to prepare for continuous shooting after the second frame. In other words, the number of image acquisitions is not limited to three times illustrated in FIG. 9, and the number of image acquisitions decreases when the preparation for shooting is early, and the number of image acquisitions increases when a long time is required for preparation for shooting.

<Estimation during blackout>
The time from the time T38 when the live view display accumulation ends to the time T44 when the live view display accumulation resumes is referred to as a blackout here. If the main subject moves greatly in the screen during blackout, the correlation calculation described above may take time, as in the case of using the optical viewfinder.

  Therefore, the time required for the correlation calculation is shortened by estimating the position of the region TM2 on the frame image P2 after the blackout and performing the correlation calculation while gradually shifting the position of interest from the estimated position. For this purpose, the correlation calculation unit 36 acquires the correlation calculation from time T31 before blackout to time T38 and stores the history of the movement vector G (n) stored in the memory 23 (in this embodiment, n is 2). 5), the absolute value of the amount of change between the movement vector G (n) and the movement vector G (n−1) | ΔG (n) | (in this embodiment, ΔG (3), ΔG (4) and ΔG (5)) are obtained respectively.

  The correlation calculation unit 36 determines that the above estimation is not necessary when the above expression (4) is satisfied (condition 1), and the above estimation is necessary when the above expression (5) is satisfied. (Condition 2).

  When the CPU 100 determines “Condition 1”, the correlation calculation unit 36 performs a first correlation calculation after blackout after time T44, and a position corresponding to the region T1 on the first frame image obtained after blackout. The region TM2 similar to the region TM1 is obtained on the first frame image after blackout by performing the correlation calculation while gradually shifting the position of interest from.

  When the CPU 100 determines “Condition 2”, the correlation calculation unit 36, when performing the first correlation calculation after the blackout after the time T44, pays attention from the following estimated positions on the first frame image obtained after the blackout. By performing correlation calculation while gradually shifting the position, a region TM2 similar to the region TM1 is obtained on the first frame image after blackout.

  The correlation calculation unit 36 extrapolates during blackout using the above equation (7), assuming that Hb per unit time is the same as that immediately before blackout. However, the live view display image acquisition count (accumulation count) before blackout by the first image sensor 17 is k, the movement information of the main subject obtained immediately before the blackout is H (k), and (K-1). The time from the accumulation end time of the first first image sensor 17 to the accumulation end time of the k-th first image sensor 17 is t (k), and the blackout time is tb.

  The correlation calculation unit 36 calculates Gb of the movement vector during blackout by the above equation (8). The correlation calculation unit 36 estimates that the main subject exists at a position separated from the position of the region TM1 in the frame image immediately before the blackout by the movement vector Gb, and performs correlation calculation while gradually shifting the position of interest from this estimated position. To obtain a region TM2 similar to the region TM1 on the first frame image after blackout.

According to the embodiment described above, the following operational effects can be obtained.
(1) The second image sensor 15 (or the first image sensor 17) for acquiring images P1 and P2 for recognizing the positions of the areas TM1 and TM2 including the main subject to be tracked in the screen, and the second image sensor A correlation calculation unit 36 for calculating a movement vector G (n) in which the main subject has moved in the image while the element 15 (or the first imaging element 17) interrupts the acquisition of the images P1 and P2 (during blackout); Based on the detection signal from the angular velocity sensor 24 that detects the posture change of the camera including the second image pickup device 15 (or the first image pickup device 17), the absolute amount of the posture change of the camera is used as the posture change information F (n). When the camera movement amount calculation unit 33 to be calculated and the second image sensor 15 (or the first image sensor 17) acquire the images P1 and P2 continuously, the main object in the screen is obtained from the images P1 and P2. When the normal recognition for recognizing the positions of the areas TM1 and TM2 including the body is performed and the second image pickup device 15 (or the first image pickup device 17) acquires the image P1, and then interrupts the acquisition of the image P2, before the interruption. Using the position of the area TM1 including the main subject, the movement vector G (n) before the interruption, and the posture change information Fb during the interruption from the image P2 acquired after the interruption. The CPU 100 for performing the first post-interruption recognition for estimating the position of the area TM2 including the main subject is provided. In the recognition after the first interruption, the CPU 100 estimates that the main subject exists at a position separated from the position corresponding to the area TM1 in the image P1 immediately before the interruption by the movement vector Gb in the image P2 after the interruption. Since the region TM2 similar to the region TM1 is obtained on the interrupted image P2 by performing the correlation calculation while gradually shifting the target position from the first position, the correlation calculation is performed while gradually shifting the target position from the position of the region TM1. Compared with recognition after 2 interruptions, the amount of calculation can be suppressed. Reducing the amount of calculation is effective for shortening the calculation time.

(2) The CPU 100 performs correlation calculation for recognizing the position of the main subject in the screen from the position corresponding to the region TM1 in the image P2 acquired by the second image sensor 15 (or the first image sensor 17) after the interruption. The recognition after the second interruption is performed instead of the recognition after the first interruption.

(3) When the CPU 100 interrupts the acquisition of the images P1, P2,... After the second image sensor 15 (or the first image sensor 17) acquires the images P1, P2,. The reliability of the recognition method is determined based on the history of recognizing the position of the main subject in the screen from each of the images P1, P2,..., And the recognition after the first interruption and the second interruption are performed based on the determination result. The choice between post-recognition was made.

(4) When the live view changeover switch 19 is on, the main monitor 16 that displays the image for observation using the photoelectric conversion signal acquired by the first image sensor 17 is provided. But you can see the monitor image.

(5) A second imaging element 15 and a first imaging element 17 that performs an imaging operation are provided, and a state in which the subject light flux is guided to the first imaging element 17 and a state in which the subject light flux is not guided to the first imaging element 17 Since switching is possible, switching according to the shooting environment becomes possible.

(Modification 1)
Although the case of a single-lens reflex electronic camera has been described as an example, the present invention may be applied to an electronic camera other than a single-lens reflex type as long as it has a so-called live view function.

(Modification 2)
In the embodiment described above, the example in which the camera control is performed at the timing illustrated in FIG. 5 (FIG. 9) by the program stored in the nonvolatile memory of the CPU 100 in the camera body 1 in advance has been described. The program may be supplied to the camera body 1 via the computer apparatus 250 shown in FIG. Thus, when a program is supplied from the computer apparatus 250 to the camera body 1, the computer apparatus 250 and the camera body 1 can communicate with each other and the program is supplied.

  The program may be supplied to the computer apparatus 250 by setting a recording medium 204 such as a CD-ROM storing the program in the computer apparatus 250, or to the computer apparatus 250 by a method via the communication line 201 such as a network. You may load. When passing through the communication line 201, the program is stored in the hard disk device 203 of the server (computer) 202 connected to the communication line 201. The program can be supplied as various types of computer program products such as provision via the recording medium 204 or the communication line 201.

  The above description is merely an example, and is not limited to the configuration of the above embodiment.

DESCRIPTION OF SYMBOLS 1 ... Camera body 2 ... Lens barrel 3, 5 Shooting optical system 6 ... Aperture 7 ... Main mirror 11 ... Pentaprism 13 ... Eyepiece part 15 ... Photometric sensor 16 ... Main monitor 17 ... Imaging element 19 ... Live view changeover switch 24 ... Angular velocity sensor 33 ... Camera movement amount calculation unit 35 ... Feature extraction unit 36 ... Correlation calculation unit 37 ... Photometry calculation unit 38 ... Lens drive amount calculation unit 100 ... CPU

Claims (8)

An imaging unit for acquiring image information for recognizing the position of the tracking target in the screen;
First calculating means for calculating, as movement information, an absolute amount by which the tracking target has moved while the imaging unit interrupts acquisition of the image information;
Second computing means for computing an absolute amount of the posture change of the imaging device as posture change information based on a detection signal from a posture detection means for detecting the posture of the imaging device including the imaging unit;
When the imaging unit acquires the image information, normal recognition for recognizing the position of the tracking target in the screen is performed from the image information, and when the imaging unit interrupts the acquisition of the image information. , Using the position of the tracking target in the screen recognized before the interruption, the movement information, and the posture change information from the image information acquired by the imaging unit after the interruption An image pickup apparatus comprising: a recognition unit configured to perform first interruption recognition that is recognized using a result of estimating the position of the tracking target. The imaging device according to claim 1,
The imaging device alternatively performs an image acquisition operation for acquiring the image information and an imaging operation for imaging a subject image on the screen. The imaging device according to claim 1 or 2,
The recognizing means recognizes the second post-interruption recognition for recognizing the position of the tracking target in the screen without using the movement information and the posture change information from the image information acquired by the imaging unit after the interruption. An imaging apparatus characterized in that it can be performed instead of the recognition after the first interruption. The imaging device according to claim 3.
In the case where the imaging unit interrupts the acquisition of the image information after performing the acquisition of the image information a plurality of times, the position of the tracking target in the screen is recognized from each of the image information acquired before the interruption Determining means for determining the reliability of the recognition method by the recognition means based on the history;
The recognizing unit selects whether to perform recognition after the first interruption or recognition after the second interruption based on a determination result of the determination unit. In the imaging device according to any one of claims 1 to 4,
When the imaging unit interrupts the acquisition of the image information after the imaging unit acquires the image information a plurality of times, the tracking unit in the screen is acquired from the image information acquired before the interruption. An imaging apparatus, wherein the movement information is calculated using a result of recognizing the position of the camera and the posture change information calculated during the interruption. In the imaging device according to any one of claims 1 to 5,
An imaging apparatus comprising: display means for displaying an image for observation using the image information acquired by the imaging unit. In the imaging device according to any one of claims 1 to 6,
The imaging unit includes a first imaging element that acquires the image information, and a second imaging element that performs the imaging operation,
An image pickup apparatus comprising switching means capable of switching between a state in which a subject light beam is guided to the first image sensor and a state in which the subject light beam is not guided to the first image sensor. Image information acquisition processing for acquiring image information for recognizing the position of the tracking target in the screen;
A first calculation process for calculating the absolute amount of movement of the tracking target as movement information while interrupting the image information acquisition process;
A second calculation process for calculating an absolute amount of the posture change of the screen as posture change information based on a detection signal from a posture detection means for detecting the posture of the screen;
When performing the image information acquisition process, normal recognition is performed for recognizing the position of the tracking target in the screen from the image information, and when the image information acquisition process is interrupted, it is recognized before the interruption. The first position for recognizing the position of the tracking target in the screen from the image information acquired after the interruption using the position of the tracking target in the screen, the movement information, and the posture change information. A tracking method characterized by performing recognition processing for performing interruption recognition.

Images