JP5506374B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus that corrects blurring of a captured image due to camera shake using motion vector detection information obtained from blurring of a moving image, and a control method for the imaging apparatus.

  In an imaging apparatus such as a video camera that captures a moving image, there is a problem that the image is blurred due to camera shake when the lens is zoomed to the telephoto side. As a method of preventing image blur due to such camera shake, a technique for detecting a motion vector of an image from a captured image signal and correcting the blur of the image based on the motion vector has been proposed (for example, Patent Document 1).

  As a method for detecting a motion vector of a moving image, a correlation method based on a correlation calculation, a block matching method, and the like have been conventionally known. However, since they are discussed in detail by Morio Onoe et al. Reference is omitted here.

  However, in the conventional imaging apparatus, when the motion vector is difficult to detect, for example, when the contrast of the subject is low or a large blur occurs, the detection accuracy of the correlation of the image decreases, and an appropriate representative vector is obtained. It becomes impossible. If an error in the detected motion vector value becomes large and an error occurs in the motion vector value, there is a possibility that the accuracy of blur correction is reduced or erroneous correction occurs, and the blur of the captured image occurs discontinuously.

  As a method of solving this problem, it is determined whether or not the detection accuracy of the motion vector is lowered. If it is determined that the detection accuracy is lowered, the detection of the representative vector is interrupted and the blur correction is stopped. There is a way.

JP-A-7-107367

Morio Onoe, Norihiko Maeda, Yu Saito, “Pattern Information Processing: Superposition of Images Using Residual Sequential Test”, Information Processing, Information Processing Society of Japan, July 1976, vol. 17, no. 7, p. 634-640

  However, if the motion vector value is more reliable than the set standard and shake correction is performed, and if the low-reliability vector information is obtained and the shake correction is stopped, the shake correction is executed intermittently. Therefore, the blur correction of the captured image becomes discontinuous. A moving image in which such blur correction is intermittently performed may cause symptoms such as video sickness to a person who appreciates the moving image.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging apparatus that can reduce discontinuity of blur correction and can stably provide a sufficiently and accurately shake-corrected image, and a control method therefor.

To solve the above problem, an imaging apparatus according to the present invention includes an imaging means for converting an object image formed by the projection optical system Taking into image signals, storage means for storing the image signal obtained by said image pickup means And a motion vector detecting means for detecting a motion vector value indicating the amount of displacement of the image from the two image signals obtained by the imaging means , and a reading position of the image signal from the storage means based on the motion vector value. a read position control means for controlling, said a reliability determination means and the image pickup device that have a determining reliability of the motion vector value, the motion vector value based on the determination result by the pre-Symbol reliability judging means comprising an interpolation means to determine the interpolated motion vector interpolation value, said interpolation means, said reliability is determined by the reliability determination unit is lower than a predetermined value, the current frame or the current off When generating a motion vector interpolation value obtained by interpolating a motion vector value indicating a displacement amount of a field image, when a change in the motion vector value of the current frame or the frame or field before the current field tends to increase, The motion vector interpolation value is increased compared to the motion vector value of the previous frame or previous field stored in the storage means, and the change in the motion vector value of the current frame or previous frame or field tends to decrease The motion vector interpolation value is reduced when compared with the motion vector value of the previous frame or the previous field stored in the storage means .

  According to the present invention, in an imaging device that corrects camera shake of a captured image using motion vector detection information obtained from motion image blur, the blur correction is sufficiently and accurately corrected by reducing the blur discontinuity. It is possible to provide a stable image.

1 is a block diagram of a shake correction circuit included in an imaging apparatus according to a first embodiment of the present invention. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to a first embodiment of the present invention. 3 is a flowchart of shake correction executed by the imaging apparatus according to the first embodiment of the present invention. The overall motion vector when blur correction is performed using the motion vector interpolation value executed by the imaging apparatus according to the first embodiment of the present invention when it is determined that the reliability of the vector value in block units is low. It is a figure explaining the change of a value and contrast with the change of the whole motion vector value at the time of stopping blurring correction. It is a block diagram of the shake correction circuit with which the imaging device which concerns on 2nd Embodiment of this invention is provided. 10 is a flowchart of shake correction executed by the imaging apparatus according to the second embodiment of the present invention. It is a graph for demonstrating the calculation process of the motion vector interpolation value in 2nd Embodiment of this invention. It is a graph which shows the relationship between the image movement amount and correlation value between the present frame and the previous frame in the third embodiment of the present invention. It is a flowchart of the reliability determination process performed in the reliability determination circuit in 3rd Embodiment of this invention. It is a graph which shows the conversion characteristic of the vector value in the interpolation process circuit for calculating a motion vector interpolation value in the blurring correction which concerns on 4th Embodiment of this invention. It is a graph which shows another conversion characteristic of a vector value in an interpolation processing circuit for calculating a motion vector interpolation value in blur correction concerning a 4th embodiment of the present invention. It is a graph for demonstrating the correction method of the motion vector interpolation value based on 5th Embodiment of this invention. It is a flowchart of the correction method of the motion vector interpolation value based on 5th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram of a shake correction circuit provided in the imaging apparatus according to the first embodiment of the present invention. This shake correction circuit is applied to an imaging apparatus that suppresses shake by a vector detection method. The processing by the shake correction circuit described below is realized by a central processing unit (CPU) developing and executing programs for executing various processes stored in the ROM in the RAM.

  In the shake correction circuit shown in FIG. 1, first, an image signal (field or frame) to be detected as a motion vector is input from the input terminal 101 to the image memory 110 and the filter 102. In the following description, the field image signal is taken up (the same applies to the second embodiment).

  The image memory 110 temporarily stores an image signal, and the filter 102 extracts a spatial frequency component useful for motion vector detection from the image signal, that is, removes a low spatial frequency component and a high spatial frequency component from the image signal. The image signal that has passed through the filter 102 is input to the binarization circuit 103. The binarization circuit 103 binarizes the image signal with reference to the zero level, and specifically outputs the sign bit of the output signal. The image signal binarized by the binarization circuit 103 is input to the correlation calculation circuit 104 and the memory 105 as a one-field period delay means. The correlation calculation circuit 104 further receives the image signal of the previous field from the memory 105.

  The correlation calculation circuit 104 divides the image area into blocks of an appropriate size according to the block matching method, and performs a correlation calculation between the current field and the previous field in units of blocks. Note that the block matching method is described in detail in Non-Patent Document 1 described above, and thus the description thereof is omitted here. The motion vector detection circuit 106 detects a motion vector value, which is a displacement amount between fields, from the calculated correlation value in units of blocks. Specifically, the block of the previous field where the correlation value is minimum (the number of different code bits is minimum) is searched, and the relative shift is set as the motion vector value. The block-by-block motion vector value obtained by the motion vector detection circuit 106 is input to the motion vector determination circuit 107 and the reliability determination circuit 121. The motion vector determination circuit 107 determines the entire motion vector value (representative vector amount between fields) from the motion vector value in units of blocks. Specifically, the median value or average value of motion vector values in units of blocks is used as the motion vector value of the entire image.

  The reliability determination circuit 121 calculates the reliability (accuracy scale) of the entire motion vector value obtained by the motion vector determination circuit 107, specifically, calculates the variance value of the motion vector value in units of blocks, The reliability of the entire motion vector value is determined based on the variance value. The reliability determination circuit 121 determines that the reliability of the entire motion vector value is high when the variance value is small (that is, not distributed), and conversely the variance value is large (that is, distributed). In this case, it is determined that the reliability of the entire motion vector value is low. This is based on the fact that when the motion direction of the video signal is uniform, the motion vector value for each block obtained from the image is a uniform vector amount. Note that a reference value serving as a criterion for determining reliability in the reliability determining circuit 121 is determined in advance. The reliability determination result by the reliability determination circuit 121 and the entire motion vector value obtained by the motion vector determination circuit 107 are input to the interpolation processing circuit 122.

  The interpolation processing circuit 122 selects whether to use the entire motion vector value output from the motion vector determination circuit 107 or the motion vector interpolation value. The “motion vector interpolation value” in the first embodiment is a value used when the reliability determination circuit 121 determines that the reliability of the motion vector value in units of blocks is low. As the motion vector interpolation value, for example, the entire motion vector value used in the previous field is used. Therefore, the interpolation processing circuit 122 sequentially stores the entire used motion vector values.

  The vector value selected by the interpolation processing circuit 122 is input to the integration circuit 108, integrated, and converted into an integrated motion vector value. The accumulated motion vector value is input to the memory read control circuit 109. The memory readout control circuit 109 controls the readout position of the image memory 110 so that the motion of the image is canceled according to the accumulated motion vector value. A signal is output.

  Of the above-described components of the blur correction circuit, the filter 102, the binarization circuit 103, the correlation operation circuit 104, the memory 105, the motion vector detection circuit 106, and the motion vector determination circuit 107 constitute the representative vector detection circuit 22. To do.

  FIG. 2 is a block diagram illustrating a schematic configuration of an imaging apparatus such as a video camera provided with the above-described shake correction circuit. In this image pickup apparatus, a subject image formed on the light receiving surface of an image pickup device 13 such as a CCD through the image pickup optical system 12 is converted into an electric signal by the image pickup device 13 and then converted into an image signal by the camera signal processing circuit 14. It is converted into a standard video signal.

  The image signal obtained by the camera signal processing circuit 14 is input to the image memory 110 for each field and also input to the representative vector detection circuit 22. As described with reference to FIG. 1, the representative vector detection circuit 22 calculates the motion vector value in units of blocks and the entire motion vector value according to the motion of the image. Then, the motion vector value for each block is output to the reliability determination circuit 121, and the entire motion vector value is output to the interpolation processing circuit 122.

  Note that the control of the reliability determination circuit 121, the interpolation processing circuit 122, the integration circuit 108, the memory readout control circuit 109, and the image memory 110 is the same as the description of FIG. In this way, the video signal whose blur has been corrected is output from the image memory 110 to the recorder 17 and recorded in the recorder 17.

  FIG. 3 is a flowchart of shake correction processing executed by the imaging apparatus shown in FIG. 2, and shows processing from detection of motion vector values in units of blocks to output of a shake-corrected video signal.

  First, the motion vector detection circuit 106 detects a motion vector value in units of blocks (step S301). The process of step S301 is repeatedly executed in synchronization every time a motion vector value in units of blocks is detected. Subsequently, the motion vector value for each block detected in step S301 is taken into the motion vector determination circuit 107 and the reliability determination circuit 121 (step S302).

  Next, it is determined whether or not the calculation of all vector values in units of blocks in the field has been completed (step S303). If the calculation is complete (“YES” in S303), the process proceeds to step S304. If the calculation is not complete (“NO” in S303), the process ends.

  In step S304, the motion vector determination circuit 107 calculates one overall motion vector value from the motion vector values in units of blocks. After step S304, the reliability determination circuit 121 obtains a variance value of a plurality of block-unit motion vector values (step S305), and determines whether the variance value is larger than a predetermined reference value (step S306). . If the variance value is larger than the reference value (“YES” in S306), it is determined that the reliability is low, and the process proceeds to step S307. On the other hand, if the variance value is less than or equal to the reference value (“NO” in S306), it is determined that the reliability is high, and the process proceeds to step S308. Note that the reference value used in step S306 may be, for example, a value corresponding to a variance value when about 1/5 to 1/3 of the motion vector values in units of blocks obtained in one field match. However, it is desirable to appropriately set an appropriate value in consideration of the difference in shaking depending on the shape of the imaging device and the optical focal length.

  In step S307, the interpolation processing circuit 122 adopts a motion vector interpolation value (for example, the previous entire motion vector value) as a motion vector value, and determines to perform blur correction, and then the process proceeds to step S310. It is advanced. On the other hand, when the process proceeds to step S308, the interpolation processing circuit 122 adopts the current overall motion vector value and determines to perform the shake correction. Since it is determined that the current motion vector value is highly reliable, the interpolation processing circuit 122 stores the value so that it can be used as the next motion vector interpolation value (step S309). The process proceeds to step S310. At this time, the previously stored entire motion vector value may be deleted.

  In step S310, a shake correction value is obtained by integration calculation by the integration circuit 108, and the memory read control circuit 109 reads out from the image memory 110 based on the obtained shake correction value (step S311). The process ends.

  As described above, based on the reliability of the detected motion vector information, if the reliability of the motion vector value in units of blocks is low, the entire motion vector value is interpolated to impair continuity. Therefore, smooth correction can be realized. As a result, a sufficiently and accurately shake-corrected image can be stably obtained.

  Next, when it is determined that the reliability of the vector value in units of blocks is low, the case where blur correction is performed using the motion vector interpolation value according to step S307 in FIG. 3 and the case where blur correction is stopped are The comparison will be described.

  FIG. 4 shows changes in the overall motion vector value when the shake correction is performed using the motion vector interpolation value (example), and changes in the overall motion vector value when the shake correction is stopped (reference example). FIG. Each of FIGS. 4A to 4C shows the time change of the entire motion vector value. The horizontal axis represents time, and the vertical axis represents the amount of image shake (that is, the calculated total amount). Motion vector value).

  FIG. 4A assumes that the entire motion vector value is detected along with the shake 402 indicated by the broken line, and shows a case where the vector value is detected without error. . The entire motion vector value used for actual blur correction is two-dimensional (two axes), but here, for convenience of explanation, the entire motion vector value is shown one-dimensional (vertical one axis).

  The unit time 401 shown on the horizontal axis in FIGS. 4A to 4C is, for example, the field period of the video signal, and the entire motion vector value is obtained for each unit time 401. In FIG. 4A, a black arrow like an arrow 403 indicates an entire motion vector value that is considered highly reliable, and a white arrow like an arrow 404 is assumed to be less reliable. The entire motion vector value is shown. Here, a motion vector value having low reliability is assumed to have a large rate of change per unit time. The reason for this is that when the amount of change per unit time is large (and the amount of fluctuation per unit time is large), blurring occurs while accumulating in the image sensor, and the contrast of the captured image due to image blurring decreases. I think. This is because it is considered that the detection accuracy by the block matching method described above is lowered.

  As shown in FIG. 4A, when the entire motion vector value accompanying the shake 402 is ideally detected, the entire motion vector value overlaps the shake 402. On the other hand, in FIG. 4B, when it is determined that the reliability of the vector value in units of blocks is low, the blur correction is stopped (the entire motion vector value is not calculated and the motion vector interpolation value is calculated). The overall motion vector value change obtained by the conventional method is also shown. More specifically, in FIG. 4B, the entire motion vector value determined to have low reliability indicated by the arrow 404 in FIG. 4A is zero (0) as indicated by the arrow 405. Zero) vector value. As is clear from FIG. 4B, the change in the overall motion vector value obtained when the vector value is zero when it is determined that the reliability of the overall motion vector value in a certain unit time 401 is low. The state of is greatly different from the actual amount of shaking. In other words, it can be seen that sufficient blur correction is not realized, resulting in discontinuous control.

  On the other hand, in FIG. 4C, the entire motion vector in the case where the arrow 411 indicating the motion vector interpolation value is used in place of the vector value indicated by the arrow 405 whose value in FIG. It shows how the value changes. Instead of the entire motion vector value with low reliability (arrow 404), interpolation is performed with an arrow 411 indicating a motion vector interpolation value that is the same value as the previous entire motion vector value 402. As a result, it can be seen that the state of change of the entire motion vector value approaches the actual shake 402 and changes smoothly.

Second Embodiment
In the second embodiment, in the outline and reliability determination method, a value indicating reliability (hereinafter referred to as “reliability indication value”) is calculated on the basis of the correlation value between the current field and the previous field in block units. Then, the motion vector interpolation value is corrected by multiplying the motion vector interpolation value by a correction coefficient (the correction coefficient will be described in detail in FIG. 7) determined based on the reliability instruction value, and shake correction is performed.

  FIG. 5 is a block diagram of a shake correction circuit provided in the imaging apparatus according to the second embodiment of the present invention. Similarly, the configuration and operation of an imaging apparatus such as a video camera incorporating the shake correction circuit having the configuration shown in FIG. 5 is the same as the configuration and operation of the imaging apparatus of the first embodiment shown in FIG. Therefore, the description which overlaps with 1st Embodiment is omitted.

  The reliability determination circuit 121a obtains the reliability (scale of accuracy) of the entire motion vector value obtained in the motion vector determination circuit 107. The reliability determination circuit 121a obtains the magnitude of the correlation between fields based on the magnitude of the correlation value in block units obtained from the correlation calculation circuit 104, and the reliability of the motion vector value calculated based on the obtained magnitude of the correlation. Determine sex. The reliability of the calculated motion vector value is high when the correlation between the fields is small (the difference in code bit between fields can be close to 0), and conversely, the correlation between the fields is large. It becomes low when the difference in the sign bit between fields is larger than a predetermined amount.

  Specifically, in a high-contrast subject, the accuracy of the code bit obtained from the video signal is obtained, and when the subject movement between fields is sufficiently small relative to the detection area in block units, the correlation between the fields is The size becomes smaller. Therefore, the reliability of the calculated motion vector value is increased. On the other hand, when the contrast is low or the movement of the subject between the fields is large and blurring occurs during the accumulation time of the image sensor, the correlation between the fields is large. Therefore, the reliability of the calculated motion vector value is low.

  The accuracy of the binarization circuit 103 affects the magnitude of the correlation between fields. The higher the contrast, the better the binarization processing result can be obtained without depending on the conversion characteristics and accuracy of the binarization circuit 103. However, when the contrast is low, the conversion characteristics of the binarization circuit 103 and The effect of accuracy appears greatly.

Specifically, the reliability determination circuit 121a trusts a value obtained by subtracting from 1 the average value of the minimum correlation values (hereinafter referred to as “correlation average value”) obtained by the correlation calculation circuit 104. It is calculated as a value indicating the reliability (hereinafter referred to as “reliability indication value”). That is,
-Reliability indication value = 1-Correlation average value. Therefore, if the correlation average value is small, that is, the result that the correlation is high is obtained, the reliability indication value is large. The sex indication value becomes smaller. However, “0 ≦ 1-correlation average value = reliability indication value ≦ 1”.

  Next, the blur correction process described with reference to FIG. 5 will be described with reference to a flowchart. FIG. 6 is a flowchart of shake correction processing executed by the imaging apparatus according to the second embodiment. This flowchart shows processing from input of a captured image to blur correction by memory reading.

  First, when an image signal capturing a subject is input to the filter 102, the filter 102 performs the above-described filter signal processing (step S401). Subsequently, the image signal subjected to the filter signal processing is binarized by the binarization circuit 103 (step S402). Thereafter, the correlation calculation circuit 104 performs the above-described correlation calculation (correlation value calculation) (step S403). Subsequently, a motion vector value for each block is calculated by the motion vector detection circuit 106 (step S404).

  Then, it is determined whether or not the vector calculation for one field is completed (step S405). When the vector calculation for one field is completed (“YES” in S405), the process proceeds to step S406, and when it is not completed (“NO” in S405), the process ends.

  In step S406, the motion vector determination circuit 107 outputs one overall motion vector value. In step S407, the reliability determination circuit 121a calculates a reliability instruction value in the current field. Subsequently, in step S408, it is determined whether or not the reliability instruction value is smaller than the reliability reference value. Since the detection accuracy decreases as the reliability instruction value becomes smaller, it is determined whether the vector detection accuracy in this field is higher or lower than the reference detection accuracy.

  If the reliability instruction value is smaller than the reliability reference value (“YES” in S408), it is determined that the detection accuracy of the obtained motion vector value is low, and the process proceeds to step S409.

In step S409, the interpolation processing circuit 122 calculates a motion vector interpolation value from the motion vector value one field before and the reliability instruction value one field before. The calculated motion vector is used as a motion vector value used for blur correction calculation. The “motion vector interpolation value” in the second embodiment refers to, for example, multiplying the entire motion vector value determined one field before by the reliability instruction value obtained by the reliability determination circuit 121a one field before. Is a vector value obtained by That is,
Motion vector interpolation value = placement coefficient determined based on the reliability indication value × total motion vector value determined one field before.

  On the other hand, when the reliability instruction value is equal to or higher than the reliability reference value (“NO” in S408), it is determined that the detection accuracy of the obtained motion vector value is high, and the process proceeds to step S410. In step S410, the interpolation processing circuit 122 sets the entire motion vector value of the current field as a motion vector value used for blur correction calculation. Since the reliability of the entire motion vector value this time is high, the interpolation processing circuit 122 stores it so that it can be used as the next motion vector interpolation value (step S411). Further, the current correlation average value is also stored in order to calculate the next motion vector complement value (step S412). In step S411, the previous entire motion vector value stored in step S411 and the previous correlation average value stored in step S412 are deleted.

  After step S409 or step S412, a shake correction value is obtained by integration calculation by the integration circuit 108 (step S413). Then, reading from the image memory 110 is performed by the memory read control circuit 109 based on the obtained blur correction value (step S414), and thus a blur-corrected image signal is obtained. Thereafter, the process ends.

  Next, the motion vector interpolation value calculation process executed by the interpolation processing circuit 122 in step S409 in FIG. 6 will be described. FIG. 7 is a graph for explaining processing of motion vector interpolation values.

  In the graph of FIG. 7, the horizontal axis is the reliability instruction value of the previous field obtained by the reliability determination circuit 121a, and the vertical axis is the correction coefficient for obtaining the motion vector interpolation value. In FIG. 7, the reliability instruction value is multiplied by 100 and displayed in percent (%).

  As described above, the interpolation processing circuit 122 applies a predetermined correction coefficient determined in advance based on the reliability instruction value obtained by the reliability determination circuit 121a to the entire motion vector value detected and stored one field before. An interpolation vector value is calculated by multiplication.

  As shown in the graph of FIG. 7, for example, based on the first-order correlation characteristic 501, if the reliability indication value is 0.8, which is the reliability reference value, that is, 80%, the correction coefficient is multiplied by 1. . Then, the correction coefficient is decreased as the reliability indication value is reduced to, for example, 0.2, that is, 20%. In this embodiment, the reliability instruction value is 0.2 or less and the correction coefficient is 0. However, the correction coefficient is 0 if the reliability instruction value is greater than 0 and less than the reliability reference value. Any value is acceptable. In this embodiment, if the reliability instruction value is 80% or more, the motion vector interpolation value is not calculated itself (No in step S408 in FIG. 6 and the process proceeds to step S410, so the process in step S409 is performed). Correction factor is not necessary.

  As described above, when the reliability of the entire motion vector value is low, smooth blur correction is performed by using the motion vector interpolation value obtained by using the reliability instruction value obtained by the reliability determination circuit 121a. Can be realized. As a result, a sufficiently and accurately shake-corrected image can be stably obtained. Although the correlation characteristic 501 has been described as a linear function, it is not necessarily a linear function.

<Third Embodiment>
In the third embodiment, the reliability determination mechanism in the reliability determination circuit 121a included in the shake correction circuit according to the second embodiment in FIG. 5 is changed. Therefore, illustration is abbreviate | omitted about the imaging device which concerns on 3rd Embodiment, and the block diagram of the blurring correction circuit with which this imaging device is provided.

  A reliability determination circuit (corresponding to 121a in FIG. 5) included in the imaging apparatus according to the third embodiment uses a minimum correlation value (hereinafter referred to as “correlation minimum value”) and a pixel address that can take the minimum correlation value. Reliability is calculated based on the difference or ratio with the correlation value of adjacent pixels. In the first and second embodiments, the field image signal is taken up. In the third embodiment, the frame image signal is taken up. Of course, the third embodiment can be applied to a field image signal as it is.

  FIG. 8 is a graph showing the relationship between the image movement amount (= motion vector amount) between the current frame and the previous frame by the correlation calculation circuit 104 and the correlation value.

  In FIG. 8, a correlation value 551 indicated by a solid line indicates a correlation value with respect to an image movement amount when a subject with a small correlation difference between frames is photographed. A correlation value 555 indicated by a broken line indicates a correlation value with respect to an image movement amount when a subject having a large correlation difference between frames is captured.

  When block matching is used, in the case of a subject image with a high contrast such as a clear pattern (bright and dark or color arrangement is clear), the correlation difference between frames becomes small and takes the minimum value 552 shown in the correlation value 551. In the case of a high-contrast subject image such as a clear picture, the change in the correlation difference with respect to the image movement amount is clear, and thus the minimum correlation value can be obtained clearly. In such a subject condition in which the correlation value is clearly obtained, since the change of the correlation value with respect to the image movement amount is steep, the correlation value 553 of the image movement amount taken by both adjacent pixels adjacent to the pixel address having the minimum value 552. , 554 also becomes larger and the sharpness becomes sharper.

  On the other hand, in the case of a subject image with low contrast, such as a subject that is out of focus, since the change in the correlation difference with respect to the image movement amount is small, it is difficult to obtain the minimum correlation value. Furthermore, in subject conditions for which it is difficult to obtain a correlation value, the change in the correlation value with respect to the image movement amount is small, and the difference between the image movement amount correlation values 557 and 558 taken by the adjacent pixels adjacent to the pixel address having the minimum value 556. And the sharpness becomes dull.

  Therefore, if the difference between the pixel address that takes the minimum correlation value obtained when calculating the image movement amount and the correlation value of the adjacent pixels is obtained, the reliability of the overall motion vector value obtained if the difference is large. Is expensive. On the contrary, if the difference is small, it can be said that the reliability of the obtained entire motion vector value is low. The above description is about the relationship between the image movement amount on one axis and the correlation value. However, since the imaging apparatus actually has two orthogonal pixel arrays, the same processing is performed on each pixel array axis. Will be done.

  FIG. 9 is a flowchart of a reliability determination process performed in the reliability determination circuit according to the third embodiment. The process shown in this flowchart is obtained by changing a part of FIG. 6 and corresponds to the processes of steps S407 and S408 shown in the flowchart of FIG. 6 referred to in the description of the second embodiment. Therefore, the reliability calculation process of FIG. 9 is started after the calculation of the motion vector value of the entire step S406 shown in the flowchart of FIG.

  In the reliability calculation process, first, the correlation calculation circuit 104 searches for a correlation value to obtain a minimum correlation value (step S601). Subsequently, the difference between the calculated minimum correlation value and the correlation value of the pixels adjacent to both pixels adjacent to the pixel having the minimum correlation value is determined (step S602). The calculation in step S602 can be performed by, for example, “difference = | minimum correlation value− (correlation value of one adjacent pixel + correlation value of the other adjacent pixel) / 2 |”.

  It is determined whether or not the difference thus obtained is larger than a predetermined reference value (step S603). The predetermined reference value here is, for example, 10% of the maximum correlation value (that is, the value when the vector detection blocks do not match). If the difference is larger than the reference value (“YES” in S603), it is determined that the reliability is high (step S604). On the other hand, when the difference is equal to or smaller than the reference value (“NO” in S603), it is determined that the reliability is low (step S604). When the determinations in steps S604 and 605 are made, the reliability determination process ends.

  If the determination in step S604 is made, the current overall motion vector value is used for shake correction calculation. If the determination in step S605 is made, the motion vector interpolation value is used for shake correction calculation. Will be used.

  By performing the reliability determination of the third embodiment as described above, continuous and smooth blur correction can be realized. As a result, a sufficiently and accurately shake-corrected image can be stably obtained.

  In the above embodiment, the reliability is determined by obtaining the difference between the correlation minimum value and the correlation value of the adjacent pixels adjacent to the pixel address at which the correlation minimum value can be obtained. However, the present invention is not limited to this, and the same reliability determination can be performed by determining the ratio between the correlation minimum value and the correlation value of the pixels adjacent to the pixel address that can take the correlation minimum value. As an arithmetic expression at that time, for example, “ratio = minimum correlation value / (correlation value of one adjacent pixel + correlation value of the other adjacent pixel)” can be used.

<Fourth embodiment>
In the fourth embodiment, the interpolation processing in the interpolation processing circuit 122 included in the shake correction circuit according to the second embodiment is changed, and the magnitude of the motion vector value used for calculating the motion vector interpolation value is changed. Accordingly, the newly calculated motion vector interpolation value is corrected. Note that the image pickup apparatus according to the fourth embodiment, the block diagram of the shake correction circuit included in the image pickup apparatus, and the flowchart of the shake correction processing are the same as those in the second embodiment (FIGS. 5 and 6). Is omitted.

  FIG. 10 is a graph showing vector value conversion characteristics in the interpolation processing circuit for calculating a motion vector interpolation value. In the graph of FIG. 10, the horizontal axis represents the entire motion vector value stored in the previous (previous field), and the vertical axis represents the motion vector interpolation actually calculated according to the entire motion vector value on the horizontal axis. The value (corrected motion vector interpolation value) is shown.

  According to the conversion characteristics (input / output characteristics) 561 shown in the graph of FIG. 10, when the entire motion vector value stored last time is small, the motion is the same value as the entire motion vector value stored last time. A vector interpolation value is output.

When the total motion vector value stored last time exceeds a predetermined value (10 pixels in FIG. 1), the output motion vector value becomes the previous value as the total motion vector value stored last time increases. Correction is made to be smaller than the entire stored motion vector value. Thus, in the fourth embodiment, the maximum value of the motion vector interpolation value is limited. That is,
-Motion vector interpolation value after correction = Total motion vector value stored last time (Total motion vector value stored last time ≤ Predetermined value),
-Motion vector interpolation value after correction <Overall motion vector value stored last time (predetermined value ≤ Overall motion vector value stored last time)
It becomes.

  The reason for performing such correction is as follows. That is, in general, a large motion vector value has a large amount of blur during the accumulation time of the image sensor as described above, and it is considered that the detection accuracy is lowered. Therefore, there is a high possibility that it is not an accurate overall motion vector value, and due to the limitation that the correction range is finite and image quality degradation due to accumulated blur in the image sensor, all continuous large motion vector values are It is considered that it is not always necessary to perform blur correction.

  The conversion characteristic of the vector value in the interpolation processing circuit for calculating the motion vector interpolation value is not limited to the conversion characteristic 561 shown in FIG. FIG. 11 is a graph showing another conversion characteristic of the vector value in the interpolation processing circuit for calculating the motion vector interpolation value.

According to the conversion characteristic 566 shown in the graph of FIG. 11, when the entire motion vector value stored last time is equal to or less than a predetermined value, motion vector interpolation is performed with the same value as the entire motion vector value stored last time. The value is output. When the entire motion vector value stored last time is larger than a predetermined value, the output motion vector value is replaced with the predetermined value. That is,
・ Motion vector interpolation value after correction = total motion vector value stored last time (total motion vector value stored last time ≦ predetermined value) ”,
-Motion vector interpolation value after correction = Predetermined value (predetermined value ≤ overall motion vector value stored in the previous time)
It becomes.

  Even if any of the conversion characteristics 561 and 566 shown in FIGS. 10 and 11 is used, continuous and smooth blur correction can be realized. As a result, a sufficiently and accurately shake-corrected image can be stably obtained.

  Note that the entire motion vector value used for actual blur correction is two-dimensional (two axes), but here, for convenience of explanation, the entire motion vector value is shown one-dimensional (vertical one axis).

<Fifth Embodiment>
In the fifth embodiment, the interpolation processing in the interpolation processing circuit 122 included in the shake correction circuit according to the second embodiment is changed, and the time of the entire motion vector value used for calculating the motion vector interpolation value is changed. The newly calculated motion vector interpolation value is corrected according to the change with respect to the axis. The block diagram of the image pickup apparatus according to the fifth embodiment and the shake correction circuit included in the image pickup apparatus is the same as that of the second embodiment, and is not shown.

  FIG. 12 is a graph for explaining a method of correcting a motion vector interpolation value. The horizontal axis of this graph is the time axis, the entire motion vector value is shown for each field in which the entire motion vector value is detected, and the vertical axis indicates the amount of shaking (the entire motion vector value). Yes. Therefore, this graph shows the change of the entire motion vector value with respect to the time axis.

  In FIG. 12, in the vectors 570, 571, 572, and 573, the amount of change in the vector value tends to increase. On the other hand, in the vectors 580, 581, 582, and 583, the amount of change in the vector value tends to decrease.

  Here, for example, since the reliability of the vector value of the vector 573 is low, when the motion vector interpolation value is used without using it, the change in the vector value tends to increase, so the motion vector interpolation value is also stored in the previous time. It increases compared to the vector value corresponding to the vector 572.

  On the contrary, since the reliability of the vector value of the vector 583 is low, when the motion vector interpolation value is used without using it, the change of the vector value tends to decrease. It decreases compared to the vector value corresponding to 582.

  For this reason, when an increasing tendency is observed, it is preferable to correct the motion vector interpolated value in consideration of the increasing amount. On the other hand, when a decreasing trend is observed, the correction is performed in consideration of the decreasing amount. This is preferable.

  Hereinafter, an interpolation process with such correction will be described below. FIG. 13 is a flowchart of a method for correcting a motion vector interpolation value. In this correction method, the interpolation processing circuit 122 needs to store two consecutive three motion vector values, and this is different from the processing in the flowchart shown in FIG.

  The calculation of the motion vector interpolation value is repeatedly performed in accordance with the timing at which the motion vector values of a plurality of blocks are calculated. First, it is determined whether or not three consecutive motion vector values stored in the interpolation processing circuit 122 tend to increase in time axis order (step S701). For example, it is determined whether or not vector 570 <vector 571 <vector 572. As a result, if it is determined that there is no tendency to increase (“NO” in S701), the process proceeds to step S702. On the other hand, if it is determined that the trend is increasing (“YES” in S701), the process proceeds to step S703.

  In step S702, it is determined whether or not the three consecutive motion vector values stored in the interpolation processing circuit 122 tend to decrease in time axis order. For example, whether or not vector 580> vector 581> vector 582 is determined. As a result, if it is determined that the trend is not decreasing (“NO” in S702), the process proceeds to step S704. On the other hand, if it is determined that the trend is decreasing (“YES” in S702), the process is The process proceeds to step S705.

  In step S703, since it is determined that there is a continuous increasing tendency, the motion vector interpolation value is corrected with an increasing tendency with respect to the previously stored motion vector value. This correction calculation can be performed using, for example, the equation “motion vector interpolation value = motion vector value stored last time + (motion vector value stored last time−motion vector value stored last time)”. it can.

  In step S704, since there is neither an increase tendency nor a decrease tendency, the motion vector value stored last time is used as it is as a motion vector interpolation value.

In step 705, since it is determined that there is a continuous decreasing tendency, the motion vector interpolation value is corrected with a decreasing tendency with respect to the previously stored motion vector value. This correction calculation is, for example,
Motion vector interpolation value = previously stored motion vector value − (motion vector value stored last time − previously stored motion vector value)
The following equation can be used. Thus, the motion vector interpolation value to be actually used is determined, and the process ends.

  Thus, in the fifth embodiment, continuous and smooth blur correction can be realized by performing correction according to the change of the motion vector value with respect to the time axis with respect to the motion vector interpolation value. As a result, a sufficiently and accurately shake-corrected image can be stably obtained.

  Note that the entire motion vector value used for actual blur correction is two-dimensional (two axes), but here, for convenience of explanation, the entire motion vector value is shown one-dimensional (vertical one axis).

<Other embodiments>
The embodiment of the present invention has been described above. However, the above-described embodiment shows an example of the present invention, and the present invention is not limited to this, and the configuration and operation of each embodiment. These can be appropriately changed within the scope of the gist of the present invention. For example, the present invention can be realized by executing the following processing.

  For example, the present invention can be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed.

DESCRIPTION OF SYMBOLS 12 Imaging optical system 22 Representative vector detection circuit 121,121a Reliability determination circuit 122 Interpolation processing circuit

Claims (4)

Imaging means for converting an object image formed on the image signals by the projection optical system shooting,
Storage means for storing an image signal obtained by the imaging means;
Motion vector detection means for detecting a motion vector value indicating the amount of displacement of the image from the two image signals obtained by the image pickup means,
Reading position control means for controlling the reading position of the image signal from the storage means based on the motion vector value ;
An imaging apparatus that have a reliable and judging means for judging the reliability of said motion vector values,
Comprising an interpolation means to determine the motion vector interpolation value interpolated with the motion vector value based on the determination result by the pre-Symbol reliability judging means,
The interpolating unit generates a motion vector interpolated value obtained by interpolating a motion vector value indicating a displacement amount of an image of the current frame or the current field when the reliability determined by the reliability determining unit is lower than a predetermined value. When the change in the motion vector value of the frame or field before the current frame or the current field tends to increase, the motion vector interpolation value is converted into the motion vector value of the previous frame or previous field stored in the storage means. When the change in the motion vector value of the current frame or the previous frame or field tends to decrease, the motion vector interpolation value of the previous frame or previous field stored in the storage means is increased. An imaging apparatus, characterized by being reduced compared to a motion vector value . The motion vector detection means includes correlation calculation means for obtaining a correlation value between the current field and the previous field or a correlation value between the current frame and the previous frame for the image signal,
The imaging apparatus according to claim 1, wherein the reliability determination unit calculates the reliability of the motion vector value based on a magnitude of a correlation value obtained from the correlation calculation unit. The reliability determination means is based on a difference or a ratio between a minimum correlation value obtained by the correlation calculation means and a correlation value between pixels adjacent to a pixel address that can take the minimum correlation value. The imaging apparatus according to claim 2 , wherein the image is determined. Imaging means, an imaging step for converting an object image formed by the projection optical system Taking into image signals,
Storage means, a storing step of storing an image signal obtained by the imaging step,
A motion vector detecting step for detecting a motion vector value indicating a displacement amount of the image from the two image signals obtained in the imaging step;
A reading step for reading position control means for controlling the reading position of the image signal stored in the storing step based on the motion vector value ;
A determination step in which reliability determination means determines the reliability of the motion vector value obtained in the motion vector detection step;
An interpolating step for determining a motion vector interpolated value obtained by interpolating the motion vector value based on a determination result in the determining step ;
In the interpolation step, when the reliability determined in the determination step is lower than a predetermined value and a motion vector interpolation value obtained by interpolating a motion vector value indicating a displacement amount of an image of the current frame or current field is generated, When the change in the motion vector value of the current frame or the frame or field before the current field tends to increase, the motion vector interpolation value is compared with the motion vector value of the previous frame or previous field stored in the storing step. When the change in the motion vector value of the current frame or the previous frame or field tends to decrease, the motion vector interpolation value is stored in the previous frame or previous field stored in the storing step. control of an imaging apparatus characterized by decreased as compared with the value Method.

Images