JP2661330B2 - Motion vector detection device and image shake correction device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector detecting device for detecting a motion amount of an image and an image shake correcting device.

2. Description of the Related Art An example of a conventional motion vector detecting device for an image is disclosed in, for example, JP-A-61-269475.

FIG. 7 shows a schematic diagram of this, wherein 1 is a latch A, 2 is a representative point memory, 3 is a latch B, 4 is a subtractor, 5 is an address controller, 6 is an absolute value converter, 7
Denotes a cumulative adder, 8 denotes a correlation search unit, 9 denotes a correlation validity / invalidity determination unit, and 10 denotes a motion vector determination unit A.

An image motion vector detecting device using the conventional correlation operation device configured as described above will be described.

First, a motion vector of an image will be described. FIG. 8A shows an image at a certain time. (B) shows an image one field or one frame later. As described above, when the image moves in parallel due to the movement of the imaging device or the like, the amount of parallel movement of the image as indicated by the arrow in FIG.

FIG. 9 shows the state of a representative point and pixels around the representative point in the representative point matching method, which is a method for detecting such a motion vector of an image. The motion vector detection is performed by detecting, using the correlation, where the image data at the position of the representative point in a certain field has moved among the surrounding pixels in the next field.

Next, an image motion vector detecting device using a conventional correlation operation device will be described with reference to FIG.

The image data at each representative point on the screen is fetched into the latch A1 by the timing pulse LP1, and is written at an address corresponding to each representative point in the representative point storage memory 2 with a timing. Then, in the next field or the next frame, the correlation between the image data in the motion vector detection area around the position of each representative point and the representative point of the previous field stored in the representative point memory 2 (here, the absolute value difference) And input it to the accumulator 7. The accumulative adder 7 accumulatively adds data having a correlation at a position where the coordinate position with respect to the representative point is the same. Then, when the cumulative addition around all the representative points is completed, the location having the highest correlation value among the cumulative added values held in the cumulative adder 7 by the correlation search unit 8 is determined. That is, the position (address) having the position having the highest correlation with respect to the position of the representative point.
Becomes the motion vector. In the configuration shown in FIG. 7, since the correlation calculation is performed by cumulative addition of the absolute values of the differences, the data value of the point with high correlation in the cumulative adder 7 has a lower level than the data values of the other points. . Furthermore, it is determined whether the motion vector obtained by the correlation operation is valid or invalid based on the distribution of the correlation values around the representative point (average value, minimum value, maximum value, etc.). When the average value is low, when the minimum value is high, and when the minimum correlation value / maximum correlation value around the minimum point of the correlation value is small, it is determined that the motion vector obtained by the correlation operation is invalid. This determination is performed by the invalidity determination unit 9.

The operation described above is performed for each area obtained by dividing the screen into a plurality. Then, the motion vector of the entire screen is determined by the determiner A10 based on the motion vector obtained from each area of the screen and the invalidity determination information thereof.

Since the operation up to this point is performed for each field (frame), the next field (frame) is used while using the correlation operation.
There is a latch 7A1 for storing the image data at the representative point for the correlation calculation. In addition, the latch B3 stores the image data of the representative point when correlating the image data of a certain representative point with the image data of its surroundings.

A seventh broken line portion, that is, a portion for detecting a motion vector by correlation calculation is referred to as a motion vector detection unit A20.

SUMMARY OF THE INVENTION However, in the above-described configuration, it is sufficient that only a point where the image is shifted due to the movement of the imaging device has a high correlation with a low correlation at each position of the subject, but a regular correlation with the subject is obtained. If there is, the following problem occurs. If the subject has a regular correlation, many points with high correlation are obtained, and to detect this, the minimum correlation value / maximum correlation value around the minimum correlation value is adjusted.
In the judgment based on the minimum correlation value / maximum correlation value, sufficient detection cannot be performed under all conditions, and the motion vector obtained from each region becomes a position of a point that is not related to the original highly correlated motion vector in the image. The motion vectors in each area are not the same. As a result, when the motion vector of the entire screen is determined by averaging the motion vectors of the respective areas and the like, the candidate point of the motion vector of the area showing a point having a high correlation of the image different from the motion of the entire screen is a cause. Has a problem that the detection of the motion vector is incorrect.

In view of such a point, the present invention detects a condition in which a subject has a lot of correlations and an obtained motion vector changes drastically, and when this condition is detected, the motion vector obtained from each region is compared with the motion vector. The purpose is to suppress.

Means for Solving the Problems To achieve the above object, a motion vector detection device of the present invention divides an input image into a plurality of regions, and calculates a motion vector in each region; A vector feature detection unit that detects the magnitude of a temporal change of a motion vector output from the unit, and a vector that controls the size of the motion vector output from the motion vector detection unit based on the output of the vector feature detection unit. A controller, and a motion vector determining unit that determines a motion vector of the input image from a motion vector whose magnitude is controlled by the vector controller, wherein the vector controller is output from the motion vector detecting unit. When the temporal change of the motion vector is larger than a predetermined level, the magnitude of the motion vector output from the motion vector detection unit is large. It is characterized by making the magnitude zero.

Further, the image shake correcting device of the present invention is characterized by including the motion vector detecting device and an output controller for canceling image shake in accordance with a motion vector output from the motion vector detecting device.

Operation The present invention employs the above-described configuration. The vector feature detection unit detects a condition in which the subject has many correlations and the obtained motion vector changes drastically, and when this condition is detected, the condition is obtained from each region. This suppresses the motion vector.

Further, since the image shake is corrected based on the motion vector obtained in this way, erroneous shake correction is considerably reduced.

Embodiment 1 FIG. 1 is a block diagram of a motion vector detecting device according to a first embodiment of the present invention, wherein 1 is a latch A, 2 is a representative point memory, 3 is a latch B, 4 is a subtractor, and 5 is an address controller. , 6 are absolute value converters, 7 is a cumulative adder, 8 is a correlation search unit, 9 is a correlation invalidity determination unit, and 20 motion vector detection units A composed of blocks 1 to 9 are conventional. And the same numbers are assigned. Reference numeral 11 denotes a motion vector feature detection unit, reference numeral 12 denotes a motion vector control unit, and reference numeral 13 denotes a motion vector determination unit.

The image data of the input signal at each representative point on the screen is taken into the latch A1 by the timing pulse LP1, and is written into the address corresponding to each representative point in the representative point storage memory 2 at a certain timing. Then, in the next field or the next frame, the correlation between the image data in the motion vector detection area around the position of each representative point and the representative point of the previous field stored in the representative point memory 2 is calculated. To enter. The correlation calculation here is performed by cumulative addition of the absolute values of the differences. The accumulative adder 7 accumulates and adds the correlated data at the same position of the coordinates with respect to the representative point. Then, when the cumulative addition around all the representative points is completed, the location having the highest correlation value among the cumulative added values held in the cumulative adder 7 by the correlation search unit 8 is determined. In other words, the position (address) having the highest correlation value with respect to the position of the representative point is the motion vector.

In the configuration of the present invention, since the correlation operation is performed by accumulative addition of the difference absolute values, the data value of the point having a high correlation in the accumulator 7 has a lower level than the data values of the other points. This point is obtained by the correlation search unit 8. Further, based on the distribution of correlation values around the representative point (average value, minimum value, gradient, etc.), it is determined whether the motion vector candidate obtained by the correlation search unit 8 is valid or invalid. The determination of validity or invalidity may be the same as in the conventional example. For example, the following condition can be used as the condition of the correlation value.

Average value≫minimum value and average value> C1 (constant) and maximum value≫minimum This determination is made by the invalidity determination unit 9.

Further, the motion vector feature detection unit 11 performs the feature of the motion vector independently of the invalidity determination. FIG. 2 shows the configuration of the motion vector feature detection unit 11. In the figure, 20 is a change detector, 21 and 24 are absolute value converters, 22 and 24 are low-pass filters, and 25 is a normalizer. The operation of the motion vector feature detecting unit 11 configured as described above will be described below.

The motion vector feature detection unit 11 separates a feature of a motion vector when the imaging device is held in a hand or an image from a car or the like from a feature of a motion vector when an image of a subject having a regular correlation in the image is captured. FIG. 3 shows typical changes in motion vectors obtained under these conditions. FIG. 11A shows a horizontal motion vector when the imaging apparatus is held in a hand, and FIG. 10B shows horizontal motion vectors when an image is captured from a car or the like. ) Is a horizontal motion vector when an image of a subject having many regular correlations is captured in the image. As shown in FIG. 3, when an image of a subject having many regular correlations is picked up in an image, the obtained motion vector changes rapidly and the number of times of change is large (high frequency). In order to separate these conditions, the change component of the input motion vector is detected by the change detector 20. The operation performed by the change detector 20 is as follows: S1x_t (n) = Vx_t (n-1) -Vx_t (n) S2x_t (n) = S1x_t (n-1) -S1x_t (n) S1y_t (n) = Vy_t (n-1) ) -Vy_t (n) S2y_t (n) = S1y_t (n-1) -S1y_t (n) Here, V is a motion vector, and subscripts x and y indicate a horizontal direction and a vertical direction. Also, t (n) indicates a field from which a vector can be obtained. S1 is an intermediate result of the calculation, and S2 is a change component of the vector. The subscript is the same as the motion vector V. The absolute value of the change component obtained in this way is obtained by an absolute value converter 21 and passed through a low-pass filter (LPF) 22 to attenuate a sudden change, and a signal CH representing the obtained characteristics is normalized by a normalizer. Send to 25.

At the same time, the signal RF to be standardized is converted to an absolute value converter 23.
And LPF24. The signal RF for normalization is obtained by obtaining the absolute value of a motion vector and attenuating a sudden change component. Using the standardization reference signal RF obtained in this way, the normalization represented by the following equation is performed.

Gv = A * CH / (RF + O) Here, A and O are coefficients, and the optimum value is selected in each system. By performing the standardization in this way, the third
The level can be relatively increased even under the condition where the amplitude of the motion vector is small as shown in FIG. 3D, and the condition of FIG. 3D can be stably separated.

Using the control signal Gv obtained as described above, a coefficient for multiplying the motion vector is obtained by the motion vector controller 12, and the coefficient is multiplied by the motion vector. The characteristics of the motion vector controller 12 are shown in FIG. The horizontal axis is the output Gv of the motion vector feature detection unit, and the vertical axis is the value of the gain multiplied by the obtained motion vector.

In this way, when the output level of the motion vector feature detection unit is high, that is, when there are many regular correlations in the image being captured, the motion vector is compressed to zero under the condition that an incorrect motion vector is detected. Becomes possible.

The operations up to this point are performed for each area obtained by dividing the screen into a plurality. Since the operation up to this point is performed every field (frame), there is a latch A1 for storing the image data at the representative point for the correlation operation of the next field (frame) while performing the correlation operation. Also, latch B3
Holds the image data of a representative point when correlating the image data of a certain representative point with the image data of its surroundings.

In this way, motion vector candidates are determined from each of the divided regions, and these candidate vectors are subjected to a median filter by the motion vector determination unit 13 to be used as motion vectors of the input signal.

By obtaining the motion vector of the input signal as described above, highly correlated points are regularly present in the image,
Under the condition for detecting an erroneous motion vector, the magnitude of the motion vector is set to zero, and the erroneously detected motion vector is not used. Therefore, even if the motion vector is invalid as a conventional motion vector, but the judgment cannot be made and an erroneous motion vector is detected, the value can be set to zero.
The output of an incorrect motion vector is eliminated.

Next, an image fluctuation correcting apparatus using the motion vector detecting apparatus of the present invention will be described. FIG. 5 is a block diagram of an embodiment of the image shake correction apparatus. In the figure, 18 is a memory for accumulating image signals, 16 is an interpolation unit for interpolating the signals of the memory, 17 is an output control unit for controlling the output of the image signal, 50 is a motion vector detection device of the present invention, The same reference numerals as those in FIG. 1 denote the same parts, and a description thereof will be omitted.

The operation of the image shake correction apparatus configured as described above will be described.

In FIG. 5, the method of obtaining the motion vector is the same as in the above-described embodiment, and the description thereof will be omitted. The motion vector of the image signal input from the motion vector detection device 50 is output and input to the output control unit 17. The output control unit 17 accumulates the motion vector input for each field (frame), and corresponds to the positive part of the accumulated motion vector (when the motion vector is represented by the address of the memory where the image signal is recorded). Thus, the address of the signal read of the memory 18 is determined. In addition, the control of the interpolation unit 16 is performed in accordance with the decimal part of the motion vector. The interpolation is linear interpolation in which weights corresponding to the reciprocal of the distance are multiplied in both the vertical and horizontal directions. In this way, the output control unit 17 shifts the image in the direction opposite to the swing according to the motion vector, and cancels the swing of the image.
Further, the output control unit 17 converts the image to be output into one of the original images,
When the image inside the memory 18 is shifted left, right, up and down and output by enlarging it from about 1 to 1.5 times, the "edge" is made invisible.

As described above, by using the motion vector detection device of the present invention and configuring the image shake correction device, under the condition that points having high correlation are regularly present in the image and an erroneous motion vector is detected, the motion vector Assuming the size to be zero,
It is possible not to use a motion vector that is erroneously detected for shake correction. Therefore, even if a highly correlated point in the conventional image is regularly present and invalid as a motion vector, the determination cannot be made and an erroneous motion vector is detected and erroneous shake correction is performed. The vector can be set to zero, and erroneous shaking correction can be eliminated. An image shake correction apparatus can be realized.

In the embodiment of the motion vector detecting device of the present invention,
Although the configuration of the motion vector feature detecting section 11 is as shown in FIG. 2, it is possible to use the HPF 30 as a substitute for the change detector 20 as shown in FIG. The cutoff frequency of the HPF is appropriately from about 5 Hz to 15 Hz.

Furthermore, in the embodiment of the motion vector detection device of the present invention, the calculation method of the motion vector feature detection unit 11 need not be limited to the method of the present embodiment, but may be a similar method that meets the purpose of the present invention. is there.

Further, in the embodiment of the motion vector detecting device according to the present invention, the motion vector feature detecting section 11 has standardized according to the magnitude of the vector. However, it goes without saying that the normalizing method may be other than this embodiment. Further, it is possible to perform the processing easily without performing the standardization.

According to the motion vector detection device of the present invention, points having high correlation are regularly present in an image, and under the condition of detecting an erroneous motion vector in the past, the size of the motion vector is assumed to be zero and erroneous detection is performed. Do not use the specified motion vector. Therefore, although it is invalid as a conventional motion vector,
Even when an erroneous motion vector is detected because the determination cannot be made, the value can be set to zero, and an erroneous motion vector is not output.

In addition, by using the motion vector detection device of the present invention and configuring the image shake correction device, points having high correlation are regularly present in the image, and the condition for detecting an erroneous motion vector in the related art is that the size of the motion vector is large. By setting the value to zero, it is possible not to use a motion vector that is erroneously detected for shake correction. Therefore, a highly correlated point is regularly present in the conventional image, and is invalid as a motion vector.
Even when erroneous shaking correction is performed, the vector can be made zero, and it is possible to eliminate the erroneous shaking correction, thereby realizing an image shaking correction device.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a motion vector detecting device according to an embodiment of the present invention. FIG. 2 is a block diagram showing a configuration of a vector feature detecting section of the motion vector detecting device according to an embodiment of the present invention. 3 is a waveform diagram showing a state of a detection vector of the motion vector detecting device according to one embodiment of the present invention, FIG. 4 is a characteristic diagram showing characteristics of a motion vector controller, and FIG. FIG. 6 is a block diagram showing the configuration of an image shake correcting device, FIG. 6 is a block diagram showing the configuration of a second embodiment of the motion vector feature detecting section of the motion vector detecting device according to one embodiment of the present invention, and FIG. Block diagram showing the configuration of an example motion vector detection device,
8 and 9 are explanatory diagrams of the operation state of the motion vector detecting device in the conventional example. 2 ... Representative point storage memory, 7 ... Cumulative adder, 8 ... Correlation search unit, 9 ... Invalidity judgment unit, 11 ... Motion vector feature detection unit, 12 ... Motion vector control unit, 13 ... A motion vector determination unit, 16 ... an interpolation unit, 17 ... output control, 50 ... a motion vector detection device;

Claims (2)

(57) [Claims] A motion vector detecting unit that divides an input image into a plurality of regions and calculates a motion vector in each region;
A vector feature detection unit that detects a magnitude of a temporal change of a motion vector output from the motion vector detection unit; and a magnitude of a motion vector output from the motion vector detection unit based on an output of the vector feature detection unit. And a motion vector determining unit that determines a motion vector of the input image from a motion vector whose size is controlled by the vector controller, wherein the vector controller includes the motion vector detecting unit. A motion vector output from the motion vector detection unit, wherein the magnitude of the motion vector output from the motion vector detection unit is set to zero when the temporal change of the motion vector output from the motion vector is greater than a predetermined level. 2. A motion vector detecting unit for decomposing an input image into a plurality of regions and calculating a motion vector in each region;
A vector feature detection unit that detects a magnitude of a temporal change of a motion vector output from the motion vector detection unit; and a magnitude of a motion vector output from the motion vector detection unit based on an output of the vector feature detection unit. And a motion vector determination unit that determines a motion vector of the input image based on a motion vector whose magnitude is controlled by the vector controller, wherein the vector controller includes the motion vector When the temporal change of the motion vector output from the detection unit is greater than a predetermined level, a motion vector detection device that sets the magnitude of the motion vector output from the motion vector detection unit to zero, and An image controller comprising an output controller for canceling image fluctuation in accordance with an output motion vector. Correction device.