JPH04117089A - Moving image motion detecting system - Google Patents

Abstract

PURPOSE:To perform motion detection without being affected by the other surrounding areas by detecting motion on an image in with only the area to be subject to motion detection extracted. CONSTITUTION:An area dividing means 11 dividing an image into plural areas and outputting area division information, an area extraction means 12 generating an area extraction image based on the area division information and a motion vector detection means 13 detecting and outputing a motion vector between the frames of the extracted area from the area extraction image are provided. An inputted image is divided into plural independent areas A to D, the area A to be subject to the motion detection is extracted based on an area division result, the area extraction image is generated, and motion detection for the area A is performed. Thus, all the signals in the surrounding area are eliminated, and an accurate motion detection can be executed without being affected by the surrounding areas in detecting the motion at the border part of the area A.

Description

Detailed Description of the Invention (Industrial Application Field) The present invention relates to a motion detection method for moving images used for motion compensated interframe predictive coding of moving @ image signals, motion recognition of moving objects in images, etc. Regarding.

(Prior Art) Conventionally, a block matching method has been used as a motion detection method for moving images. This detection method divides the supplied image into small blocks of a predetermined size, performs interframe block matching for each small block, detects the position with the strongest correlation on the previous frame, and Outputs the interframe motion vector of the block.

Additionally, a gradient method may be used as another motion detection method. This detection method assumes that the brightness change on the image plane is continuous, and calculates the interframe motion vector for each pixel or small block from the brightness gradient at the motion detection position and the amount of interframe brightness change. Find it by

(Problem to be Solved by the Invention) By the way, assuming that there is an independent region that moves uniformly on the image plane, and considering this as the object of motion detection, motion detection using block masoching for each small block is possible. With this method, motion vector errors are often detected in blocks that span multiple regions with different motions. Furthermore, since the block matching method assumes that the movement on the image plane is parallel movement, there are many false detections when the image is enlarged, reduced, or rotated.

On the other hand, since the gradient method assumes that the brightness on the image plane changes continuously, the error rate of motion detection increases at area boundary parts where the brightness changes sharply. Similarly, the false detection rate also increases in areas where there is little change in brightness within the area. In other words, in the block matching method and gradient method,
Different motion vectors may be detected within a region where a uniform motion vector should be generated.

An object of the present invention is to provide a moving image that can detect the movement of each independent area on the image plane with more precision, and can also perform highly accurate motion detection of enlargement, reduction, and rotational movements of the image. The object of the present invention is to provide a motion detection method.

(Means for Solving the Problems) A first motion detection method for moving images according to the present invention is a motion detection method that performs interframe motion detection for each divided region of a moving image, and divides an image into multiple regions. a region dividing means for dividing the region and outputting region division information; a region extracting means for generating a region extraction image in which each region is extracted based on the region division information; and an inter-frame movement of the corresponding extraction region from the region extraction image. It is characterized by comprising a motion vector detection means for detecting a vector and outputting the motion vector.

The second moving image motion detection method according to the present invention is a motion detection method that performs interframe motion detection for each divided region of a moving image, and divides the image into a plurality of regions and outputs region division information. a region dividing means, a region extracting means for generating a region extraction image in which each region is extracted based on the region division information, and a corner of a region outline that is a feature point of the extraction region based on the region extraction image. The present invention is characterized by comprising a corner detection means, and a motion vector detection means for detecting the movement of the corner from the inter-frame corresponding position of the corner detected for each extraction area, and outputting the movement of the corner as a motion vector of the corresponding area. shall be.

The third motion detection method for moving images according to the present invention is a motion detection method that performs interframe motion detection for each divided region of a moving image, and divides the image f into a plurality of regions and uses the region division information. A region dividing means for outputting, a region extracting means for generating a region extraction image by extracting each region based on the region division information, and a region extracting means for generating a region extraction image by extracting each region based on the region division information; a corner detection means for detecting a corner; a patterning means for generating a patterned image from the detected corner and a series of edge points connected thereto; a patterning means for cutting out a region near the corner from the patterned image as a template; The present invention is characterized by comprising a motion vector detecting means for detecting inter-frame motion of the corner by performing weighted inter-frame matching, and outputting the motion of the corner as a motion vector of the corresponding extraction region.

The fourth motion detection method for moving images according to the present invention is a motion detection method that performs interframe motion detection for each divided region of a color moving image, and is a motion detection method that performs inter-frame motion detection for each divided region of a color moving image, in which consecutive regions with little chromaticity change in a color image are a region dividing means for dividing the region into regions with motion and outputting region division information; a region extracting means for generating a region extraction image in which each region is extracted based on the region division information; and a region extraction means for generating a region extraction image by extracting each region based on the region division information; The apparatus is characterized by comprising a motion vector detection means for detecting an inter-frame motion vector of a region and outputting the motion vector.

A fifth moving image motion detection method according to the present invention is a motion detection method that performs interframe motion detection for each divided region of a color moving image, and includes a signal separation means for decomposing the color image into three components; representative color detection means for detecting a plurality of representative colors constituting the color image from the three components and outputting each representative color; and binarization for generating a binarized image from the color image for each representative color. and a motion vector detection means for detecting a motion vector between frames for each binarized image.

In the first moving image motion detection method according to the present invention, an input image is first divided into a plurality of independent regions, and motion detection is performed on the assumption that each of the divided regions has motion. Therefore, the principle of this motion detection method will be explained using FIG. 6. In general, the input image is composed of multiple regions, as shown in Fig. 6 (
The area can be divided as shown in a). This kind of region division involves converting the input image into a multivalued image with a limited signal level,
This is achieved by etch detection, etc., which detects steep portions of signal level fluctuations. Next, motion detection is performed for each divided region, but here, we focus on region A, assuming that region A exhibits movement independent of other regions. Now, the 6th
If motion detection is performed by block matching on an image in which multiple regions coexist as shown in Figure (a), small blocks that straddle surrounding regions B, D, etc. at the boundary of region A will be affected by these motions. The false positive rate increases due to the influence of different surrounding areas. Therefore, a region A to be subjected to motion detection is extracted based on the region division results, and a region extracted image as shown in FIG. 6(b) is generated. This area extracted image is area A
This is an image in which only O is left, and all other parts are replaced with a constant value such as O. Motion detection for area A is performed on this area extracted image. By doing this, all the signals of the surrounding area are discarded, so that more accurate motion detection can be performed at the boundary of area A without being affected by the surrounding area.

In the second moving image motion detection method according to the present invention, a plurality of corners are detected as feature points of an extraction region, and the motion of the corresponding extraction region is determined from the interframe corresponding positions of these corners. Therefore, the principle of this motion detection method will be explained using FIG. 7. Assume now that region A has been extracted from the input image, as shown in FIG. 7(a). Region A is closed by connecting a plurality of continuous and smooth etch point arrays, and the connecting portions of individual etch point arrays are called corners. Note that the corner is detected by tracing a series of etch points, as shown in FIG. 7(b). That is, when a continuous etch point sequence is traced, a point that deviates from a straight line or a sharp curve is detected as a corner. Next, as shown in Figure 7(a), the detected corners are defined as feature points of area A, and the movement of each corner is detected by finding the corresponding position of each corner on the previous frame. be done. As a corner motion detection method, when the corner position in the current frame is (xc, yc), the corners existing in the search area (xc±sx, yc±sy) on the previous frame are matched between frames. It can be a corner. Here, sx and sy are values given in advance to determine the width of the motion detection area. The motion of the entire corresponding area is defined from the motion vector detected for each corner as described above.

For example, if the movement between frames of area A is parallel movement, the motion vectors of multiple corners belonging to area A all exhibit the same value. Therefore, the motion of all pixels within region A is given by a single motion vector. Furthermore, when area A is rotating, the motion vectors at each corner have different values to represent the rotation of area A. At this time gt$, A
It is possible to approximately determine the internal motion by interpolation from the motion vectors of these multiple corners. As described above, motion detection is performed at the corner, which is a feature point of the region, and the motion of the entire region is determined based on this result, so that motion detection can be performed regardless of the state inside the region. Furthermore, since calculations for motion detection are performed only for corners, the amount of calculations for motion detection can be reduced.

The principle of the third moving image motion detection method according to the present invention is described in the ninth section.
This will be explained using figures. Now, multiple corners and a series of etch points connected to them are detected by edge/corner detection,
It is assumed that a patterned image is generated based on this result. FIG. 9 shows a block of N×N pixels [p
i j ]. Here, each pixel within the block takes a value of C+e, n, or m. C
is a value indicating a corner, and e is a value indicating a series of etch points connected to the corner t. Further, n is a value indicating the outside of the area, and m is a value indicating the inside of the egg area. Corner motion detection is performed by interframe matching using this cutout block [pij] as a template. That is, a block signal [qij] of N×N pixels is read out from near the corner detected in the previous frame, and [piJ and [qi
The matching with J-co is highly evaluated. Evaluation value is <1
) can be used.

J (k is a natural number such as 1.2) Calculation of D I4 is performed by variously changing the cutting position of the block [qiJ] and the corner on the target previous frame, and the block [qfJ ] is determined as the position of the corresponding corner on the previous frame. Note that in the matching evaluation method using equation (1), different effects can be expected depending on how the intra-block pixel values Ce, n, and m are given.

First, as a first example, if C≠0 and e''=m=n≠C, then motion detection is performed only by the correspondence of corner points. Motion detection can be performed.

Next, as a second example, C≠O, e≠O, c≠e and m=
If n≠C, motion detection takes into account the position of the corner point and the shape of the etch near the corner. Incidentally, even when the movement between frames is enlargement/reduction or rotational movement at a small angle, the shape of the etch near the corner remains unchanged. Therefore, motion detection can be performed even for the above-mentioned motions. Furthermore, if m#n, then fil in the area
The motion detection also takes into account information indicating l/outside. As explained above, by using template matching, it is possible to obtain interframe motion vectors at corners that characterize the area.

In the fourth moving image motion detection method according to the present invention, by dividing a color human-powered image into regions based on chromaticity signals, it is possible to improve the precision of motion detection for each divided region. When the input image is an image of an object with three-dimensional unevenness, the brightness distribution of this object is likely to change greatly depending on the direction of illumination and the movement of the object itself. Therefore, a region segmentation method that takes advantage of changes in brightness level will result in many segmentation errors.

However, if it can be assumed that the original color of an object is uniform, changes in chromaticity on a two-dimensional image are smaller than changes in brightness. Therefore, by dividing an area with little chromaticity change into one continuous area, more accurate area division can be expected. Specific methods for such area division include a method in which the etched area where the chromaticity changes rapidly is used as the boundary of the area, and a limited color expression in which the entire image is expressed using only multiple representative colors detected from the original image. You may also use this method.

In the fifth moving image motion detection method according to the present invention, first, a plurality of representative colors are detected from an input image. The value of each pixel of the color human image is separated into three RGB components and coarsely quantized, and a combination value P of the three RGB components of each pixel is determined.

Next, a histogram of each pixel value P is created for the entire image, and a plurality of frequently occurring values are selected as representative colors. In addition to RGB, the pixel value P can be calculated using the luminance value Y and 2.
A combination of three components with two color difference values CI and C2 may be used, or other chromaticity expression methods may be used.

Next, binarized images are generated from the person and force images for each of the detected representative colors, and the principle of this will be explained using FIG. 10. In Figure 10, 1 is used to simplify the explanation.
Figure 3 shows a histogram of dimensional pixel values. Now, the values PL, P2., which occur frequently on this histogram. Suppose that P3 is selected as the representative color, the histogram is created based on each representative color at the threshold value T1. Regions R1, R2, R at T2
It is divided into 3 parts. These threshold values TI and T2 are representative color values P
1□P2. It is set as the center value of each P3. Furthermore, the following method may be used as another method for region division and representative color selection. In this method, the histogram is formed in regions R1, R2, R3 using appropriately determined thresholds T1 and T2.
and calculate the variance of pixel values within each region. This operation is performed for various threshold values, and a region division that minimizes the sum of the variances of each region is determined. Next, representative colors Pi and P2P3 are determined from the average value of pixel values belonging to each area.

Although the explanation has been made using a one-dimensional histogram in FIG. 10, selection of a representative color can also be achieved in a similar manner using a histogram of pixel values having three components such as RGB. The pixel values of each region R1, R2°R3 of the histogram divided by the method described above are determined by the representative color PIP2 of each region. Replaced by P3. By doing this, the input image becomes a limited □ color image expressed only by a plurality of representative colors. A binarized image is generated by extracting only each representative color portion from this limited color image. Motion detection is performed for each generated binarized image by template matching using feature points of the area extracted by binarization as a template.

As described above, by using the method of the present invention, it is possible to separate and detect the movement of regions expressed by different representative colors in a color image.

Therefore, if there is a subject in a color image with independent movement expressed using a specific representative color, the movement of this subject can be detected without the influence of surrounding subjects and background.

(Embodiment) FIG. 1 is a block diagram showing an embodiment of the first moving image motion detection method according to the present invention. First, area division circuit 1
1 is supplied with an input image 101.

The region division circuit 11 divides the input image 101 into a plurality of regions and supplies region division information 102 to the region extraction circuit 12 . Next, the two-region extraction circuit 12 generates a region extraction image 103 from the input image 101 based on the region division information 102, and extracts the region extraction surface #AlO3 from the motion vector detection time #1.
13 and previous frame information memory 14. The motion vector detection circuit 13 detects a motion vector 105 of the corresponding extraction region from the region extraction image 103 in the current frame and the region extraction image 104 in the previous frame supplied from the previous frame information memory 14, and detects the motion vector 105 of the corresponding extraction region. 105
Output.

FIG. 2 is a block diagram showing an embodiment of a motion detection method for a second moving image according to the present invention. Area division circuit 2
5 area dividing circuit 21 to which input image f@101 is supplied to 1
divides the input image 101 into a plurality of regions and supplies region division information 102 to the region extraction circuit 22 . Next, area extraction port F! @22 generates a region extraction image 103 from the input image 101 based on the region division information 102, and supplies the region extraction image 103 to the corner detection circuit 23. The corner detection circuit 23 detects corner information 201 for each extraction region from the region extraction image 103, and
is supplied to the motion vector detection circuit 24 and the previous frame information memory 24. The motion vector detection circuit 24 stores corner information 201 in the current frame and previous frame information memory 2.
The motion of each corner is detected from the corner information 202 of the previous frame supplied from 5, and the motion of the corner is output as a motion vector 105 of the corresponding extraction region.

FIG. 8 is a block diagram showing an embodiment of a motion vector detection circuit using the motion detection method shown in FIG. It is divided into a number 802 and a corner number 803 within the area. These decomposed information 801, 802, 803 are supplied to the previous frame information memory 82 and stored therein. At the same time, the previous frame information memory 82 stores the area number 802 and corner number 803.
The position information 804 of the corresponding corner in the previous frame specified by is supplied to the subtractor 83. The subtractor 83 calculates the difference between the position information 801 in the current frame of the corresponding corner and the position information 804 in the previous frame, and outputs the difference as a motion vector 105 of the corresponding extraction area.

FIG. 3 is a block diagram showing an embodiment of the third moving image motion detection method according to the present invention.9 First, an input image 101 is supplied to the IFt area dividing circuit 31. As shown in FIG.

The region division circuit 31 divides the input image 101 into a plurality of regions and supplies region division information 102 to the region extraction circuit 32 . Next, the region extraction circuit 32 generates a region extraction image 103 from the input image 101 based on the region division information 102,
The region extraction surface 1103 is supplied to the corner detection circuit 33. The corner detection circuit 33 detects corner information 201 for each extraction region from the region extraction image 103 and supplies the corner information 201 to the patterning circuit 34 . The patterning circuit 34 generates a patterned image 301 consisting of patterns indicating the etch corners of the extraction region and the inside and outside of the region from the corner information 201, and transfers the patterned image 301 to the motion vector detection circuit 35 and the previous frame. Information memory 36
supply to. The motion vector detection circuit 35 cuts out a region near a corner to be subjected to motion detection from the patterned image 301 in the current frame, and uses the cut out region near the corner as a template for the corner. The motion vector detection circuit 35 performs template matching between this template and the patterned image 302 of the previous frame supplied from the previous frame information memory 36, and the movement of the corner is detected. The detection result is output as a motion vector 105 of the corresponding extraction area.

FIG. 4 is a block diagram showing an embodiment of the fourth moving image motion detection method according to the present invention. First, color human power image 1
01 is supplied to the signal separation circuit 41, and the chromaticity signal 401 in the input image is supplied to the area division circuit 42. The area dividing circuit 42 divides the image into areas using the chromaticity signal 401. That is, the input image is divided into a plurality of regions by determining that consecutive image parts with small chromaticity changes belong to the same region, and the region division information 102 is transmitted to the region extraction circuit 4.
3. The region extraction circuit 43 extracts region division information 1
A region extraction plane @103 is generated from the input image 1101 based on 02, and the region extraction plane 1103 is supplied to the motion vector detection circuit 44 and the previous frame information memory 45. The motion vector detection circuit 44 extracts area extraction image 1 in the current frame.
03 and the region extraction image @104 in the previous frame supplied from the previous frame information memory 45, a motion vector 105 of the corresponding extraction region is detected, and the motion vector 105 is output.

FIG. 5 is a block diagram showing an embodiment of the fifth moving image motion detection method according to the present invention. First, the color human image 101 is decomposed into three signals of 5IS2 and S3 by the signal separation circuit 51, and each signal of 31 and 52S3 is supplied to the representative color detection circuit 52.

Gokode, 31. As three signals S2 and S3 (R, G
, B) or (Y, I, Q) or (YR, -YB-Y)
Components such as the following can be used.

Next, the representative color detection circuit 52 receives the supplied Sl, S2, and
A plurality of representative colors representing one input image 101 are detected from the three signals, and the representative colors are supplied as representative color information 501 to the binarization circuit 53. The binarization circuit 53 converts the color manual image 10
Generates a binary image (@502 from 1. Binarized pixel 50
2 is generated for each representative color supplied from the representative color detection circuit 52, and each binarized image 502 is supplied to the motion vector detection circuit 54 and the previous frame information memory 55.

The motion vector detection circuit 54 detects the motion vector 105 of the corresponding extraction area from the binarized image @50 of the current frame and the binarized image 503 of the previous frame supplied from the previous frame information memory 55. A motion vector 105 is output.

(Effects of the Invention) As explained above, according to the present invention, motion detection is performed on an image in which only the region to be detected by VJ is extracted, so motion detection is performed while eliminating the influence of other surrounding regions. It can be performed. Further, since the motion of the corresponding region is detected by detecting a plurality of corners as feature points for each region and determining the interframe correspondence of these corners, the amount of calculation for motion detection can be reduced. moreover,
Since the shape of the corner remains unchanged even when the image is enlarged/reduced or rotated by a small angle, more accurate motion detection can be performed even for these movements.

In motion detection of a color image, since region division is performed based on the chromaticity signal, more accurate region division can be achieved, and motion detection can be performed while eliminating the influence of the surrounding pond region.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of the first moving picture motion detection method according to the present invention, and FIG. 2 is a block diagram showing an embodiment of the second moving picture motion detection method according to the present invention. 3 is a block diagram showing an embodiment of the third moving picture motion detection method according to the present invention, and FIG. 4 is a block diagram showing an embodiment of the fourth moving picture motion detection method according to the present invention. Block diagram shown, No. 5
The figure is a block diagram showing an embodiment of the fifth moving image motion detection method according to the present invention, and FIGS. 6(a) and 6(b) illustrate the first moving image motion detection method according to the present invention. diagram for,
FIG. 7 is a diagram for explaining the second moving image motion detection method according to the present invention, and FIG. 8 is an example of corner motion detection in the second moving image motion detection method according to the present invention. 9 is a diagram for explaining the third moving image motion detection method according to the present invention, and FIG. 10 is a diagram for explaining the binarization in the fifth moving image motion detection method according to the present invention. FIG. 3 is a diagram for explaining an image generation method. 11.21, 31.42... area division circuit, 1222
．． 32.43...Area extraction circuit, 13.2435.4
4.54...Motion vector detection circuit, 14°25.3
6.45, 55.82...Previous frame information memory, 2
3.33... Corner detection circuit, 34... Patterning circuit, 41.51... Signal separation circuit, 52... Representative color detection circuit, 53... Binarization circuit, 81... Corner information separation circuit, 83...Differentiator, 101...Input image, 102...Region division information, 103, 104...
・Region extraction image, 105...Motion vector, 201・
... corner information, 301, 302 ... patterned image, 401 ... chromaticity signal, 501 ... representative color information, 5
02°503...Binarized image, 801,804...
Corner position information, 802...area number, 803...
...Corner number one. (a) Input the key to the left. ! i! Hidden stress of J# value 4 Figure 1o

Claims (5)

[Claims] (1) In a method for detecting motion between frames of a moving image, a region dividing means divides the image into a plurality of regions and outputs region division information, and region extraction extracts each region based on the region division information. A motion detection method for a moving image, comprising a region extracting means for generating an image, and a motion vector detecting means for detecting an inter-frame motion vector of a corresponding extraction region from the region extracted image and outputting the motion vector. . (2) In a method for detecting motion between frames of a moving image, a region dividing means divides the image into a plurality of regions and outputs region division information, and region extraction extracts each region based on the region division information. A region extracting means for generating an image, a corner detecting means for detecting a corner of a region outline that is a feature point of the extracted region based on the region extracted image, and inter-frame corresponding positions of corners detected for each extraction region. 1. A motion detection method for a moving image, comprising: a motion vector detection means for detecting a corner motion from a region and outputting the corner motion as a motion vector of a corresponding area. (3) In a method for detecting motion between frames of a moving image, a region dividing means divides the image into a plurality of regions and outputs region division information, and region extraction extracts each region based on the region division information. A region extracting means for generating an image, a corner detecting means for detecting a corner of a region outline that is a feature point of the extracted region based on the region extracted image, and a sequence of edge points connected to the detected corner. a patterning unit that generates a patterned image; a patterning unit that extracts a region near the corner from the patterned image as a template, and detects movement between frames of the corner by performing weighted interframe matching using the template; A motion detection method for a moving image, comprising: motion vector detection means for outputting the motion of the corner as a motion vector of the corresponding extraction area. (4) In a method for detecting motion between frames of a moving image, an area dividing means divides continuous areas with little chromaticity change in a color image as areas having the same movement, and outputs area division information; A region extracting means for generating a region extraction image by extracting each region based on the region division information, and a motion vector detection means for detecting an inter-frame motion vector of a corresponding extraction region from the region extraction image and outputting the motion vector. A motion detection method for moving images, comprising: (5) In a method for detecting motion between frames of a moving image, a signal separating means separates a color image into three components, detects a plurality of representative colors constituting the color image from the three components, and detects each of the plurality of representative colors constituting the color image from the three components. representative color detection means for outputting a representative color;
A moving image characterized by comprising: binarization means for generating a binarized image from the color image for each representative color; and motion vector detection means for detecting a motion vector between frames for each binarized image. motion detection method.