JP3856262B2 - Motion compensation encoding apparatus, motion compensation encoding method, and motion compensation code recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
In order to efficiently transmit, store, and display images, it is related to high-efficiency coding that converts image information into digital signals with a smaller code amount, and in particular performs motion compensation inter-picture prediction processing, where motion vector information is encoded. The transmission to the decoding side.
[0002]
[Prior art]
<Motion compensated image coding>
In high-efficiency coding of moving images, there is a method of performing prediction after moving each part of an image for each block in accordance with movement when performing inter-image prediction. Such spatial movement processing of an image is called motion compensation and is widely used in international standard systems such as MPEG. The motion compensation is generally performed in units of 16 × 16 pixels to 8 × 8 pixels, and the motion (motion vector) of the image is also determined in that unit.
As the accuracy of the motion vector (MV), one pixel or one-half pixel (0.5 pixel) is generally used. In the case of 0.5 pixel, a pixel corresponding to the middle of the original pixel is created by further interpolating the spatially moved pixel. When motion compensation is used in inter-picture prediction, it is necessary to perform motion compensation that is the same as that in encoding, so that MV is encoded and transmitted to the decoding side.
[0003]
<Conventional Encoding Device>
FIG. 7 shows a configuration example of an encoding apparatus that performs motion compensation as described above.
The image signal coming from the image input terminal 1 is subtracted from the inter-picture prediction signal given from the motion compensated predictor 72 in the prediction subtracter 2 and given to the DCT 3 as a prediction residual. The DCT 3 performs discrete cosine transform (DCT) conversion processing in units of 8 × 8 pixels, and gives the obtained coefficient to the quantizer 4.
The quantizer 4 quantizes the coefficient with a predetermined step width, and supplies the coefficient that has become a fixed-length code to the variable-length encoder 5 and the inverse quantizer 9. In general, the quantization step width is controlled by the amount of generated codes in order to keep the transfer rate constant.
The variable-length encoder 5 converts a two-dimensional 8 × 8 coefficient into a one-dimensional array in an order called a zigzag scan, and encodes the coefficient with a Huffman code as a continuous number of zeros and a coefficient value other than zero. To do. The inter-picture prediction residual signal thus converted into a code string is multiplexed with the code string of the motion vector (MV) by the multiplexer 10 and output from the code output terminal 11.
[0004]
On the other hand, the inverse quantizer 9 and the inverse DCT 16 perform the inverse processing of the DCT 3 and the quantizer 4 to reproduce the inter-picture prediction residual. A prediction signal is added to the obtained prediction residual by the adder 15 to form a reproduced image, which is stored in the image memory 14. A reproduced image is supplied from the image memory 14 to the motion estimator 71 and the motion compensated predictor 72.
The processing from the inverse quantizer 9 to the image memory 14 is called local decoding and is basically the same processing as the decoding device. The motion compensation predictor 72 moves the image stored in the image memory 14 for each block according to the motion vector given from the motion estimator 71, and obtains an inter-image prediction signal. The motion-compensated prediction signal is supplied to the subtracter 2 and the adder 15.
[0005]
The motion estimator 71 moves the reproduced image stored in the image memory 14 in units of motion compensation blocks to perform block matching with the input image, and sets MV as the movement with the best matching (small error). . The obtained MV is given not only to the motion compensated predictor 72 but also to the MV encoder 73 for encoding. The accuracy of MV is 0.5 pixel.
In the MV encoder 73, the value of the previous (normally left) block that has been encoded and the vector value of the block to be encoded are compared for each horizontal component and vertical component, and the difference value is encoded by the Huffman code. To do. The obtained MV code string is multiplexed by the multiplexer 10 with the code string of the inter-picture prediction residual.
[0006]
<Conventional Image Decoding Device>
A decoding apparatus corresponding to the motion compensation encoding apparatus of FIG. 7 will be described below.
FIG. 8 shows the configuration of the decoding apparatus.
The code coming from the code input terminal 21 is separated by the demultiplexer 22 into an inter-picture prediction residual code string and a motion vector (MV) code string. The code sequence of the inter-picture prediction residual is returned to a fixed length code by the variable length decoder 23, and the obtained 8 × 8 coefficient is given to the inverse quantizer 9. A reproduction prediction residual is obtained by the inverse quantizer 9 and the inverse DCT 16, and a prediction signal is added by the adder 15 to obtain a reproduction image.
[0007]
The reproduced image thus obtained is output from the image output terminal 24 and is given to the image memory 14. The motion compensation predictor 72 performs motion compensation on the image stored in the image memory 14 in accordance with the MV supplied from the MV decoder 81, and provides the obtained inter-image prediction signal to the adder 15. Here, the operations of the adder 15, the inverse quantizer 9, and the inverse DCT 16 are the same as those of the encoding apparatus of FIG.
On the other hand, the MV code string separated by the demultiplexer 22 is subjected to the inverse process of the variable length encoder 73 of FIG. 7 by the MV decoder 81, and the obtained MV information is sent to the motion compensated predictor 72. Given.
[0008]
[Problems to be solved by the invention]
In the conventional motion compensation encoding apparatus, the accuracy of the motion vector (MV) is fixedly set regardless of the content of the image. For this reason, an image with low autocorrelation (many high frequency components) has an insufficient MV accuracy and a prediction error occurs, and an image with high autocorrelation (high frequency components decrease) due to fast movement or the like is high. The accuracy of MV is not necessary, and MV information is wasted. Such a problem becomes more prominent as the motion compensation block is reduced and the information amount of MV is increased.
The present invention has been made paying attention to the above points. The MV is a unit obtained by bundling a plurality of MVs by using the degree of change in MV temporarily obtained with a predetermined accuracy and the degree of high frequency components of the image as a criterion. An object of the present invention is to provide a motion compensation coding apparatus, a motion compensation coding method, and a motion compensation code recording medium capable of reducing a code amount by performing motion compensation with appropriate MV precision by selecting precision. And
[0009]
[Means for Solving the Problems]
Accordingly, the present invention provides the following devices and methods in order to solve the above Symbol challenges.
(1) In a motion compensation coding apparatus for moving pictures that performs coding by performing motion compensation inter-picture prediction,
Provisional motion estimation means for obtaining a provisional motion vector with a predetermined accuracy for each block of motion compensation from an input image;
Motion vector activity detection means for obtaining a degree of change of the temporary motion vector in an integrated block unit in which a plurality of the blocks are integrated;
High-frequency component detection means for obtaining the degree of spatially high frequency components of the input image in units of the integrated blocks;
From the degree of change of the provisional motion vector and the degree of the high frequency component, the degree of change of the provisional motion vector is low, and when the degree of the high frequency component is large, the accuracy of the motion vector is high. On the other hand, the motion vector accuracy for selecting the accuracy of the motion vector in units of the integrated blocks so that the accuracy of the motion vector is low when the degree of change of the temporary motion vector is high and the degree of the high frequency component is small. A selection means;
The present motion estimation means for obtaining a motion vector used in actual motion compensation with the selected motion vector accuracy;
Means for forming a motion compensated inter-picture prediction signal using the motion vector;
A motion compensation coding apparatus comprising: means for coding the motion vector.
(2) In the motion compensation encoding apparatus described in (1) above,
The motion vector activity detection means includes
Difference detection means for obtaining a spatial difference of the provisional motion vector;
Absolute value means for obtaining an absolute value of the difference by supplying an output of the difference detection means;
A motion compensation coding apparatus comprising: cumulative addition means for supplying an output of the absolute value converting means and adding the absolute values in units of integrated blocks to obtain an average value.
(3) In a motion compensation coding method for a moving image in which motion compensation inter prediction is performed for encoding,
A temporary motion vector with a predetermined accuracy is obtained for each motion compensation block from the input image,
The degree of change in the temporary motion vector is determined in units of integrated blocks in which a plurality of the blocks are integrated,
The degree of spatially high frequency components of the input image is determined in units of the integrated block,
From the degree of change of the provisional motion vector and the degree of the high frequency component, the degree of change of the provisional motion vector is low, and when the degree of the high frequency component is large, the accuracy of the motion vector is high. Conversely, when the degree of change of the temporary motion vector is high and the degree of the high frequency component is small, the accuracy of the motion vector is selected in units of the integrated blocks so that the accuracy of the motion vector is low,
A motion vector used in actual motion compensation with the accuracy is obtained,
Using the motion vector to form a motion compensated inter-picture prediction signal;
A motion compensation encoding method, wherein the motion vector information is encoded.
[0010]
(Work)
In the present invention, the degree of change in the motion vector (MV) obtained and the degree of the high frequency component of the image are used as criteria. However, the degree of change in the MV is closely related to the amount of motion vector information. The degree of the high frequency component is closely related to the accuracy of the motion vector required for the motion compensated inter-picture prediction. Based on these, by selecting the MV accuracy in a unit in which a plurality of MVs are bundled, the accuracy of the MV that reduces the total code amount is selected based on the relationship between the code amount of the inter-picture prediction residual and the MV code amount. .
Information on motion compensation accuracy is added, but it is negligible because the information amount is about 1 to 2 bits in tens of blocks.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
<Example of Motion Compensation Encoding Device>
An embodiment of the motion compensation coding apparatus of the present invention will be described below with reference to FIGS. 1 and 2 to 6. FIG. 1 shows the configuration, and the same reference numerals are given to the same components as those of the conventional example of FIG. 1 includes a temporary motion estimator 6, a high frequency detector 7, an activity detector 12, and an MV accuracy selector 13 as compared with FIG. 7. In addition, the operations of the motion estimator 17, the motion compensation predictor 8, and the variable length encoder 18 are different from the conventional example.
In the embodiment, the difference from the conventional example is a motion compensated inter-picture prediction process, and the encoding process and the local decoding process of the prediction residual are the same. Accordingly, in FIG. 1, the operations of the image input terminal 1, the subtractor 2, the DCT 3, the quantizer 4, the variable length encoder 5, the inverse quantizer 9, the adder 15, the inverse DCT 16, and the code output terminal 11 are conventional examples. Is the same.
[0012]
The following describes motion estimation, motion compensation, and MV information encoding, which are features of the present invention.
An input signal (input image) coming through the image input terminal 1 is supplied to the temporary motion estimator 6 and the high frequency detector 7 in addition to the subtracter 2.
In the temporary motion estimator 6, a temporary MV with one pixel accuracy is obtained from the input signal in units of motion compensation blocks similar to the conventional example, and is supplied to the activity detector 12.
In the activity detector 12, the MV activity is obtained in units of integrated blocks in which 64 motion compensation blocks as shown in FIG. 6 are bundled, and is given to the MV accuracy determiner 13.
[0013]
The configuration of an embodiment of a specific processing circuit of the MV activity detector 12 is shown in FIG.
The activity detection is performed for each of the horizontal component and the vertical component of the MV value.
The MV value is differenced by the difference detector 31, and the difference value is given to the absolute value calculator 32. In accordance with the processing of the MV encoder 18, if the difference is the difference with the horizontal adjacent block, the difference is also taken here with the horizontal adjacent block. The absolute value calculator 32 converts the difference value into an absolute value and gives it to the cumulative adder 33. The cumulative adder 33 adds 64 absolute block values for the integrated block, and further adds a horizontal component and a vertical component to obtain an average value. When there are two MVs such as bi-directional prediction, both MVs are added separately and the one with the smaller value is output.
[0014]
On the other hand, the high frequency detector 7 detects how much high frequency components are in units of integrated blocks from the input signal and gives the result to the MV accuracy determiner 13.
The configuration of an embodiment of the internal processing circuit of the high frequency detector 7 is shown in FIG. The input image is a large block unit of 64 × 64 pixels, and a high-pass filter (HPF) is applied to each pixel.
In the space HPF 41, low frequency components including DC components are suppressed by two-dimensional calculation as shown in the figure or vertical and horizontal cascade processing. The output signal of the spatial HPF 41 in which the low frequency is suppressed is converted to an absolute value by the absolute value converter 42 and is supplied to the cumulative adder 43.
The cumulative adder 43 adds the absolute values for 4096 pixels of the integrated block to obtain an average value.
[0015]
The MV accuracy determiner 13 determines whether the MV accuracy is 1 pixel, 0.5 pixels, or two values obtained from the output of the high frequency detector 7 and the output of the activity detector 12, or , 0.25 pixel is determined for each integrated block in light of the criteria of the graph of the MV accuracy judgment table as shown in FIG. The determined MV accuracy (1, 0.5, or 0.25 pixel accuracy) is provided to the motion estimator 17. The MV accuracy determiner 13 determines the accuracy of the MV based on a two-dimensional threshold that is statistically obtained in advance from the binary values input for each large block and set based on the generated code amount. Specifically, using the graph as shown in FIG. 5 as an MV accuracy determination table, when the MV activity is low and the high frequency (high frequency component) is large, it is determined as 0.25 pixel accuracy, and conversely, the MV activity is If it is high and the high frequency is small, it is determined that the accuracy is one pixel. In the case of an intermediate level between the two, 0.5 pixel accuracy is set.
[0016]
The motion estimator 17 obtains the MV from the given input image and the locally decoded image with the given MV accuracy. The MV information obtained by the motion estimator 17 is provided to the motion compensated predictor 8 and the MV encoder 18. The MV information includes pixel accuracy information.
Although the MV may be newly obtained by the motion estimator 17, the temporary MV obtained by the temporary motion estimator 6 may be used. The method with the smallest processing amount is a method in which the temporary MV is used as it is when the MV accuracy is one pixel. If the accuracy is 0.5 pixel, the temporary MV with 1 pixel accuracy is used as a reference and the surrounding MV is searched again with 0.5 pixel accuracy, and the obtained 0.5 pixel accuracy MV is set as the final MV. When the accuracy of the MV is 0.25 pixel, after obtaining the MV of 0.5 pixel by the above method, the search is performed again with the accuracy of 0.25 based on the MV, and the obtained 0.25 pixel accuracy MV is finally obtained. MV.
[0017]
The motion compensated predictor 8 spatially moves the reproduced image held in the image memory 14 according to the given MV, and provides it to the subtracter 2 and the adder 15 as an inter-picture prediction signal. Since the accuracy of MV changes, in the case of one pixel, only a pixel-unit moving process is required, but in the case of 0.5 precision and 0.25 precision, resampling processing is required, and the precision differs.
[0018]
The basic processing operation of the MV encoder 18 is the same as that of the conventional example, but it is reasonable to normalize the values of each MV with the accuracy of the MV and separately attach the information of the accuracy of the MV. In other words, a variable length code table for the difference is prepared for one type for integer values, and in the case of 1 pixel accuracy, the value is doubled in the case of 0.5 pixel accuracy, and in the case of 0.25 pixel accuracy. The value is multiplied by 4 and a common code table is used as all integers. And the code | symbol of the precision of MV is attached in an integrated block unit.
[0019]
The information on the accuracy of MV is 1 to 2 bits for 64 × 64 pixels of the integrated block, and the code amount is negligible.
Although there is no particular description in the configuration in FIG. 1, it takes a processing time for at least one integrated block to determine the accuracy of the MV. Therefore, it is necessary to delay the processing after the motion estimator 17 by that time.
In addition, an image signal of a normal raster scan is converted in order of blocks and integrated blocks in accordance with processing.
[0020]
<Example of Motion Compensation Code Recording Medium>
The code string output from the motion compensation coding apparatus of the present invention shown in FIG. 1 is recorded on the motion compensation code recording medium.
In this case, the motion compensation code recording medium selects the accuracy of the MV for each block in which a plurality of motion compensation blocks are integrated from the input image, and obtains the MV to be used in the actual motion compensation with the accuracy, thereby inter-picture prediction code. Obtained by multiplexing the prediction residual code string obtained by performing the encoding and the MV code string obtained by encoding the MV information in a form having the MV accuracy information in integrated block units. The code string is recorded.
This code string has a smaller code amount per unit time than a code string output from a conventional encoding apparatus, and can record information for a longer time on a recording medium of the same capacity. .
[0021]
<Example of Motion Compensated Decoding Device>
An embodiment of a decoding apparatus corresponding to the motion compensation encoding apparatus of the present invention will be described below with reference to the drawings.
FIG. 2 shows the configuration of the decoding apparatus, and the same reference numerals are given to the same components as in the conventional example. 2 differs from FIG. 8 in the operations of the MV decoder 25 and the motion compensated predictor 8. In the embodiment, the motion-compensated inter-picture prediction process is different from the conventional example. The code input terminal 21, the demultiplexer 22, the variable length decoder 23, the inverse quantizer 9, the inverse DCT 16, the adder 15, the image The operations of the output terminal 24 and the image memory 14 are basically the same as in the conventional example.
[0022]
The MV information separated by the demultiplexer 22 shown in FIG. 2 is reversely processed by the MV decoder 25 in the MV encoder 18 of FIG. At this time, the information on the pixel accuracy of the inserted MV is also decoded for each integrated block.
The obtained integer value of the MV is multiplied by the accuracy value of the MV to reproduce the MV, and is given to the motion compensation predictor 8. The motion compensated predictor 8 forms a prediction signal similar to that in FIG. 1 based on the MV.
[0023]
【The invention's effect】
In the present invention, the degree of change in the temporary motion vector (MV) with a predetermined accuracy and the degree of the high frequency component of the image are obtained. The degree of change in the MV is closely related to the amount of motion vector information, and the high frequency of the image. The degree of the component is closely related to the accuracy of motion compensation required for motion compensated inter-picture prediction. Based on these, by selecting the MV accuracy in a unit of a plurality of MVs, the accuracy with which the total code amount decreases most reasonably is selected based on the relationship between the code amount of the inter-picture prediction residual and the MV code amount. Is done.
[0024]
Therefore, the code amount is reduced in the part of the image where the accuracy of the conventional motion compensation is insufficient and the prediction residual code amount is large, or the MV accuracy is unnecessarily fine and the MV code amount is large, and the total code amount is reduced. Hold down low. On the other hand, if the generated code amount (transfer rate) is constant, the quantization becomes fine by the code amount control, and the quality of the reproduced image is improved.
Since the temporary MV can be used as it is as the basis of the actual MV, the increase in the processing amount is small.
[0025]
Further, in the present invention, the code amount per unit time can be reduced compared to the code string output from the conventional encoding apparatus, and recording capable of recording information for a longer time on a recording medium of the same capacity is possible. Media can be provided.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration example of an embodiment of a motion compensation encoding apparatus according to the present invention.
FIG. 2 is a diagram illustrating a configuration example of an embodiment of a motion compensation decoding apparatus according to the present invention.
FIG. 3 is a diagram showing a processing configuration of an embodiment of an MV activity detector according to the present invention.
FIG. 4 is a diagram showing a processing configuration of an embodiment of the high frequency detector of the present invention.
FIG. 5 is a diagram showing an example of a threshold in MV accuracy determination according to the present invention.
FIG. 6 is a diagram illustrating a relationship between a motion compensation block and an integrated block according to an embodiment of the present invention.
FIG. 7 is a diagram illustrating an example of a configuration of a conventional motion compensation encoding apparatus.
FIG. 8 is a diagram illustrating an example of a configuration of a conventional motion compensation decoding apparatus.
[Explanation of symbols]
1 Image input terminal 2 Subtractor 3 DCT
4 Quantizer 5 Variable length encoder 6 Temporary motion estimator (provisional motion estimation means)
7 High frequency detector (high frequency component detection means)
8 motion compensated predictor (means for forming a motion compensated inter-picture prediction signal)
9 Inverse quantizer 10 Multiplexer 11 Code output terminal 12 MV activity detector (motion vector activity detector)
13 MV accuracy determiner (motion vector accuracy selection means)
14 Image memory 15 Adder 16 Inverse DCT
17 Motion estimator (main motion estimation means)
18 MV encoder (means for encoding motion vector)
21 Code input terminal 22 Demultiplexer 23 Variable length decoder 24 Image output terminal 25, 81 MV decoder 31 Difference detector 32, 42 Absolute value converter 33, 43 Cumulative adder 41 Spatial HPF
71 Motion Estimator 72 Motion Compensated Predictor 73 MV Encoder

Claims (3)

In a motion compensation coding apparatus for a motion picture that performs coding by performing motion compensation inter-picture prediction,
Provisional motion estimation means for obtaining a provisional motion vector with a predetermined accuracy for each block of motion compensation from an input image;
Motion vector activity detection means for obtaining a degree of change of the temporary motion vector in an integrated block unit in which a plurality of the blocks are integrated;
High-frequency component detection means for obtaining the degree of spatially high frequency components of the input image in units of the integrated blocks;
From the degree of change of the provisional motion vector and the degree of the high frequency component, the degree of change of the provisional motion vector is low, and when the degree of the high frequency component is large, the accuracy of the motion vector is high. On the other hand, the motion vector accuracy for selecting the accuracy of the motion vector in units of the integrated blocks so that the accuracy of the motion vector is low when the degree of change of the temporary motion vector is high and the degree of the high frequency component is small. A selection means;
The present motion estimation means for obtaining a motion vector used in actual motion compensation with the selected motion vector accuracy;
Means for forming a motion compensated inter-picture prediction signal using the motion vector;
A motion compensation coding apparatus comprising: means for coding the motion vector. The motion compensation encoding apparatus according to claim 1,
The motion vector activity detection means includes
Difference detection means for obtaining a spatial difference of the provisional motion vector;
Absolute value means for obtaining an absolute value of the difference by supplying an output of the difference detection means;
A motion compensation coding apparatus comprising: cumulative addition means for supplying an output of the absolute value converting means and adding the absolute values in units of integrated blocks to obtain an average value. In a motion compensation coding method of a moving image that performs coding by performing motion compensation inter-image prediction,
A temporary motion vector with a predetermined accuracy is obtained for each motion compensation block from the input image,
The degree of change in the temporary motion vector is determined in units of integrated blocks in which a plurality of the blocks are integrated,
The degree of spatially high frequency components of the input image is determined in units of the integrated block,
From the degree of change of the provisional motion vector and the degree of the high frequency component, the degree of change of the provisional motion vector is low, and when the degree of the high frequency component is large, the accuracy of the motion vector is high. Conversely, when the degree of change of the temporary motion vector is high and the degree of the high frequency component is small, the accuracy of the motion vector is selected in units of the integrated blocks so that the accuracy of the motion vector is low ,
A motion vector used in actual motion compensation with the accuracy is obtained,
Using the motion vector to form a motion compensated inter-picture prediction signal;
A motion compensation encoding method, wherein the motion vector information is encoded.

Images