JP2013077865A - Image encoding apparatus, image decoding apparatus, image encoding method and image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve encoding efficiency by generating a time direct vector close to a motion vector of a macro block subjected to encoding.SOLUTION: A time direct vector generator 13 identifies a macro block at the spatially same position as a macro block subjected to encoding from among macro blocks constituting an encoded picture temporally near the macro block subjected to encoding and generates a time direct vector of a time direct mode from motion vectors of four rectangular blocks positioned in the center of the identified macro block.

Description

  The present invention relates to an image encoding device, an image decoding device, an image encoding method, and an image decoding method used for image compression encoding technology, compressed image data transmission technology, and the like.

For example, in an international standard video encoding method such as MPEG (Moving Picture Experts Group) or “ITU-T H.26x”, a luminance signal 16 × 16 pixels and a color difference signal 8 × corresponding to the luminance signal 16 × 16 pixels are used. A method is adopted in which block data (hereinafter referred to as “macroblock”) including eight pixels is used as a unit for compression based on a motion compensation technique or an orthogonal transform / transform coefficient quantization technique.
In the motion compensation processing in the image encoding device and the image decoding device, a motion vector is detected and a predicted image is generated in units of macroblocks with reference to a front or rear picture.
At this time, a picture that performs inter-frame prediction encoding with reference to only one picture is referred to as a P picture, and a picture that performs inter-frame prediction encoding with reference to two pictures at the same time is referred to as a B picture. .

AVC / H. Is an international standard system. H.264 (ISO / IEC 14496-10 | ITU-T H.264) can select a coding mode called a direct mode when coding a B picture (see, for example, Non-Patent Document 1). .
That is, the encoding target macroblock does not have motion vector encoded data, and the motion vector of another encoded macroblock or the motion vector of a spatially surrounding encoded macroblock A coding mode for generating a motion vector of a macroblock to be coded can be selected by a predetermined calculation process to be used.

There are two types of direct mode: temporal direct mode and spatial direct mode.
In the temporal direct mode, the motion vector scaling process is performed according to the time difference between the coded picture and the picture to be coded by referring to the motion vector of the other picture that has been coded. Generate a motion vector of.
In the spatial direct mode, the motion vector of at least one encoded macroblock located around the macroblock to be encoded is referenced, and the motion vector of the macroblock to be encoded is determined from those motion vectors. Generate.
In this direct mode, by using “direct_spatial_mv_pred_flag” which is a flag provided in the slice header, it is possible to select either the temporal direct mode or the spatial direct mode in units of slices.

Here, FIG. 12 is a schematic diagram showing a method of generating a motion vector in the temporal direct mode.
In FIG. 12, “P” represents a P picture, and “B” represents a B picture.
Numbers 0 to 3 indicate the display order of pictures and indicate that the images are displayed at times T0, T1, T2 and T3.
It is assumed that the picture encoding process is performed in the order of P0, P3, B1, and B2.

For example, it is assumed that the macroblock MB1 in the picture B2 is encoded in the temporal direct mode.
In this case, the motion of the macroblock MB2 which is the motion vector of the picture P3 closest to the picture B2 among the encoded pictures located on the rear side of the picture B2 on the time axis, and is in the same spatial position as the macroblock MB1. Vector MV is used.
The motion vector MV refers to the picture P0, and the motion vectors MVL0 and MVL1 used when encoding the macroblock MB1 are obtained by the following equation (1).

  However, when the macroblock MB2 in the picture P3 is divided into two or more rectangular blocks, the motion vector of the rectangular block including the upper left pixel of the macroblock MB2 is used as the motion vector MV as shown in FIG. However, AVC / H. H.264.

MPEG-4 AVC (ISO / IEC 14496-10) / ITU-TH H.264 standard

  Since the conventional image coding apparatus is configured as described above, the motion vector of the rectangular block including the upper left pixel of the macro block MB2 is always used as the motion vector MV. For this reason, when the macro block MB2 in the picture P3 is divided into two or more rectangular blocks, the motion vector MV to be used does not necessarily indicate the motion at the same pixel position as the macro block MB1 to be encoded. However, there is a problem that the accuracy of the temporal direct vector is lowered and the coding efficiency may be deteriorated.

The present invention has been made in order to solve the above-described problems. An image encoding apparatus and an image which can generate a time direct vector close to the motion of a macroblock to be encoded and can increase encoding efficiency. The object is to obtain an encoding method.
Another object of the present invention is to obtain an image decoding apparatus and an image decoding method capable of decoding an encoded stream with high encoding efficiency by generating a temporal direct vector by means similar to the encoding apparatus. To do.

  In the image encoding device according to the present invention, the direct vector generation means spatially links the encoding target block from among the blocks constituting the encoded picture that is temporally adjacent to the encoding target block. The block at the same position is specified, and the time direct vector of the time direct mode is generated from the motion vectors of a plurality of divided regions located at the center of the block.

  According to the present invention, the direct vector generation means is located in the same spatial position as the block to be encoded among the blocks constituting the encoded picture that is temporally adjacent to the block to be encoded. Since a certain block is specified and the time direct vector of the time direct mode is generated from the motion vectors of multiple divided regions located at the center of the block, the time direct that is close to the motion of the block to be encoded It becomes possible to generate a temporal direct vector of the mode, and as a result, there is an effect that the encoding efficiency of the block to be encoded can be increased.

It is a block diagram which shows the image coding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the motion compensation prediction part 1 of the image coding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the image decoding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the motion compensation estimation part 23 of the image decoding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content of the image coding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content of the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the production _| generation method of the time direct vector MV _direct of time direct mode. It is explanatory drawing which shows the production _| generation method of the time direct vector _MVdirect by the weighting addition according to block size. It is explanatory drawing which shows the production _| generation method of the time direct vector _MVdirect by the weighting addition according to a difference motion vector. It is explanatory drawing which shows the method of producing _| generating the time direct vector _MVdirect using the motion vector of all the blocks in a macroblock. It is explanatory drawing which shows the prediction process of the motion vector of a non-rectangular block. It is a schematic diagram which shows the method of producing | generating a motion vector in time direct mode. It is explanatory drawing which shows the motion vector MV used when macroblock MB2 in the picture P3 is divided | segmented into two or more rectangular blocks.

Embodiment 1 FIG.
In the first embodiment, an image encoding device that performs encoding that is closed in a frame in units obtained by equally dividing a video frame constituting an input image into rectangular regions (macroblocks) of 16 × 16 pixels. An image decoding device corresponding to the image encoding device will be described.
In the first embodiment, the image encoding device and the image decoding device are both AVC / H. An example in which the encoding method employed in the H.264 standard is used will be described.
However, in the first embodiment, when the image encoding device and the image decoding device use the direct mode, the temporal direct mode is used. However, the spatial direct mode is used if necessary. Also good.

FIG. 1 is a block diagram showing an image coding apparatus according to Embodiment 1 of the present invention.
In FIG. 1, the motion compensation prediction unit 1 selects a reference image of one frame from the reference images for motion compensation prediction stored in the frame memory 8 for one or more frames, and constitutes an input image. (Or a sub-macroblock obtained by dividing a macroblock), a motion compensation prediction process is executed, and a motion vector of the macroblock (macroblock to be encoded) is generated to generate a prediction image, A process of outputting the identification number, motion vector, predicted image, and the like of the reference image selected for each block is performed.
However, here, for convenience of explanation, it is assumed that a predicted image is generated by generating a motion vector in units of macroblocks.

  That is, the motion compensation prediction unit 1 stores a motion vector memory 11 (see FIG. 2) that stores a motion vector of a macro block (or a sub macro block obtained by dividing the macro block) that constitutes an encoded picture. In the case of temporal direct mode, for each macroblock constituting the input image, the macro is selected from among the macroblocks constituting an encoded picture that is temporally close to the macroblock. A macroblock located in the same spatial position as the block is identified, and motion vectors (motion vectors stored in the motion vector memory 11) of, for example, four blocks (divided areas) located at the center of the macroblock Generate a time direct vector for time direct mode from the time direct vector By carrying out can compensation prediction process, we carry out a process of generating the predicted image.

The subtracter 2 calculates a difference image between the prediction image generated by the motion compensation prediction unit 1 and the input image, and outputs a prediction difference signal indicating the difference image to the encoding mode determination unit 3.
The encoding mode determination unit 3 evaluates the prediction efficiency of the prediction difference signal output from the subtracter 2, and among the at least one or more prediction difference signals output from the subtracter 2, the prediction difference having the highest prediction efficiency. A motion vector, a macroblock type / sub-macroblock type (for example, a coding mode used in the macroblock) used to generate a predicted image related to the prediction difference signal in the motion compensation prediction unit 1 Is output to the variable-length encoding unit 9 as encoding mode information, and the prediction differential signal with the highest prediction efficiency is output. Is output to the compression unit 4.

The compression unit 4 performs DCT (discrete cosine transform) processing on the prediction difference signal output from the coding mode determination unit 3 to calculate a DCT coefficient, quantizes the DCT coefficient, and performs DCT after quantization. A process of outputting compressed data (quantization coefficient) as a coefficient to the local decoding unit 5 and the variable length coding unit 9 is performed.
Note that the subtracter 2, the encoding mode determination unit 3, and the compression unit 4 constitute quantization means.

The local decoding unit 5 performs inverse quantization on the compressed data output from the compression unit 4 to obtain DCT coefficients, and performs inverse DCT (Inverse Discrete Cosine Transform) processing on the DCT coefficients. Processing for calculating a prediction error signal corresponding to the output prediction difference signal is performed.
The adder 6 adds a prediction error signal calculated by the local decoding unit 5 and a prediction signal indicating the prediction image generated by the motion compensated prediction unit 1 to generate a local decoded image signal indicating a locally decoded image. To implement.

The loop filter 7 compensates for the coding distortion included in the locally decoded image signal output from the adder 6 and stores the locally decoded image indicated by the locally decoded image signal after the coding distortion compensation in the frame memory 8 as a reference image. Perform the output process.
The frame memory 8 is a recording medium such as a RAM for storing the reference image output from the loop filter 7.

The variable length encoding unit 9 receives the compressed data output from the compression unit 4 and the encoding mode information (macroblock type / sub macroblock type, motion vector, reference image identification number) output from the motion compensated prediction unit 1. Entropy encoding is performed, a bit stream (encoded data) indicating the encoding result is generated, and processing for outputting the bit stream is performed.
The variable length coding unit 9 constitutes variable length coding means.
However, the motion vector information may be encoded as it is. As in H.264 / AVC, a prediction vector may be generated using a motion vector of an encoded macroblock, and a difference from the prediction vector may be encoded.

When predicting a motion vector of a non-rectangular block, for example, a method as shown in FIG. 11 is conceivable. In FIG. 11, an arrow represents a peripheral motion vector used for deriving a prediction vector.
The prediction vector of the divided area indicated by the three motion vectors surrounded by circles is obtained by the median (median value) of the three motion vectors surrounded by circles.

FIG. 2 is a block diagram showing the motion compensated prediction unit 1 of the image coding apparatus according to Embodiment 1 of the present invention.
In FIG. 2, a motion vector memory 11 is a recording medium such as a RAM that stores motion vectors of macroblocks (or sub-macroblocks obtained by dividing the macroblock) that constitute an encoded picture.
When the motion vector search unit 12 receives information indicating that the coding mode is the inter mode (for example, receives information indicating that the inter mode is used from the outside), the motion vector search unit 12 searches for an optimal motion vector in the inter mode, Processing for outputting the motion vector to the motion compensation processing unit 14 and the motion vector memory 11 is performed.

When the time direct vector generation unit 13 receives information indicating that the coding mode is the time direct mode, the temporal direct vector generation unit 13 configures, for each macroblock to be encoded, a coded picture that is temporally adjacent to the macroblock. A macro block located in the same spatial position as the macro block is identified, and motion vectors (motion vectors) of, for example, four blocks (divided regions) located at the center of the macro block are identified. A time direct vector in the time direct mode is generated from the motion vector stored in the memory 11 and the time direct vector is used as a motion vector to output to the motion compensation processing unit 14 and the motion vector memory 11.
The time direct vector generation unit 13 constitutes a direct vector generation unit.

  The motion compensation processing unit 14 performs a motion compensation prediction process using the motion vector output from the motion vector search unit 12 or the temporal direct vector generation unit 13 and the reference image of one frame stored in the frame memory 8. Then, the process which produces | generates an estimated image is implemented. The motion compensation processing unit 14 constitutes a predicted image generation unit.

3 is a block diagram showing an image decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 3, a variable length decoding unit 21 receives a bit stream (encoded data) output from the image encoding apparatus of FIG. 1, and from this bit stream, compressed data (quantization coefficient) and encoding mode information (macro Block type / sub-macroblock type, motion vector, reference image identification number) are entropy decoded, the compressed data is output to the prediction error decoding unit 22, and the coding mode information is output to the motion compensation prediction unit 23. Perform the process. The variable length decoding unit 21 constitutes variable length decoding means.

  The prediction error decoding unit 22 inversely quantizes the compressed data output from the variable length decoding unit 21 to obtain a DCT coefficient, and performs an inverse DCT process on the DCT coefficient to thereby generate a prediction error signal indicating a difference image (FIG. 1). The prediction error signal corresponding to the prediction difference signal output from the encoding mode determination unit 3 is calculated. The prediction error decoding unit 22 constitutes an inverse quantization unit.

The motion compensation prediction unit 23 reads out a reference image indicated by the identification number output from the variable length decoding unit 21 from one or more reference images stored in the frame memory 26, and outputs the reference image from the variable length decoding unit 21. When the macro block type / sub macro block type indicates that the inter mode is used, the motion compensation prediction process is performed using the motion vector output from the variable length decoding unit 21 and the reference image. Thus, a process for generating a predicted image is performed.
On the other hand, when the macroblock type / sub-macroblock type output from the variable length decoding unit 21 indicates that the direct mode is used, the same as the motion compensation prediction unit 1 in the image encoding device of FIG. Then, a temporal direct vector is generated, and a motion compensation prediction process is performed using the temporal direct vector and the reference image, thereby executing a process for generating a predicted image.

The adder 24 adds the prediction image generated by the motion compensation prediction unit 23 and the difference image indicated by the prediction error signal output from the prediction error decoding unit 22, and outputs the result from the adder 6 of the image encoding device in FIG. A process of generating a decoded image signal indicating a decoded image corresponding to the locally decoded image that has been performed is performed.
The loop filter 25 compensates for the encoding distortion included in the decoded image signal generated by the adder 24, stores the decoded image indicated by the decoded image signal after the encoding distortion compensation in the frame memory 26 as a reference image. Then, a process of outputting the decoded image to the outside is performed.
The adder 24 and the loop filter 25 constitute image adding means.
The frame memory 26 is a recording medium such as a RAM that stores the reference image output from the loop filter 25.

FIG. 4 is a block diagram showing the motion compensation prediction unit 23 of the image decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 4, a motion vector memory 31 is a recording medium such as a RAM that stores motion vectors of macroblocks (or sub-macroblocks obtained by dividing the macroblock) constituting a decoded picture.

In the case where the macro block type / sub macro block type output from the variable length decoding unit 21 indicates that the direct mode is used, the temporal direct vector generation unit 32 indicates, for each macro block to be decoded, the macro block Among the macroblocks constituting the decoded picture that is near in time, a macroblock that is spatially located at the same position as the macroblock is identified, and is located at the center of the macroblock, for example, 4 A temporal direct vector in the temporal direct mode is generated from motion vectors (motion vectors stored in the motion vector memory 31) of the individual blocks (divided regions), and the motion compensation processing unit 33 and the temporal direct vector are used as motion vectors. A process of outputting to the motion vector memory 31 is performed.
The time direct vector generation unit 32 constitutes a direct vector generation unit.

  The motion compensation processing unit 33 reads the reference image indicated by the identification number output from the variable length decoding unit 21 from one or more reference images stored in the frame memory 26, and performs macroblock type / sub macroblock type. Indicates that the inter mode is used, a motion compensation prediction process is performed using the motion vector output from the variable length decoding unit 21 and the reference image, and a predicted image is generated. When the block type / sub-macro block type indicates that the direct mode is used, the motion compensation prediction process is performed using the motion vector output from the temporal direct vector generation unit 32 and the reference image. The process which produces | generates an estimated image is implemented. The motion compensation processing unit 33 constitutes a predicted image generation unit.

In FIG. 1, a motion compensation prediction unit 1, a subtracter 2, a coding mode determination unit 3, a compression unit 4, a local decoding unit 5, an adder 6, a loop filter 7 and a variable length code which are components of the image coding apparatus. It is assumed that each of the encoding units 9 is configured by dedicated hardware (for example, a semiconductor integrated circuit on which a CPU is mounted, or a one-chip microcomputer), but the image encoding device is a computer. When configured, the processing contents of the motion compensation prediction unit 1, the subtractor 2, the coding mode determination unit 3, the compression unit 4, the local decoding unit 5, the adder 6, the loop filter 7 and the variable length coding unit 9 are described. The stored program may be stored in the memory of the computer, and the CPU of the computer may execute the program stored in the memory.
FIG. 5 is a flowchart showing the processing contents of the image coding apparatus according to Embodiment 1 of the present invention.

In FIG. 3, each of the variable length decoding unit 21, the prediction error decoding unit 22, the motion compensation prediction unit 23, the adder 24, and the loop filter 25, which are components of the image decoding apparatus, is installed with dedicated hardware (for example, a CPU is installed). In the case where the image decoding device is configured by a computer, a variable length decoding unit 21, a prediction error decoding unit 22, A program describing the processing contents of the motion compensation prediction unit 23, the adder 24, and the loop filter 25 is stored in the memory of the computer, and the CPU of the computer executes the program stored in the memory. Also good.
FIG. 6 is a flowchart showing the processing contents of the image decoding apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
First, the processing contents of the image encoding device in FIG. 1 will be described.
When a moving image signal indicating an input image is input, the motion compensation prediction unit 1 divides each frame of the moving image signal into macro block units (or sub macro block units).
When the motion compensation prediction unit 1 divides the video signal into macroblock units (or sub-macroblock units), one frame from the reference images for motion compensation prediction stored in the frame memory 8 is stored in one frame. By selecting a reference image and executing motion compensation prediction processing for each color component in units of macroblocks (or submacroblocks), so that the motion vector of the macroblock (or submacroblock) to be encoded To generate a predicted image.

When the motion compensated prediction unit 1 generates a prediction image by generating a motion vector of a macroblock (or sub-macroblock) to be encoded, the motion compensation prediction unit 1 outputs the prediction image to the subtracter 2 and generates the prediction image. Indicates whether the coding mode used in the macroblock type / sub-macroblock type (for example, the macroblock (or sub-macroblock) is the inter mode or the direct mode). Information) and the identification number of the reference image are output to the encoding mode determination unit 3.
Hereinafter, the processing content of the motion compensation prediction part 1 is demonstrated concretely.
However, here, for convenience of explanation, it is assumed that a predicted image is generated by generating a motion vector in units of macroblocks.

When the motion vector search unit 12 of the motion compensation prediction unit 1 receives information indicating that the encoding mode is the inter mode (for example, receives information indicating that the inter mode is used from the outside) (step of FIG. 5). ST1) Search for an optimal motion vector in the inter mode, and output the motion vector to the motion compensation processing unit 14 (step ST2).
Since the process of searching for an optimal motion vector in the inter mode is a known technique, detailed description thereof is omitted.

When receiving the information indicating that the coding mode is the direct mode (step ST1), the temporal direct vector generation unit 13 of the motion compensated prediction unit 1 receives the macroblock temporally for each macroblock to be encoded. From the macroblocks constituting the encoded picture in the vicinity, the macroblock located at the same spatial position as the macroblock is identified (step ST3).
For example, when the macroblock to be encoded is the macroblock MB1 of the picture B2, as shown in FIG. 12, the macroblocks constituting the encoded picture P3 that is temporally adjacent to the macroblock MB1. The macro block MB2 located in the same spatial position as the macro block MB1 is identified from the inside.

When the temporal direct vector generation unit 13 identifies the macroblock MB2 that is in the same spatial position as the encoding target macroblock MB1, as shown in FIG. From the motion vectors, motion vectors MV _i (i = 1, 2, 3, 4) of four rectangular blocks (divided regions) located at the center of the macro block MB2 are acquired (step ST4).
In the example of FIG. 7, the motion vector of the upper left rectangular block is MV ₁ , the motion vector of the lower right rectangular block is MV _4, and numbers are assigned in the raster scan order.
When the temporal direct vector generation unit 13 acquires the motion vectors MV _i (i = 1, 2, 3, 4) of the four rectangular blocks, as shown in the following equation (2), the four motion vectors MV _By calculating the average of _i , the temporal direct vector MV _direct in the temporal direct mode in the macro block MB1 to be encoded is generated (step ST5).

When generating the time direct vector MV _direct in the time direct mode, the time direct vector generation unit 13 outputs the time direct vector MV _direct as a motion vector to the motion compensation processing unit 14 and the motion vector memory 11.

When the motion compensation processing unit 14 of the motion compensation prediction unit 1 receives a motion vector from the motion vector search unit 12 when the coding mode is the inter mode, the motion vector and one frame stored in the frame memory 8 are received. A predicted image is generated by performing motion compensation prediction processing using the reference image (step ST6).
On the other hand, when the encoding mode is the direct mode, when the temporal direct vector MV _direct is received from the temporal direct vector generation unit 13 as a motion vector, the temporal direct vector MV _direct and one frame stored in the frame memory 8 are referenced. By performing the motion compensation prediction process using the image, a predicted image is generated (step ST6).
Note that the motion compensation prediction process of the motion compensation processing unit 14 is a known technique, and thus detailed description thereof is omitted.

When the motion compensated prediction unit 1 generates a prediction image, the subtracter 2 calculates a difference image between the prediction image and the input image, and outputs a prediction difference signal indicating the difference image to the encoding mode determination unit 3 ( Step ST7).
The encoding mode determination unit 3 evaluates the prediction efficiency of the prediction difference signal every time the prediction difference signal is received from the subtracter 2, and among the at least one prediction difference signal output from the subtractor 2, A prediction differential signal with the highest prediction efficiency is selected.
Since the process itself for evaluating the prediction efficiency of the prediction difference signal in the encoding mode determination unit 3 is a known technique, detailed description thereof is omitted.

When the coding mode determination unit 3 selects the prediction difference signal with the highest prediction efficiency, the motion compensation prediction unit 1 uses the motion vector used to generate the prediction image related to the prediction difference signal, and the macroblock type / sub Encoding mode information including a macroblock type (for example, including information indicating whether the encoding mode used in the macroblock is an inter mode or a direct mode) and a reference image identification number; It outputs to the variable length encoding part 9.
Also, the encoding mode determination unit 3 outputs a prediction difference signal with the highest prediction efficiency to the compression unit 4.
However, if the encoding mode is the inter mode, the encoding mode determination unit 3 includes the motion vector used for generating the predicted image in the encoding mode information and includes the encoding mode information including the motion vector. Is output to the variable-length encoding unit 9, but when the encoding mode is the direct mode, the motion vector used for generating the predicted image is not included in the encoding mode information but includes the motion vector. The encoding mode information which is not present is output to the variable length encoding unit 9.

Upon receiving the prediction difference signal from the coding mode determination unit 3, the compression unit 4 performs DCT processing on the prediction difference signal, thereby calculating a DCT coefficient and quantizing the DCT coefficient (step ST8). .
The compression unit 4 outputs the compressed data that is the DCT coefficient after quantization to the local decoding unit 5 and the variable length coding unit 9.

When receiving the compressed data from the compression unit 4, the local decoding unit 5 dequantizes the compressed data to obtain a DCT coefficient, and performs an inverse DCT process on the DCT coefficient to output from the coding mode determination unit 3. A prediction error signal corresponding to the predicted prediction difference signal is calculated.
When the local decoding unit 5 calculates the prediction error signal, the adder 6 adds the prediction error signal and the prediction signal indicating the prediction image generated by the motion compensated prediction unit 1, thereby local decoding indicating the local decoded image. An image signal is generated.
In order to prepare for the next encoding process, the loop filter 7 compensates for the encoding distortion included in the local decoded image signal output from the adder 6, and the local decoded image signal after the encoding distortion compensation indicates The decoded image is stored in the frame memory 8 as a reference image.

  When the variable length encoding unit 9 receives the compressed data from the compression unit 4, the variable length encoding unit 9 encodes the compressed data and the encoding mode information (macroblock type / sub macroblock type, motion vector (encoding) When the mode is the inter mode), the reference image identification number) is entropy-encoded, a bit stream indicating the encoding result is generated, and the bit stream is output (step ST9).

Next, processing contents of the image decoding apparatus in FIG. 3 will be described.
When the variable length decoding unit 21 receives the bit stream output from the image encoding device in FIG. 1, the variable length decoding unit 21 receives compressed data and encoding mode information (macroblock type / sub macroblock type, motion vector (encoding) from the bitstream. When the mode is the inter mode), the reference image identification number) is entropy decoded, the compressed data is output to the prediction error decoding unit 22, and the coding mode information is output to the motion compensation prediction unit 23 (FIG. 6). Step ST11).
When the prediction error decoding unit 22 receives the compressed data from the variable length decoding unit 21, the prediction error decoding unit 22 inversely quantizes the compressed data to obtain a DCT coefficient, and performs an inverse DCT process on the DCT coefficient, thereby predicting a difference image. An error signal (a prediction error signal corresponding to the prediction difference signal output from the encoding mode determination unit 3 in FIG. 1) is calculated (step ST12).

When the motion compensation prediction unit 23 receives the reference image identification number from the variable length decoding unit 21, the motion compensation prediction unit 23 reads the reference image indicated by the identification number from one or more reference images stored in the frame memory 26. Do.
When the motion compensation prediction unit 23 receives the macroblock type / sub-macroblock type from the variable-length decoding unit 21, the image coding apparatus in FIG. It is determined whether the inter mode or the direct mode is used as the conversion mode (step ST13).

The motion compensation prediction unit 23 uses the motion vector output from the variable length decoding unit 21 and the above reference image when the image coding apparatus in FIG. 1 uses the inter mode as the coding mode. A prediction image is generated by performing the prediction process.
On the other hand, when the image coding apparatus in FIG. 1 uses the direct mode as the coding mode, a temporal direct vector is generated in the same manner as the motion compensation prediction unit 1 in the image coding apparatus in FIG. A predicted image is generated by performing motion compensation prediction processing using the temporal direct vector.

Hereinafter, the processing content of the motion compensation prediction part 23 is demonstrated concretely.
The temporal direct vector generation unit 32 of the motion compensated prediction unit 23 indicates that the macroblock type / sub-macroblock type output from the variable length decoding unit 21 indicates that the direct mode is used. For each block, a macroblock located in the same spatial position as the macroblock is identified from among the macroblocks constituting the decoded picture that is temporally adjacent to the macroblock (step ST14).
For example, when the macroblock to be decoded is the macroblock MB1 of the picture B2 as shown in FIG. 12, the macroblock constituting the decoded picture P3 that is temporally adjacent to the macroblock MB1 is selected. The macro block MB2 located in the same spatial position as the macro block MB1 is specified.

When the temporal direct vector generation unit 32 specifies the macroblock MB2 that is spatially the same position as the macroblock MB1 to be decoded, the decoded motion vector stored in the motion vector memory 31 as shown in FIG. The motion vectors MV _i (i = 1, 2, 3, 4) of the four rectangular blocks (divided regions) located at the center of the macro block MB2 are acquired (step ST15).
When the temporal direct vector generation unit 32 acquires the motion vectors MV _i (i = 1, 2, 3, 4) of the four rectangular blocks, the four motion vectors MV are obtained as shown in the above equation (2). _{By obtaining} the average of _i , the temporal direct vector MV _direct in the temporal direct mode in the decoding target macroblock MB1 is generated (step ST16).
Temporal direct vector generation unit 32, when generating a temporal direct vector MV _direct the temporal direct mode, as a motion vector the temporal direct vector MV _direct, and outputs the motion compensation processing unit 33 and the motion vector memory 31.

When the motion compensation processing unit 33 of the motion compensation prediction unit 23 receives the identification number of the reference image from the variable length decoding unit 21, the identification number is selected from the reference images of one frame or more stored in the frame memory 26. The reference image shown is read out.
When the macroblock type / sub macroblock type output from the variable length decoding unit 21 indicates that the inter mode is used, the motion compensation processing unit 33 outputs the motion vector output from the variable length decoding unit 21. Using the reference image, the motion compensation prediction process is performed to generate a predicted image (step ST17).
On the other hand, when the macroblock type / sub-macroblock type output from the variable length decoding unit 21 indicates that the direct mode is used, the temporal direct vector MV _direct is transmitted from the temporal direct vector generation unit 32 as a motion vector. Then, using the temporal direct vector MV _direct and the reference image, a motion compensation prediction process is performed to generate a prediction image (step ST17).
Since the motion compensation prediction process of the motion compensation processing unit 33 is a known technique, detailed description thereof is omitted.

When the motion compensated prediction unit 23 generates a prediction image, the adder 24 adds the prediction image and the difference image indicated by the prediction error signal output from the prediction error decoding unit 22, and the adder 24 of the image encoding device in FIG. A decoded image signal indicating a decoded image corresponding to the locally decoded image output from the adder 6 is generated (step ST18).
When the adder 24 generates the decoded image signal, the loop filter 25 compensates for the encoding distortion included in the decoded image signal, and uses the decoded image indicated by the decoded image signal after the encoding distortion compensation as a reference image as a frame. While being stored in the memory 26, the decoded image is output to the outside (step ST19).

  As is apparent from the above, according to the first embodiment, the temporal direct vector generation unit 13 is a macroblock that constitutes an encoded picture that is temporally close to the macroblock to be encoded. To identify a macroblock that is in the same spatial position as the macroblock to be encoded, and generate a temporal direct vector in temporal direct mode from the motion vectors of the four rectangular blocks located at the center of the macroblock As a result, it is possible to generate a temporal direct vector in temporal direct mode that is close to the motion vector of the macroblock to be encoded, and as a result, increase the encoding efficiency of the macroblock to be encoded. There is an effect that an image encoding device that can be used is obtained.

  Further, according to the first embodiment, the temporal direct vector generation unit 32 selects the decoding target macroblock from among the macroblocks constituting the decoded picture that is temporally adjacent to the decoding target macroblock. Since the macro block located in the same spatial position is identified and the temporal direct vector of the temporal direct mode is generated from the motion vectors of the four rectangular blocks located at the center of the macro block. As a result, it is possible to generate a temporal direct vector in temporal direct mode that is close to the motion vector of the target macroblock. As a result, a stream encoded by the encoding device with high encoding efficiency of the decoding target macroblock can be obtained. There is an effect that an image decoding device capable of decoding can be obtained.

In the first embodiment, the temporal direct vector generation units 13 and 32 are based on the motion vectors MV _i (i = 1, 2, 3, 4) of the four rectangular blocks located at the center of the macroblock. Although it has been shown that the temporal direct vector MV _direct is generated in the temporal direct mode, the number of blocks located at the center of the macroblock is not limited to four, but three blocks located at the center of the macroblock. The time direct vector MV _direct in the time direct mode may be generated from 3 or less or 5 or more blocks if the number is 5 or less.
In the first embodiment, the temporal direct vector is generated as the motion vector. However, this vector may be used as the prediction vector.

Embodiment 2. FIG.
In the first embodiment, the temporal direct vector generation units 13 and 32 have the motion vectors MV _i (i = 1, 2) of the four rectangular blocks positioned at the center of the macroblock to be encoded (decoded). , 3, 4) has been shown to generate a temporal direct vector MV _direct in temporal direct mode, but the block sizes of a plurality of divided regions within a macroblock to be encoded (decoding target) Accordingly, the temporal direct vector MV _direct in the temporal direct mode may be generated by weighted addition of the motion vectors MV _i of a plurality of divided regions.

Hereinafter, a method of generating the time direct vector MV _direct will be specifically described.
FIG. 8 is an explanatory diagram showing a method of generating the time direct vector MV _direct by weighted addition according to the block size.
The temporal direct vector generation units 13 and 32, for each macroblock to be encoded (decoding target), encode pictures (decoded pictures) that are temporally adjacent to the macroblock, as in the first embodiment. ) Is identified from among the macroblocks constituting the same macroblock.
For example, if the macroblock to be encoded (decoding target) is the macroblock MB1 of the picture B2, as shown in FIG. 12, the encoded picture P3 (decoded picture) that is temporally adjacent to the macroblock MB1 Among the macroblocks constituting P3), the macroblock MB2 located in the same spatial position as the macroblock MB1 is specified.

When the temporal direct vector generation units 13 and 32 specify the macroblock MB2 that is spatially the same position as the macroblock MB1 to be encoded (decoding target), as shown in FIG. Among the stored encoded motion vectors (decoded motion vectors), motion vectors MV _i (i = 1, 1) of four blocks (divided regions) located at the center of the macro block MB2. 2, 3, 4).
In the example of FIG. 8, the motion vector of the upper left block is MV ₁ , the motion vector of the lower right block is MV _4, and numbers are assigned in raster scan order.
In addition, the temporal direct vector generation units 13 and 32 obtain distances d _i (i = 1, 2, 3, 4) from the center of the macroblock MB2 to the centers of the four blocks.
Note that the distance d _i from the center of the macro block MB2 to the centers of the four blocks is proportional to the size of the block.

When the temporal direct vector generation units 13 and 32 obtain the motion vector MV _i of the four blocks and the distance d _i from the center of the macroblock MB2 to the center of the four blocks, the following equation (3) is obtained. As shown in the drawing, the time direct vector MV _direct in the time direct mode is generated by weighted addition of the four motion vectors MV _i according to the distance d _i to the center of the four blocks.

As is apparent from the above, according to the second embodiment, the temporal direct vector generation units 13 and 32 perform weighted addition of the motion vectors MV _i of the plurality of divided regions according to the block sizes of the plurality of divided regions. Thus, since the temporal direct vector MV _direct in the temporal direct mode is generated, the motion vectors MV _i of the four rectangular blocks positioned at the center of the macro block are obtained as in the first embodiment. than for generating time by adding the average direct vector MV _direct, it will be able to accurately estimate a motion vector at the central position of the macro block MB1 of the encoding target (decoded), temporal direct vector MV _direct The effect which can improve the precision of is produced.

Embodiment 3 FIG.
In the first embodiment, the temporal direct vector generation units 13 and 32 have the motion vectors MV _i (i = 1, 2) of the four rectangular blocks positioned at the center of the macroblock to be encoded (decoded). , 3, 4) to generate the temporal direct vector MV _direct in the temporal direct mode by calculating the addition average, the differential motion of a plurality of divided regions in the macroblock to be encoded (decoding target) The time direct vector MV _direct in the time direct mode may be generated by weighted addition of the motion vectors MV _i of a plurality of divided regions according to the vector.

Hereinafter, a method of generating the time direct vector MV _direct will be specifically described.
FIG. 9 is an explanatory diagram showing a method of generating the temporal direct vector MV _direct by weighted addition according to the difference motion vector.
The temporal direct vector generation units 13 and 32, for each macroblock to be encoded (decoding target), encode pictures (decoded pictures) that are temporally adjacent to the macroblock, as in the first embodiment. ) Is identified from among the macroblocks constituting the same macroblock.
For example, if the macroblock to be encoded (decoding target) is the macroblock MB1 of the picture B2, as shown in FIG. 12, the encoded picture P3 (decoded picture) that is temporally adjacent to the macroblock MB1 Among the macroblocks constituting P3), the macroblock MB2 located in the same spatial position as the macroblock MB1 is specified.

When the temporal direct vector generation units 13 and 32 identify the macroblock MB2 that is spatially the same position as the encoding target (decoding target) macroblock MB1, as shown in FIG. Among the stored encoded motion vectors (decoded motion vectors), motion vectors MV _i (i = 1, 1) of four blocks (divided regions) located at the center of the macro block MB2. 2, 3, 4).
In the example of FIG. 9, the motion vector of the upper left block is MV ₁ , the motion vector of the lower right block is MV _4, and numbers are assigned in raster scan order.
In addition, the temporal direct vector generation units 13 and 32 acquire the differential motion vectors MVd _i (i = 1, 2, 3, 4) of the four blocks located at the center of the macroblock MB2.

When the temporal direct vector generation units 13 and 32 obtain the motion vector MV _i of four blocks and the differential motion vector MVd _i of four blocks, as shown in the following equation (5), depending on the differential motion vector MVd _i of the block, by weighted addition of four motion vectors MV _i, to generate a temporal direct vector MV _direct the temporal direct mode.
However, when the difference motion vector MVd _i is “0”, a fixed value A is given as a weight.

The magnitude of the difference motion vector MVd _i represents the degree of isolation of the vector, and the greater the magnitude of the difference motion vector MVd _i , the higher the possibility that the degree of isolation is greater. in 6), by incorporating the size inverse of the differential motion vector MVd _i, it is possible to reduce the weight of the isolated vector.

As is apparent from the above, according to the third embodiment, the temporal direct vector generation units 13 and 32 obtain the motion vectors MV _i of the plurality of divided regions according to the difference motion vectors MVd _i of the plurality of divided regions. Since the time direct vector MV _direct in the time direct mode is generated by weighting addition, the accuracy of the time direct vector MV _direct can be improved as compared with the first embodiment.

Embodiment 4 FIG.
In the second embodiment, the motion vectors MV _i of the plurality of divided regions are weighted and added according to the block sizes of the plurality of divided regions. In the third embodiment, the difference motion vectors MVd _i of the plurality of divided regions are added. Accordingly, the motion vector MV _i of the plurality of divided regions is weighted and added. However, the motion vector MV _i of the plurality of divided regions is weighted and added according to the encoding mode of the plurality of divided regions. Also good.

For example, when a divided region that is an encoded block (decoded block) is an intra block, a method of generating the temporal direct vector MV _direct by excluding the divided region from the calculation target can be considered.
Specifically, in FIG. 8, for example, when the upper left block (divided area) is an intra block (in FIG. 8, the motion vector MV ₁ is represented in the upper left block, but when it is an intra block, Time-direct mode time is obtained by excluding the upper left block from the calculation target and weighting and adding the motion vectors MV _i (i = 2, 3, 4) of the remaining three blocks. A direct vector MV _direct is generated.
Since an intra block does not have motion vector information, by excluding an intra block from the calculation target, the time direct vector does not need to be a zero vector unnecessarily, and the accuracy of the time direct vector is improved.

Embodiment 5 FIG.
In the above first to third embodiments, the generation of the temporal direct vector MV _direct using the vectors of the four blocks has been described. However, the temporal direct vector MV using the motion vectors of all the blocks in the macroblock. _Direct may be generated.
FIG. 10 is an explanatory diagram showing a method of generating the temporal direct vector MV _direct using the motion vectors of all the blocks in the macroblock.

MV _i (0 ≦ i ≦ k) is the motion vector of k blocks in the macro block MB2 in the same spatial position as the macro block MB1 to be encoded (decoding target), and the size of the k blocks is When S _i (0 ≦ i ≦ k), the time direct vector MV _direct in the time direct mode is generated by calculating the following equation (7).

  DESCRIPTION OF SYMBOLS 1 Motion compensation prediction part, 2 Subtractor (quantization means), 3 Encoding mode determination part (quantization means), 4 Compression part (quantization means), 5 Local decoding part, 6 Adder, 7 Loop filter, 8 Frame memory, 9 variable length coding unit (variable length coding unit), 11 motion vector memory, 12 motion vector search unit, 13 time direct vector generation unit (direct vector generation unit), 14 motion compensation processing unit (prediction image generation) Means), 21 variable length decoding section (variable length decoding means), 22 prediction error decoding section (inverse quantization means), 23 motion compensation prediction section, 24 adder (image addition means), 5 loop filter (image addition means) , 26 frame memory, 31 motion vector memory, 32 time direct vector generation unit (direct vector generation means), 33 motion compensation processing unit (preliminary) Image generation means).

Claims (10)

  For each block constituting the input image, a direct vector generation unit that generates a temporal direct vector in temporal direct mode from a motion vector of an encoded picture that is temporally adjacent to the block, and the direct vector generation unit By performing motion compensation prediction processing using the generated temporal direct vector, a prediction image generation unit that generates a prediction image, a difference image between the prediction image generated by the prediction image generation unit and the input image Quantizing means for quantizing and outputting the quantized coefficient of the difference image, and variable length code for encoding the quantized coefficient output from the quantizing means and outputting encoded data of the quantized coefficient And the direct vector generating means is temporally adjacent to the block to be encoded. A block that is in the same spatial position as the encoding target block is identified from among the blocks that make up a certain encoded picture, and the motion vectors of a plurality of divided regions that are located at the center of the block are used. An image encoding apparatus for generating a temporal direct vector in temporal direct mode.   2. The direct vector generation unit generates a temporal direct vector in a temporal direct mode by weighted addition of motion vectors of a plurality of divided regions according to block sizes of the plurality of divided regions. Image coding apparatus.   2. The direct vector generation means generates a temporal direct vector in temporal direct mode by weighted addition of motion vectors of a plurality of divided regions according to differential motion vectors of the plurality of divided regions. The image encoding device described.   2. The direct vector generation means generates a temporal direct vector in temporal direct mode by weighted addition of motion vectors of a plurality of divided regions according to encoding modes of the plurality of divided regions. The image encoding device described.   Variable length decoding means for decoding quantized coefficients from encoded data, inverse quantization means for inverse quantizing the quantized coefficients decoded by the variable length decoding means, and temporally close to the block to be decoded Prediction is achieved by performing a motion compensation prediction process using a direct vector generation unit that generates a temporal direct vector in temporal direct mode from a motion vector of a decoded picture and the temporal direct vector generated by the direct vector generation unit. Corresponding to the input image of the image encoding device by adding the prediction image generating means for generating an image, the prediction image generated by the prediction image generating means and the difference image indicated by the inverse quantization result of the inverse quantization means Image addition means for obtaining a decoded image to be decoded, and the direct vector generation means includes a block to be decoded. A block that is located in the same position as the decoding target block is identified from among the blocks that make up the decoded picture that is near in time, and a plurality of divided regions that are located at the center of the block An image decoding device that generates a temporal direct vector in temporal direct mode from a motion vector of the temporal motion vector.   6. The direct vector generation means generates a temporal direct vector in a temporal direct mode by weighted addition of motion vectors of a plurality of divided regions according to block sizes of the plurality of divided regions. Image decoding apparatus.   6. The direct vector generation means generates a temporal direct vector in a temporal direct mode by weighted addition of motion vectors of a plurality of divided regions according to differential motion vectors of the plurality of divided regions. The image decoding device described.   6. The direct vector generation means generates a temporal direct vector in a temporal direct mode by weighted addition of motion vectors of a plurality of divided regions in accordance with encoding modes of the plurality of divided regions. The image decoding device described.   A direct vector generation processing step for generating a temporal direct vector in temporal direct mode from a motion vector of an encoded picture that is temporally adjacent to the block for each block in which the direct vector generation means constitutes an input image; The prediction image generation means performs the motion compensation prediction processing using the temporal direct vector generated in the direct vector generation processing step, and the prediction means generation processing step for generating a prediction image, and the quantization means performs the prediction Quantize the difference image between the prediction image generated in the image generation processing step and the input image, and output the quantization coefficient of the difference image, and the variable length coding means outputs in the quantization processing step The encoded quantization coefficient is variable-length encoded and encoded data of the quantization coefficient A variable-length encoding processing step for outputting, and when the direct vector generation means performs the direct vector generation processing step, an encoded picture that is temporally adjacent to the block to be encoded is configured. A block located in the same spatial position as the block to be encoded is identified from the blocks, and a temporal direct vector in temporal direct mode is generated from motion vectors of a plurality of divided regions located at the center of the block. An image encoding method characterized by the above.   A variable length decoding unit in which the variable length decoding unit decodes the quantized coefficient from the encoded data, and an inverse quantization unit in which the inverse quantizing unit dequantizes the quantized coefficient decoded in the variable length decoding step. A direct vector generation processing step in which a direct vector generation means generates a temporal direct vector in temporal direct mode from a motion vector of a decoded picture that is temporally adjacent to a block to be decoded; and a predicted image generation means includes the direct vector The motion compensation prediction process is performed using the temporal direct vector generated in the generation process step, so that a predicted image generation process step for generating a predicted image, and the image addition unit is generated in the predicted image generation process step. Difference between the predicted image and the result of inverse quantization in the inverse quantization process step And an image addition processing step for obtaining a decoded image corresponding to the input image of the image encoding device, and when the direct vector generation means performs the direct vector generation processing step, A block that is in the same position as the decoding target block is identified from among the blocks that make up the decoded picture that is temporally nearby, and a plurality of divided regions that are located at the center of the block are identified. An image decoding method, wherein a temporal direct vector in temporal direct mode is generated from a motion vector.

Images