JP6961781B2 - Image coding method and image decoding method - Google Patents

Description

The present invention relates to coding and decoding methods for moving and still images.

In recent years, video coding methods with significantly improved coding efficiency have been developed in collaboration with ITU-T and ISO / IEC, such as ITU-T Rec. H.264 and ISO / IEC 14496-10 (hereinafter referred to as H.H. It is recommended as (264). In H.264, prediction processing, conversion processing, and entropy coding processing are performed in rectangular block units (for example, 16 × 16 pixel block units, 8 × 8 pixel block units, etc.). In the prediction process, motion compensation is performed on the rectangular block to be encoded (block to be encoded) by referring to the already encoded frame (reference frame) to make a prediction in the time direction. In such motion compensation, it is necessary to encode the motion information including the motion vector as the spatial shift information between the coded block and the block referenced in the reference frame and send it to the decoding side. Further, when motion compensation is performed using a plurality of reference frames, it is necessary to encode the reference frame number together with the motion information. Therefore, the amount of code related to the motion information and the reference frame number may increase.

As an example of the method of obtaining the motion vector in the motion compensation prediction, the motion vector to be assigned to the coded block is derived from the motion vector assigned to the already encoded block, and the motion vector is predicted based on the derived motion vector. There is a direct mode for generating an image (see Patent Document 1 and Patent Document 2). In the direct mode, since the motion vector is not encoded, the code amount of the motion information can be reduced. The direct mode is adopted in H.264 / AVC, for example.

Patent No. 4020789 U.S. Pat. No. 7,233,621

In the direct mode, the motion vector of the coded block is predicted and generated by a fixed method of calculating the motion vector from the median value of the motion vector of the coded block adjacent to the coded block. Therefore, the degree of freedom in calculating the motion vector is low.

In order to increase the degree of freedom in calculating the motion vector, a method has been proposed in which one is selected from a plurality of coded blocks and a motion vector is assigned to the coded block. In this method, selection information that identifies the selected block must always be transmitted so that the decoding side can identify the selected encoded block. Therefore, when one of a plurality of coded blocks is selected to determine the motion vector to be assigned to the coded block, there is a problem that the amount of code related to the selection information is increased.

The present invention has been made to solve the above problems, and an object of the present invention is to provide an image coding and image decoding method having high coding efficiency.

The image coding method according to the embodiment of the present invention includes a first step of selecting at least one motion reference block from encoded pixel blocks having motion information, and a motion applied to the coded block. A second step of selecting at least one available block having motion information different from each other, which is a pixel block having information candidates, from the motion reference blocks, and one selection from the available blocks. The third step of selecting a block, the fourth step of generating a predicted image of the coded block using the motion information of the selected block, and the prediction error between the predicted image and the original image are coded. It includes a fifth step of encoding the selection information for identifying the selection block with reference to a code table predetermined according to the number of available blocks.

The image decoding method according to another embodiment of the present invention is applied to the first step of selecting at least one motion reference block from the decoded pixel blocks having motion information, and the decoding target block. A second step of selecting at least one available block having motion information candidates and having different motion information from the motion reference blocks from the motion reference blocks, and in advance according to the number of available blocks. A third step of obtaining selection information for identifying a selected block by decoding the input encoded data with reference to a defined code table, and in the available blocks according to the selection information. A fourth step of selecting one selected block from the above, a fifth step of generating a predicted image of the decoding target block using the motion information of the selected block, and the decoding target block from the coded data. It includes a sixth step of decoding the predicted residual of the above, and a seventh step of obtaining the decoded image from the predicted image and the predicted residual.

According to the present invention, the coding efficiency can be improved.

It is a block diagram which shows schematic structure of the image coding apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the size of the macro block which is the processing unit of the coding of the image decoding part shown in FIG. It is a figure which shows another example of the size of the macro block which is the processing unit of the coding of the image decoding unit shown in FIG. It is a figure which shows the order in which the image coding part shown in FIG. 1 encodes a pixel block in a frame to be coded. It is a figure which shows an example of the motion information frame held by the motion information memory shown in FIG. It is a flowchart which shows an example of the procedure for processing the input image signal of FIG. It is a figure which shows an example of the inter-prediction processing executed by the motion compensation part of FIG. It is a figure which shows another example of the inter-prediction processing executed by the motion compensation part of FIG. It is a figure which shows an example of the size of the motion compensation block used for inter prediction processing. It is a figure which shows another example of the size of the motion compensation block used for inter prediction processing. It is a figure which shows still another example of the size of the motion compensation block used for inter prediction processing. It is a figure which shows another example of the size of the motion compensation block used for inter prediction processing. It is a figure which shows an example of arrangement of a movement reference block in a space direction and a time direction. It is a figure which shows another example of arrangement of a spatial direction movement reference block. It is a figure which shows the relative position of the spatial direction movement reference block with respect to the coded object block shown in FIG. 8B. It is a figure which shows another example of arrangement of the time direction movement reference block. It is a figure which shows still another example of arrangement of a time direction movement reference block. It is a figure which shows still another example of arrangement of a time direction movement reference block. FIG. 5 is a flowchart showing an example of a method in which the available block acquisition unit of FIG. 1 selects an available block from the motion reference blocks. It is a figure which shows an example of the available block selected according to the method of FIG. 9 from the motion reference block shown in FIG. It is a figure which shows an example of the available block information output by the available block acquisition part of FIG. It is a figure which shows an example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a figure which shows another example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a figure which shows still another example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a figure which shows another example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a figure which shows still another example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a figure which shows another example of the identity determination of the motion information between blocks by the available block acquisition part of FIG. It is a block diagram which shows schematic structure of the prediction part of FIG. It is a figure which shows the group of the motion information output by the time direction motion information acquisition part of FIG. It is explanatory drawing explaining the interpolation processing of the minority pixel accuracy that can be used in the motion compensation processing by the motion compensation part of FIG. It is a flowchart which shows an example of the operation of the prediction part of FIG. It is a figure which shows a mode that the motion compensation part of FIG. 13 copies the motion information of a motion reference block in a time direction to a block to be encoded. It is a block diagram which shows schematic structure of the variable length coding part of FIG. It is a figure which shows the example which generates a syntax according to available block information. It is a figure which shows the example of binarization of the selected block information syntax corresponding to available block information. It is explanatory drawing explaining the scaling of motion information. It is a figure which shows the syntax structure according to embodiment. It is a figure which shows an example of the macroblock layer synctus according to 1st Embodiment. It is a figure which shows another example of the macroblock layer synctus according to 1st Embodiment. H. It is a figure which shows the code table corresponding to mb_type and mb_type at the time of B slice in 264. It is a figure which shows an example of the code table which concerns on embodiment. H. It is a figure which shows the code table corresponding to mb_type and mb_type at the time of P slice in 264. It is a figure which shows another example of the code table which concerns on embodiment. It is a figure which shows an example of the code table corresponding to mb_type and mb_type in B slice according to embodiment. It is a figure which shows the other example of the code table corresponding to mb_type and mb_type in P slice according to embodiment. It is a block diagram which shows schematic structure of the image coding apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the prediction part of FIG. 26 schematicly. It is a block diagram which shows the structure of the 2nd prediction part of FIG. 27 schematicly. It is a block diagram which shows schematic structure of the variable length coding part of FIG. It is a figure which shows an example of the macroblock layer syntax according to 2nd Embodiment. It is a figure which shows another example of the macroblock layer syntax according to 2nd Embodiment. It is a block diagram which shows schematic the image decoding apparatus which concerns on 3rd Embodiment. It is a block diagram which shows the coded sequence decoding part shown in FIG. 31 in more detail. It is a block diagram which shows the prediction part shown in FIG. 31 in more detail. It is a block diagram which shows schematic the image decoding apparatus which concerns on 4th Embodiment. It is a block diagram which shows the coded sequence decoding part shown in FIG. 33 in more detail. It is a block diagram which shows the prediction part shown in FIG. 33 in more detail.

Hereinafter, the method and apparatus for image coding and image decoding according to the embodiment of the present invention will be described with reference to the drawings as necessary. In the following embodiments, the same operation is performed for the parts with the same number, and the description thereof will be omitted.

(First Embodiment)
FIG. 1 schematically shows the configuration of an image coding device according to the first embodiment of the present invention. As shown in FIG. 1, this image coding device includes an image coding unit 100, a coding control unit 150, and an output buffer 120. This image coding device may be realized by hardware such as an LSI chip, or may be realized by causing a computer to execute an image coding program.

An original image (input image signal) 10 which is a moving image or a still image is input to the image coding unit 100, for example, in pixel block units obtained by dividing the original image. The image coding unit 100 compresses and encodes the input image signal 10 to generate the coded data 14, as will be described in detail later. The generated coded data 14 is temporarily stored in the output buffer 120 and sent to a storage system (storage medium) or a transmission system (communication line) (not shown) at an output timing managed by the coding control unit 150. ..

The coding control unit 150 controls the overall coding processing of the image coding unit 100, such as feedback control of the generated code amount, quantization control, prediction mode control, and entropy coding control. Specifically, the coding control unit 150 gives the coding control information 50 to the image coding unit 100, and appropriately receives the feedback information 51 from the image coding unit 100. The coding control information 50 includes prediction information, motion information 18, quantization parameter information, and the like. The prediction information includes prediction mode information and block size information. The motion information 18 includes a motion vector, a reference frame number, and a prediction direction (unidirectional prediction, bidirectional prediction). The quantization parameter information includes a quantization parameter such as a quantization width (quantization step size) and a quantization matrix. The feedback information 51 includes the amount of code generated by the image coding unit 100 and is used, for example, to determine the quantization parameter.

The image coding unit 100 encodes the input image signal 10 in units of pixel blocks (for example, macroblocks, subblocks, one pixel, etc.) obtained by dividing the original image. Therefore, the input image signal 10 is sequentially input to the image coding unit 100 in pixel block units obtained by dividing the original image. In the present embodiment, the coding processing unit is a macro block, and the pixel block (macro block) to be coded corresponding to the input image signal 10 is simply referred to as a code target block. Further, an image frame including a code target block, that is, an image frame to be coded is referred to as a code target frame.

Such a coded block may be, for example, a 16 × 16 pixel block as shown in FIG. 2A or a 64 × 64 pixel block as shown in FIG. 2B. Further, the coding target block may be a 32 × 32 pixel block, an 8 × 8 pixel block, or the like. Further, the shape of the macroblock is not limited to the square shape shown in FIGS. 2A and 2B, and may be set to an arbitrary shape such as a rectangular shape. Further, the processing unit is not limited to a pixel block such as a macro block, and may be a frame or a field.

The coding process for each pixel block in the coded frame may be executed in any order. In the present embodiment, for the sake of simplicity, as shown in FIG. 3, the pixel blocks from the upper left pixel block of the coded frame to the lower right pixel block are line by line, that is, in the order of raster scan. It is assumed that the coding process is executed on the device.

The image coding unit 100 shown in FIG. 1 includes a prediction unit 101, a subtractor 102, a conversion / quantization unit 103, a variable length coding unit 104, an inverse quantization / inverse conversion unit 105, an adder 106, and a frame memory 107. , The motion information memory 108 and the available block acquisition unit 109 are provided.

In the image coding unit 100, the input image signal 10 is input to the prediction unit 101 and the subtractor 102. The subtractor 102 receives the input image signal 10 and also receives the predicted image signal 11 from the prediction unit 101, which will be described later. The subtractor 102 calculates the difference between the input image signal 10 and the predicted image signal 11 to generate the predicted error image signal 12.

The conversion / quantization unit 103 receives the prediction error image signal 12 from the subtractor 102, performs conversion processing on the received prediction error image signal 12, and generates a conversion coefficient. The transformation process is, for example, an orthogonal transform such as a Discrete Cosine Transform (DCT). In another embodiment, the transform / quantization unit 103 may generate transform coefficients by using techniques such as wavelet transform and independent component analysis instead of the discrete cosine transform. Further, the conversion / quantization unit 103 quantizes the generated conversion coefficient based on the quantization parameter given by the coding control unit 150. The quantized conversion coefficient (conversion coefficient information) 13 is output to the variable-length coding unit 104 and the inverse quantization / inverse conversion unit 105.

The inverse quantization / inverse conversion unit 105 dequantizes the quantized conversion coefficient 13 according to the quantization parameter given by the coding control unit 150, that is, the same quantization parameter as the conversion / quantization unit 103. .. Subsequently, the inverse quantization / inverse transformation unit 105 performs inverse transformation on the inverse quantization conversion coefficient to generate the decoding prediction error signal 15. The inverse conversion process by the inverse quantization / inverse conversion unit 105 corresponds to the inverse transformation process of the conversion process by the conversion / quantization unit 103. For example, the inverse transform process is an inverse discrete cosine transform (IDCT) or an inverse wavelet transform.

The adder 106 receives the decoding prediction error signal 15 from the inverse quantization / inverse conversion unit 105, and further receives the prediction image signal 11 from the prediction unit 101. The adder 106 adds the decoding prediction error signal 15 and the prediction image signal 11 to generate the locally decoded image signal 16. The generated locally decoded image signal 16 is stored in the frame memory 107 as a reference image signal 17. The reference image signal 17 stored in the frame memory 107 is read out and referenced by the prediction unit 101 when encoding the subsequent coded block.

The prediction unit 101 receives the reference image signal 17 from the frame memory 107, and also receives the available block information 30 from the available block acquisition unit 109, which will be described later. Further, the prediction unit 101 receives the reference motion information 19 from the motion information memory 108 described later. The prediction unit 101 generates the prediction image signal 11, the motion information 18, and the selection block information 31 of the coded block based on the reference image signal 17, the reference motion information 19, and the available block information 30. Specifically, the prediction unit 101 is based on the motion information selection unit 118 that generates the motion information 18 and the selection block information 31 based on the available block information 30 and the reference motion information 19, and the motion information 18. A motion compensation unit 113 that generates a predicted image signal 11 is provided. The predicted image signal 11 is sent to the subtractor 102 and the adder 106. The motion information 18 is stored in the motion information memory 108 for subsequent prediction processing for the coded target block. Further, the selected block information 31 is sent to the variable length coding unit 104. The prediction unit 101 will be described in detail later.

The motion information 18 is temporarily stored in the motion information memory 108 as the reference motion information 19. FIG. 4 shows an example of the configuration of the motion information memory 108. As shown in FIG. 4, the reference motion information 19 is held in the motion information memory 108 in frame units, and the reference motion information 19 forms the motion information frame 25. Motion information 18 relating to the coded block is sequentially input to the motion information memory 108, and as a result, the motion information memory 108 holds a plurality of motion information frames 25 having different coding times.

The reference motion information 19 is held in the motion information frame 25 in predetermined block units (for example, 4 × 4 pixel block units). The motion vector block 28 shown in FIG. 4 shows a pixel block having the same size as the coded target block, the available block, the selected block, and the like, and is, for example, a 16 × 16 pixel block. A motion vector is assigned to the motion vector block 28, for example, for each 4 × 4 pixel block. Inter-prediction processing using motion vector blocks is called motion vector block prediction processing. The reference motion information 19 held by the motion information memory 108 is read out by the prediction unit 101 when the motion information 18 is generated. The motion information 18 included in the available block as described later refers to the reference motion information 19 held in the area where the available block is located in the motion information memory 108.

The motion information memory 108 is not limited to the example in which the reference motion information 19 is held in units of 4 × 4 pixel blocks, and the reference motion information 19 may be held in units of other pixel blocks. For example, the pixel block unit for the reference motion information 19 may be one pixel or a 2 × 2 pixel block. Further, the shape of the pixel block related to the reference motion information 19 is not limited to the square shape, and may be an arbitrary shape.

The usable block acquisition unit 109 of FIG. 1 acquires the reference motion information 19 from the motion information memory 108, and based on the acquired reference motion information 19, the prediction unit 101 is obtained from a plurality of blocks for which coding has already been completed. Select the available blocks that can be used for the prediction process of. The selected available block is sent to the prediction unit 101 and the variable length coding unit 104 as available block information 30. A coded block that is a candidate for selecting an available block is called a motion reference block. The method of selecting the motion reference block and the available block will be described in detail later.

In addition to the conversion coefficient information 13, the variable-length coding unit 104 can use the selection block information 31 from the prediction unit 101 and the prediction information and coding parameters such as quantization parameters from the coding control unit 150. Receive the available block information 30 from 109. The variable-length coding unit 104 entropy-codes the quantized conversion coefficient 13, the selected block information 31, the available block information 30, and the coding parameters (for example, equal-length coding, Huffman coding, arithmetic coding, etc.). ) To generate the encoded data 14. The coding parameter includes the selection block information 31 and the prediction information, as well as all parameters required for decoding such as information on the conversion coefficient and information on the quantization. The generated coded data 14 is temporarily stored in the output buffer 120 and sent to a storage system or a transmission system (not shown).

FIG. 5 shows a processing procedure of the input image signal 10. As shown in FIG. 5, first, the prediction image signal 11 is generated by the prediction unit 101 (step S501). In the generation of the predicted image signal 11 in step S501, one of the available blocks described later is selected as the selected block, and the predicted image is used by using the selected block information 31, the motion information of the selected block, and the reference image signal 17. Signal 11 is created. The difference between the predicted image signal 11 and the input image signal 10 is calculated by the subtractor 102, and the predicted error image signal 12 is generated (step S502).

Subsequently, the prediction error image signal 12 is subjected to orthogonal transformation and quantization by the conversion / quantization unit 103, and the conversion coefficient information 13 is generated (step S503). The conversion coefficient information 13 and the selected block information 31 are sent to the variable-length coding unit 104, subjected to variable-length coding, and coded data 14 is generated (step S504). Further, in step S504, the code table is switched so that the code table has the same number of entries as the number of available blocks according to the selected block information 31, and the selected block information 31 is variable-length coded. The bit stream 20 of the coded data is sent to a storage system or a transmission line (not shown).

The conversion coefficient information 13 generated in step S503 is inversely quantized by the inverse quantization / inverse conversion unit 105, subjected to inverse transformation processing, and becomes a decoding prediction error signal 15 (step S505). The decoding prediction error signal 15 is added to the reference image signal 17 used in step S501 to become a locally decoded image signal 16 (step S506), and is stored as a reference image signal in the frame memory 107 (step S507).

Next, each configuration of the image coding unit 100 described above will be described in more detail.
The image coding unit 100 of FIG. 1 is provided with a plurality of prediction modes, and the method of generating the predicted image signal 11 and the motion compensation block size are different from each other in each prediction mode. The method by which the prediction unit 101 generates the prediction image signal 11 is roughly divided into an intra prediction (inside the frame) in which the prediction image is generated by using the reference image signal 17 relating to the coded frame (or field). Prediction) and inter-prediction (inter-frame prediction) that generates a prediction image using the reference image signal 17 for one or more encoded reference frames (reference fields). The prediction unit 101 selectively switches between the intra prediction and the inter prediction to generate the prediction image signal 11 of the coded block.

FIG. 6A shows an example of inter-prediction by the motion compensation unit 113. In the inter-prediction, as shown in FIG. 6A, from the block (also referred to as the prediction block) 23 which is a block in the reference frame one frame before the coding has already been completed and is at the same position as the coded target block. The predicted image signal 11 is generated by using the reference image signal 17 relating to the block 24 at the position spatially shifted according to the motion vector 18a included in the motion information 18. That is, in the generation of the predicted image signal 11, the reference image signal 17 relating to the block 24 in the reference frame specified by the position (coordinates) of the coded block and the motion vector 18a included in the motion information 18 is used. .. In inter-prediction, motion compensation with a small number of pixel accuracy (for example, 1/2 pixel accuracy or 1/4 pixel accuracy) is possible, and the value of the interpolated pixel is generated by performing filtering processing on the reference image signal 17. Will be done. For example, H. In 264, interpolation processing up to 1/4 pixel accuracy is possible for the luminance signal. When motion compensation with 1/4 pixel accuracy is performed, the amount of information of the motion information 18 is four times the integer pixel accuracy.

Note that the inter-prediction is not limited to the example of using the reference frame one frame before as shown in FIG. 6A, and any encoded reference frame may be used as shown in FIG. 6B. .. When the reference image signals 17 relating to a plurality of reference frames having different time positions are held, the information indicating from which the reference image signal 17 at which time position the predicted image signal 11 is generated is represented by the reference frame number. The reference frame number is included in the motion information 18. The reference frame number can be changed in units of areas (pictures, blocks, etc.). That is, a different reference frame can be used for each pixel block. As an example, if the encoded reference frame one frame before is used for prediction, the reference frame number in this area is set to 0, and if the encoded reference frame two frames before is used for prediction, The reference frame number in this area is set to 1. As another example, when the reference image signal 17 for only one frame is held in the frame memory 107 (the number of reference frames is 1), the reference frame number is always set to 0.

Further, in the inter-prediction, a block size suitable for the coded block can be selected from a plurality of motion compensation blocks. That is, the coded block may be divided into a plurality of small pixel blocks, and motion compensation may be performed for each small pixel block. 7A to 7C show the size of the motion compensation block in macroblock units, and FIG. 7D shows the size of the motion compensation block in subblocks (pixel blocks of 8 × 8 pixels or less). As shown in FIG. 7A, when the coded block has 64 × 64 pixels, the motion compensation block includes a 64 × 64 pixel block, a 64 × 32 pixel block, a 32 × 64 pixel block, a 32 × 32 pixel block, and the like. Can be selected. Further, as shown in FIG. 7B, when the coded block has 32 × 32 pixels, the motion compensation block includes a 32 × 32 pixel block, a 32 × 16 pixel block, a 16 × 32 pixel block, or a 16 × 16 pixel. Blocks and the like can be selected. Further, as shown in FIG. 7C, when the coded block is 16 × 16 pixels, the motion compensation block is a 16 × 16 pixel block, a 16 × 8 pixel block, an 8 × 16 pixel block or an 8 × 8 pixel. It can be set as a block or the like. Furthermore, as shown in FIG. 7D, when the coded block is 8 × 8 pixels, the motion compensation block is an 8 × 8 pixel block, an 8 × 4 pixel block, a 4 × 8 pixel block or a 4 × 4 Pixel blocks and the like can be selected.

As described above, since the small pixel block (for example, 4 × 4 pixel block) in the reference frame used for the inter-prediction has the motion information 18, it is optimal according to the local property of the input image signal 10. The shape and motion vector of the motion compensation block can be used. Further, the macroblocks and sub-macroblocks of FIGS. 7A to 7D can be arbitrarily combined. When the coded block is a 64 × 64 pixel block as shown in FIG. 7A, the block size shown in FIG. 7B is set for each of the four 32 × 32 pixel blocks obtained by dividing the 64 × 64 pixel block. By selecting, blocks of 64 × 64 to 16 × 16 pixels can be used hierarchically. Similarly, when the block size shown in FIG. 7D can be selected, the block size of 64 × 64 to 4 × 4 can be used hierarchically.

Next, the motion reference block will be described with reference to FIGS. 8A to 8F.
The motion reference block is selected from the coded frame and the coded area (block) in the reference frame according to the method agreed upon by both the image coding device of FIG. 1 and the image decoding device described later. NS. FIG. 8A shows an example of the arrangement of the motion reference block selected according to the position of the coded block. In the example of FIG. 8A, nine motion reference blocks A to D and TA to TE are selected from the coded frame and the coded region within the reference frame. Specifically, four blocks A, B, C, and D adjacent to the left, top, upper right, and upper left of the coded block are selected as motion reference blocks from the coded frame, and from the reference frame, the four blocks A, B, C, and D are selected as motion reference blocks. A block TA at the same position as the coded block and four pixel blocks TB, TC, TD, and TE adjacent to the right, bottom, left, and top of this block TA are selected as motion reference blocks. In the present embodiment, the motion reference block selected from the coded frame is referred to as a spatial motion reference block, and the motion reference block selected from the reference frame is referred to as a temporal motion reference block. The symbol p assigned to each motion reference block in FIG. 8A indicates an index of the motion reference block. This index is numbered in the order of the movement reference block in the time direction and the space direction, but the index is not limited to this, and it does not have to be in this order as long as the indexes do not overlap. For example, the movement reference blocks in the temporal and spatial directions may be numbered out of order.

The spatial movement reference block is not limited to the example shown in FIG. 8A, and as shown in FIG. 8B, the block to which the pixels a, b, c, and d adjacent to the coded target block belong (for example, a macro block or a sub). It may be a macroblock, etc.). In this case, the relative positions (dx, dy) from the upper left pixel e in the coded target block to each pixel a, b, c, d are set as shown in FIG. 8C. Here, in the examples shown in FIGS. 8A and 8B, the macroblock is shown as being an N × N pixel block.

Further, as shown in FIG. 8D, all blocks A1 to A4, B1, B2, C, and D adjacent to the coded block may be selected as spatial movement reference blocks. In the example of FIG. 8D, the number of spatial movement reference blocks is eight.

Further, in the time direction movement reference block, as shown in FIG. 8E, a part of each block TA to TE may overlap, and as shown in FIG. 8F, the blocks TA to TE are arranged apart from each other. It doesn't matter if you do. In FIG. 8E, the portion where the time direction movement reference blocks TA and TB overlap is indicated by diagonal lines. Furthermore, the time direction movement reference block is not necessarily limited to the block at the position (collocation position) corresponding to the coded block and the block located around it, and is arranged at any position in the reference frame. It does not matter if it is a block. For example, the block in the reference frame specified by the position of the reference block and the motion information 18 of any of the coded blocks adjacent to the coded block is used as the central block (for example, block TA). The central block and surrounding blocks may be selected as temporal movement reference blocks. Further, the time direction reference blocks do not have to be arranged at equal intervals from the central block.

In any of the above cases, if the number and position of the spatial and temporal movement reference blocks are determined in advance by the encoding device and the decoding device, how can the number and position of the motion reference blocks be set? It doesn't matter. Further, the size of the motion reference block does not necessarily have to be the same size as the coded block. For example, as shown in FIG. 8D, the size of the motion reference block may be larger or smaller than the coded block. Further, the motion reference block is not limited to a square shape, and may be set to an arbitrary shape such as a rectangular shape. Further, the motion reference block may be set to any size.

Further, the motion reference block and the available block may be arranged in only one of the temporal direction and the spatial direction. Further, the movement reference block and the available block in the time direction may be arranged according to the type of slice such as P slice and B slice, and the movement reference block and the available block in the spatial direction may be arranged.

FIG. 9 shows a method in which the available block acquisition unit 109 selects an available block from the motion reference blocks. The usable block is a block to which motion information can be applied to a block to be encoded, and has motion information different from each other. The usable block acquisition unit 109 refers to the reference motion information 19 and determines whether or not each motion reference block is an available block according to the method shown in FIG. 9, and outputs the available block information 30.

As shown in FIG. 9, first, the motion reference block having the index p of zero is selected (S800). In the explanation of FIG. 9, it is assumed that the motion reference blocks are processed in order from 0 to M-1 (where M indicates the number of motion reference blocks). Further, it is assumed that the availability determination process for the motion reference block whose index p is 0 to p-1 is completed and the index of the motion reference block to be determined whether or not it is available is p. ..

The available block acquisition unit 109 determines whether or not the motion reference block p has motion information 18, that is, whether or not at least one motion vector is assigned (S801). When the motion reference block p does not have a motion vector, that is, the temporal motion reference block p is a block in the I slice that does not have motion information, or is in the temporal motion reference block p. If all the small pixel blocks are intra-predictively encoded, the process proceeds to step S805. In step S805, the motion reference block p is determined to be an unusable block.

If the motion reference block p has motion information in step S801, the process proceeds to step S802. The usable block acquisition unit 109 selects the motion reference block q (usable block q) that has already been selected as the usable block. Here, q is a value smaller than p. Subsequently, the available block acquisition unit 109 compares the motion information 18 of the motion reference block p with the motion information 18 of the available block q, and determines whether or not they have the same motion information (S803). .. If it is determined that the motion information 18 of the motion reference block p and the motion information 18 of the motion reference block q selected as the available block are the same, the process proceeds to step S805, and the motion reference block p is the unusable block. Is determined.

If it is determined in step S803 that the motion information 18 of the motion reference block p and the motion information 18 of the available block q are not the same for all the available blocks q satisfying q <p, the process proceeds to step S804. .. In step S804, the available block acquisition unit 109 determines the motion reference block p as the available block.

When it is determined that the motion reference block p is an available block or an unavailable block, the available block acquisition unit 109 determines whether or not the availability determination has been executed for all the motion reference blocks ( S806). If there is a motion reference block for which the availability determination has not been executed, for example, if p <M-1, the process proceeds to step S807. Subsequently, the available block acquisition unit 109 increments the index p by 1 (step S807), and executes steps S801 to S806 again. When the availability determination is executed for all the motion reference blocks in step S806, the availability determination process ends.

By executing the availability determination process described above, it is determined whether each motion reference block is an available block or an unavailable block. The available block acquisition unit 109 generates available block information 30 including information about available blocks. By selecting the available block from the motion reference blocks in this way, the amount of information regarding the available block information 30 can be reduced, and as a result, the amount of coded data 14 can be reduced.

FIG. 10 shows an example of the result of executing the availability determination process on the motion reference block shown in FIG. 8A. In FIG. 10, it is determined that the two spatial movement reference blocks (p = 0,1) and the two temporal movement reference blocks (p = 5,8) are usable blocks. An example of the available block information 30 relating to the example of FIG. 10 is shown in FIG. As shown in FIG. 11, the available block information 30 includes the index, availability, and motion reference block name of the motion reference block. In the example of FIG. 11, index p = 0, 1, 5, and 8 are available blocks, and the number of available blocks is 4. The prediction unit 101 selects the optimum one available block from these available blocks as a selection block, and outputs information (selection block information) 31 regarding the selection block. The selected block information 31 includes the number of available blocks and the index value of the selected available blocks. For example, when the number of available blocks is 4, the corresponding selected block information 31 is encoded by the variable length coding unit 104 using a code table having a maximum entry of 4.

In step S801 of FIG. 9, when at least one of the blocks in the time direction motion reference block p is an intra prediction encoded block, the available block acquisition unit 109 uses the motion reference block p. It may be determined as an impossible block. That is, the process may proceed to step S802 only when all the blocks in the time direction movement reference block p are encoded by the inter-prediction.

12A to 12E show an example in which the motion information 18 of the motion reference block p and the motion information 18 of the available block q are determined to be the same in the comparison of the motion information 18 in step S803. 12A to 12E show a plurality of shaded blocks and two whitewashed blocks, respectively. In FIGS. 12A to 12E, for the sake of simplicity, it is assumed that the motion information 18 of these two white-painted blocks is compared without considering the shaded blocks. It is assumed that one of the two whitewashed blocks is the motion reference block p and the other is the motion reference block q (available block q) that has already been determined to be available. Unless otherwise specified, either of the two white blocks may be the motion reference block p.

FIG. 12A shows an example in which both the motion reference block p and the available block q are spatially oriented blocks. In the example of FIG. 12A, if the motion information 18 of the blocks A and B is the same, it is determined that the motion information 18 is the same. At this time, the sizes of blocks A and B do not have to be the same.

FIG. 12B shows an example in which one of the motion reference block p and the available block q is the block A in the spatial direction and the other is the block TB in the time direction. In FIG. 12B, there is one block having motion information in the block TB in the time direction. If the motion information 18 of the block TB in the time direction and the motion information 18 of the block A in the spatial direction are the same, it is determined that the motion information 18 is the same. At this time, the sizes of blocks A and TB do not have to be the same.

FIG. 12C shows another example in which one of the motion reference block p and the available block q is the block A in the spatial direction and the other is the block TB in the time direction. FIG. 12C shows a case where the block TB in the time direction is divided into a plurality of small blocks, and there are a plurality of small blocks having motion information 18. In the example of FIG. 12C, if all the blocks having the motion information 18 have the same motion information 18 and the motion information 18 is the same as the motion information 18 of the block A in the spatial direction, the motion information 18 is the same. Is determined. At this time, the sizes of blocks A and TB do not have to be the same.

FIG. 12D shows an example in which both the motion reference block p and the available block q are blocks in the time direction. In this case, if the motion information 18 of the blocks TB and TE is the same, it is determined that the motion information 18 is the same.

FIG. 12E shows another example in which the motion reference block p and the available block q are both blocks in the time direction. FIG. 12E shows a case where the blocks TB and TE in the time direction are each divided into a plurality of small blocks, and each of them has a plurality of small blocks having motion information 18. In this case, the motion information 18 is compared for each small block in the block, and if the motion information 18 is the same for all the small blocks, the motion information 18 of the block TB and the motion information 18 of the block TE are the same. It is determined that there is.

FIG. 12F shows yet another example in which the motion reference block p and the available block q are both blocks in the time direction. FIG. 12F shows a case where the block TE in the time direction is divided into a plurality of small blocks, and the block TE has a plurality of small blocks having motion information 18. When all the motion information 18 of the block TE is the same motion information 18 and is the same as the motion information 18 of the block TD, it is determined that the motion information 18 of the block TD and the TE is the same.

In this way, in step S803, it is determined whether or not the motion information 18 of the motion reference block p and the motion information 18 of the available block q are the same. In the examples of FIGS. 12A to 12F, the number of available blocks q to be compared with the motion reference block p has been described as 1, but when the number of available blocks q is 2 or more, the motion information of the motion reference block p has been described. 18 may be compared with the motion information 18 of each available block q. Further, when scaling described later is applied, the motion information 18 after scaling becomes the motion information 18 described above.

The determination that the motion information of the motion reference block p and the motion information of the available block q are the same is not limited to the case where the motion vectors included in the motion information completely match. For example, if the norm of the difference between the two motion vectors is within a predetermined range, the motion information of the motion reference block p and the motion information of the available block q may be regarded as substantially the same.

FIG. 13 shows a more detailed configuration of the prediction unit 101. As described above, the prediction unit 101 inputs the available block information 30, the reference motion information 19, and the reference image signal 17, and outputs the prediction image signal 11, the motion information 18, and the selected block information 31. As shown in FIG. 13, the motion information selection unit 118 includes a spatial direction motion information acquisition unit 110, a time direction motion information acquisition unit 111, and a motion information changeover switch 112.

The available block information 30 and the reference motion information 19 regarding the spatial direction motion reference block are input to the spatial direction motion information acquisition unit 110. The spatial direction motion information acquisition unit 110 outputs motion information 18A including motion information possessed by each available block located in the spatial direction and an index value of the available block. When the information shown in FIG. 11 is input as the available block information 30, the spatial direction motion information acquisition unit 110 generates two motion information outputs 18A, and each motion information output 18A is the available block and its use. The motion information 19 possessed by the possible block is included.

The available block information 30 and the reference motion information 19 regarding the time direction motion reference block are input to the time direction motion information acquisition unit 111. The time direction motion information acquisition unit 111 outputs the motion information 19 included in the available time direction motion reference block specified by the available block information 30 and the index value of the available block as motion information 18B. The time direction movement reference block is divided into a plurality of small pixel blocks, and each small pixel block has motion information 19. As shown in FIG. 14, the motion information 18B output by the time direction motion information acquisition unit 111 includes a group of motion information 19 included in each small pixel block in the available block. When the motion information 18B includes the group of the motion information 19, the motion compensation prediction can be executed for the coded block in units of small pixel blocks obtained by dividing the coded block. When the information shown in FIG. 11 is input as the available block information 30, the time direction motion information acquisition unit 111 generates two motion information outputs 18B, and each motion information output is the available block and this available block. Includes a group of motion information 19 possessed by the block.

The time direction motion information acquisition unit 111 obtains the average value or representative value of the motion vector included in the motion information 19 of each pixel small block, and outputs the average value or representative value of the motion vector as motion information 18B. It doesn't matter.

The motion information changeover switch 112 of FIG. 13 selects an appropriate available block as a selection block based on the motion information 18A and 18B output from the spatial direction motion information acquisition unit 110 and the temporal direction motion information acquisition unit 111. Then, the motion information 18 (or a group of motion information 18) corresponding to the selected block is output to the motion compensation unit 113. Further, the motion information changeover switch 112 outputs the selection block information 31 regarding the selection block. The selection block information 31 includes the name of the index p or the motion reference block, and is also simply referred to as selection information. The selection block information 31 is not limited to the names of the index p and the motion reference block, and may be any information as long as the position of the selection block can be specified.

The motion information changeover switch 112 selects, for example, the available block that minimizes the coding cost derived by the cost formula shown in the following formula 1 as the selection block.

Here, J indicates the coding cost, and D indicates the coding distortion representing the sum of squared errors between the input image signal 10 and the reference image signal 17. Also, R indicates the amount of code estimated by provisional coding, and λ is. The Lagrange undetermined coefficient determined by the quantization width and the like is shown. The coding cost J may be calculated by using only the coding amount R or the coding distortion D instead of the equation 1, and the value obtained by approximating the coding amount R or the coding distortion D may be used to calculate the coding cost J. You may create a cost function. Further, the coding distortion D is not limited to the sum of squared errors, and may be the sum of absolute differences (SAD) of the prediction errors. As the code amount R, only the code amount related to the motion information 18 may be used. Further, the example is not limited to the example in which the available block having the minimum coding cost is selected as the selection block, and one available block having a value within a certain range equal to or higher than the value having the lowest coding cost is selected as the selection block. It may be selected.

The motion compensation unit 113 derives the position of the pixel block from which the reference image signal 17 is taken out as the predicted image signal 11 based on the motion information (or the group of motion information) of the selection block selected by the motion information selection unit 118. do. When a group of motion information is input to the motion compensation unit 113, the motion compensation unit 113 divides the pixel block from which the reference image signal 17 is taken out as the predicted image signal 11 into small pixel blocks (for example, 4 × 4 pixel blocks). In addition, the predicted image signal 11 is acquired from the reference image signal 17 by applying the corresponding motion information 18 to each of these small pixel blocks. As shown in FIG. 4A, for example, the position of the block from which the predicted image signal 11 is acquired is a position shifted in the spatial direction from the small pixel block according to the motion vector 18a included in the motion information 18.

The motion compensation processing for the coded block is described in H.I. The same as the motion compensation process of 264 can be used. Here, as an example, an interpolation method with 1/4 pixel accuracy will be specifically described. In 1/4 pixel precision interpolation, if each component of the motion vector is a multiple of 4, the motion vector points to an integer pixel position. Otherwise, the motion vector points to the predicted position corresponding to the fractionally accurate interpolation position.

Here, x and y indicate vertical and horizontal indexes indicating the head position (for example, the upper left vertex) of the prediction target block, and x_pos and y_pos indicate the corresponding predicted positions of the reference image signal 17. (mv_x, mv_y) indicates a motion vector with 1/4 pixel accuracy. Next, the predicted pixel is generated by compensating or interpolating the corresponding pixel position of the reference image signal 17 with respect to the determined pixel position. In FIG. 15, H. An example of 264 predicted pixel generation is shown. In FIG. 15, the square indicated by the uppercase alphabet (square with a diagonal line) indicates the pixel at the integer position, and the square displayed in shading indicates the interpolated pixel at the 1/2 pixel position. There is. Further, the squares displayed in white indicate the interpolated pixels corresponding to the 1/4 pixel positions. For example, in FIG. 15, the interpolation process of 1/2 pixel corresponding to the positions of the alphabets b and h is calculated by the following mathematical formula 3.

Here, the alphabets (for example, b, h, C1, etc.) shown in the formula 3 and the following formula 4 indicate the pixel values of the pixels to which the same alphabet is given in FIG. Further, ">>" indicates a right shift operation, and ">> 5" corresponds to division by 32. That is, the interpolated pixel at the 1/2 pixel position is calculated using a 6-tap FIR (Finite Impulse Response) filter (tap coefficient: (1, -5, 20, 20, -5, 1) / 32).

Further, the interpolation process of 1/4 pixel corresponding to the positions of the alphabets a and d in FIG. 15 is calculated by the following mathematical formula 4.

In this way, the interpolated pixels at the 1/4 pixel position are calculated using a 2-tap average value filter (tap coefficient: (1/2, 1/2)). The 1/2 pixel interpolation process corresponding to the alphabet j existing in the middle of the four integer pixel positions is generated using both the vertical 6 taps and the horizontal 6 taps. Interpolated pixel values are generated in the same manner for pixel positions other than those described.

The interpolation process is not limited to the examples of Equation 3 and Equation 4, and may be generated by using other interpolation coefficients. Further, as the interpolation coefficient, a fixed value given by the coding control unit 150 may be used, or the interpolation coefficient is optimized for each frame based on the above-mentioned coding cost, and the optimized interpolation coefficient is obtained. It may be generated using.

Further, in the present embodiment, the processing related to the motion vector block prediction processing in which the motion reference block is in units of macroblocks (for example, 16 × 16 pixel blocks) has been described, but the processing is not limited to macroblocks, but in units of 16 × 8 pixel blocks. The prediction process may be executed in units of 8 × 16 pixel blocks, 8 × 8 pixel blocks, 8 × 4 pixel blocks, 4 × 8 pixel blocks, or 4 × 4 pixel blocks. In this case, the information about the motion vector block is derived in pixel block units. Further, the above prediction processing may be performed in units larger than 16 × 16 pixel blocks, such as 32 × 32 pixel block units, 32 × 16 pixel block units, and 64 × 64 pixel block units.

When substituting the reference motion vector in the motion vector block as the motion vector of the small pixel block in the coded block, (A) the negative value (inversion vector) of the reference motion vector may be substituted. (B) A weighted average value or a median value, a maximum value, and a minimum value using a reference motion vector corresponding to a small block and a reference motion vector adjacent to the reference motion vector may be substituted.

FIG. 16 schematically shows the operation of the prediction unit 101. As shown in FIG. 16, first, a reference frame (movement reference frame) including the time direction reference movement block is acquired (step S1501). The motion reference frame is typically a reference frame having the shortest time distance from the coded frame, and is a reference frame in the past in time. For example, the motion reference frame is a frame encoded immediately before the coded frame. In another example, any reference frame in which the motion information 18 is stored in the motion information memory 108 may be acquired as the motion reference frame. Next, the spatial direction motion information acquisition unit 110 and the temporal direction motion information acquisition unit 111 each acquire the available block information 30 output from the available block acquisition unit 109 (step S1502). Next, the motion information changeover switch 112 selects one of the available blocks as a selection block according to, for example, Equation 1 (step S1503). Subsequently, the motion compensation unit 113 copies the motion information of the selected selected block to the coded block (step S1504). At this time, when the selected block is a spatial direction reference block, the motion information 18 possessed by this selected block is copied to the coded reference block as shown in FIG. When the selected block is a time-direction reference block, the group of motion information 18 included in the selected block is copied to the coded target block together with the position information. Next, motion compensation is executed using the motion information 18 or the group of motion information 18 copied by the motion compensation unit 113, and the predicted image signal 11 and the motion information 18 used for the motion compensation prediction are output.

FIG. 18 shows a more detailed configuration of the variable length coding unit 104. As shown in FIG. 18, the variable length coding unit 104 includes a parameter coding unit 114, a conversion coefficient coding unit 115, a selection block coding unit 116, and a multiplexing unit 117. The parameter coding unit 114 encodes parameters required for decoding such as prediction mode information, block size information, and quantization parameter information, excluding conversion coefficient information 13 and selection block information 31, and generates coded data 14A. .. The conversion coefficient coding unit 115 encodes the conversion coefficient information 13 to generate the coded data 14B. Further, the selection block coding unit 116 encodes the selection block information 31 with reference to the available block information 30, and generates the coded data 14C.

As shown in FIG. 19, when the available block information 30 includes an index and the availability of the motion reference block corresponding to the index, the motion that is not available among the plurality of preset motion reference blocks. Exclude reference blocks and convert only available motion reference blocks to syntax (stds_idx). In FIG. 19, since 5 motion reference blocks out of 9 motion reference blocks are not available, the syntax stds_idx is assigned in order from 0 to the 4 motion reference blocks excluding these. In this example, the selected block information to be encoded is not selected from nine, but is selected from four available blocks, so that the code amount (bin number) to be allocated can be small on average.

FIG. 20 shows an example of a code table showing binary information (bin) of syntax stds_idx and syntax stds_idx. As shown in FIG. 18, the smaller the number of motion reference blocks available, the lower the average number of bins required to code the syntax stds_idx. For example, if the number of available blocks is 4, the syntax stds_idx can be represented by 3 bits or less. The binar information (bin) of the syntax stds_idx may be binarized so that all stds_idx have the same number of bins for each available block, and is binarized according to the binarization method determined by pre-learning. It doesn't matter. Further, a plurality of binarization methods are prepared, and may be appliedly switched for each coded block.

Entropy coding (for example, equal length coding, Huffman coding, arithmetic coding, etc.) can be applied to these coding units 114, 115, 116, and the generated coding data 14A, 14B, The 14C is multiplexed and output by the multiplexing unit 117.

In the present embodiment, an example in which a frame encoded one frame before the coded target frame is referred to as a reference frame is described, but the motion vector and the reference in the reference motion information 19 of the selected block are used. The motion vector may be scaled (or normalized) by using the frame number, and the reference motion information 19 may be applied to the coded block.

This scaling process will be specifically described with reference to FIG. The tc shown in FIG. 21 indicates the time distance (POC (number indicating the display order) distance) between the coded frame and the motion reference frame, and is calculated by the following mathematical formula 5. The tr [i] shown in FIG. 21 indicates the time distance between the motion reference frame and the frame i referenced by the selected block, and is calculated by the following mathematical formula 6.

Here, curPOC indicates the POC (Picture Order Count) of the frame to be encoded, colPOC indicates the POC of the motion reference frame, and refPOC indicates the POC of the frame i referenced by the selected block. Clip (min, max, target) outputs min when target is smaller than min, outputs max when it is larger than max, and outputs target in other cases. Clip function to do. DiffPicOrderCnt (x, y) is a function that calculates the difference between two POCs.

Assuming that the motion vector of the selected block is MVr = (MVr_x, MVr_y) and the motion vector applied to the coded block is MV = (MV_x, MV_y), the motion vector MV is calculated by the following formula 7.

Here, Abs (x) shows a function that extracts the absolute value of x. In this way, in the motion vector scaling, the motion vector MVr assigned to the selected block is converted into the motion vector MV between the coded frame and the motion first reference frame.

In addition, another example of scaling the motion vector will be described below.
First, for each slice or frame, the scaling coefficient (DistScaleFactor [i]) is obtained for all the time distance trs that the motion reference frame can take according to the following formula 8. The number of scaling factors is equal to the number of frames referenced by the selected block, i.e. the number of reference frames.

The calculation of tx shown in Equation 8 may be tabulated in advance.

When scaling each block to be encoded, the motion vector MV can be calculated only by multiplication, addition, and shift operation by using the following mathematical formula 9.

When such scaling processing is performed, the motion information 18 after scaling is applied to the processing of the available block acquisition unit 109 together with the prediction unit 101. When scaling processing is performed, the reference frame referenced by the coded block becomes a motion reference frame.

FIG. 22 shows the syntax structure in the image coding unit 100. As shown in FIG. 22, the syntax mainly includes three parts: high level syntax 901, slice level syntax 904 and macroblock level syntax 907. The high-level syntax 901 holds the syntax information of the upper layer above the slice. The slice level syntax 904 holds the information required for each slice, and the macroblock level syntax 907 holds the data required for each macroblock shown in FIGS. 7A to 7D.

Each part contains more detailed syntax. High-level syntax 901 includes sequence and picture-level syntax such as sequence parameter set syntax 902 and picture parameter set syntax 903. The slice level syntax 904 includes a slice header syntax 905, a slice data syntax 906, and the like. Further, the macroblock level syntax 907 includes a macroblock layer syntax 908, a macroblock prediction syntax 909, and the like.

23A and 23B show examples of macroblock layer syntax. Available_block_num shown in FIGS. 23A and 23B indicates the number of available blocks, and if this is a value greater than 1, the selected block information needs to be encoded. Further, stds_idx indicates the selected block information, and stds_idx is encoded using the above-mentioned code table according to the number of available blocks.

FIG. 23A shows the syntax when encoding the selected block information after mb_type. Encode stds_idx when the mode indicated by mb_type is the specified size or the specified mode (TARGET_MODE) and available_block_num is a value greater than 1. For example, the motion information of the selected block can be used when the block size is 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, or when the direct mode is used, stds_idx is encoded.

FIG. 23B shows the syntax when encoding the selected block information before mb_type. Encode stds_idx if available_block_num is greater than 1. If available_block_num is 0, then H. Since the conventional motion compensation represented by 264 is performed, the mb_type is encoded.

It is possible to insert a syntax element not specified in the present invention between the rows of the table shown in FIGS. 23A and 23B, and other descriptions regarding conditional branching may be included. .. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Further, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used. Further, each syntax element described in the macroblock layer syntax may be changed as specified in the macroblock data syntax described later.

In addition, it is possible to reduce the information of mb_type by using the information of stds_idx. FIG. 24A shows H. It is a code table corresponding to mb_type and mb_type at the time of B slice in 264. N shown in FIG. 24A is a value representing the size of the coded block such as 16, 32, 64, and M is a half value of N. Therefore, when mb_type is 4 to 21, it indicates that the coded block is a rectangular block. Further, L0, L1, and Bi in FIG. 24A show unidirectional prediction (List0 direction only), unidirectional prediction (List1 direction only), and bidirectional prediction, respectively. When the coded block is a rectangular block, mb_type contains information indicating which of L0, L1, and Bi is predicted for each of the two rectangular blocks in the coded block. Further, B_Sub means that the above processing is performed for each of the pixel blocks obtained by dividing the macro block into four. For example, when the coded block is a 64 × 64 pixel macroblock, the coded block is further assigned mb_type for each of the four 32 × 32 pixel blocks obtained by dividing the macroblock into four blocks. Be transformed.

Here, when the selection block indicated by stds_idx is Spatial Left (pixel block adjacent to the left side of the coded block), the motion information of the pixel block adjacent to the left side of the coded block is the motion information of the coded block. Since it is motion information, stds_idx is predicted for the block to be encoded using the horizontally long rectangular block shown by mb_type = 4,6,8,10,12,14,16,18,20 in FIG. 24A. Has the same meaning as executing. Further, when the selection block indicated by stds_idx is Spatial Up, the motion information adjacent to the upper side of the coded block is used as the motion information of the coded block. Therefore, stds_idx is mb_type = 5,7, in FIG. 24A. It has the same meaning as executing the prediction in the vertically long rectangular block shown by 9,11,13,15,17,19,21. Therefore, by using stds_idx, it is possible to create a code table in which the columns of mb_type = 4 to 21 in FIG. 24A as shown in FIG. 24B are reduced. Similarly, H. Regarding the code table corresponding to the mb_type and mb_type at the time of P-slicing in 264, it is possible to create a code table in which the number of mb_types is reduced as shown in FIG. 24D.

Further, the stds_idx information may be included in the mb_type information and encoded. FIG. 25A is a code table when the information of stds_idx is included in the information of mb_type, and shows an example of the code table corresponding to mb_type and mb_type in the B slice. B_STDS_X (X = 0,1,2) in FIG. 25A indicates a mode corresponding to stds_idx, and B_STDS_X is added by the number of available blocks (in FIG. 25A, the number of available blocks is 3). Similarly, another example of mb_type relating to P slice is shown in FIG. 25B. The description of FIG. 25B is omitted because it is the same as the B slice.

The order and binarization method of mb_type is not limited to the examples shown in FIGS. 25A and 25B, and mb_type may be encoded according to other order and binarization method. B_STDS_X and P_STDS_X do not have to be contiguous and may be placed between each mb_type. Further, the binarization method (binization) may be designed based on the selection frequency learned in advance.

In the present embodiment, the present invention is also applicable to an extended macroblock in which a plurality of macroblocks are grouped together to perform motion compensation prediction. Further, in the present embodiment, the coding scan order may be any order. For example, the present invention is also applicable to line scans, Z scans, and the like.

As described above, the image coding apparatus according to the present embodiment selects an available block from a plurality of motion reference blocks, and applies the motion reference block to the coded target block according to the number of the selected available blocks. Information is generated to identify the information, and this information is encoded. Therefore, according to the image coding apparatus according to the present embodiment, motion compensation can be performed in units of small pixel blocks finer than the coding target block while reducing the amount of coding related to motion vector information, so that high coding efficiency is achieved. Can be realized.

(Second Embodiment)
FIG. 26 shows an image coding apparatus according to a second embodiment of the present invention. In the second embodiment, parts and operations different from those in the first embodiment will be mainly described. As shown in FIG. 26, the image coding unit 200 according to the present embodiment has a different configuration of the prediction unit 201 and the variable length coding unit 204 from the first embodiment. As shown in FIG. 27, the prediction unit 201 includes a first prediction unit 101 and a second prediction unit 202, and selectively switches between the first and second prediction units 101 and 202 to generate the prediction image signal 11. .. The first prediction unit 101 has the same configuration as the prediction unit 101 (FIG. 1) according to the first embodiment, and follows a prediction method (first prediction method) for motion compensation using motion information 18 possessed by the selected block. , Generates the predicted image signal 11. The second prediction unit 202 uses one motion vector to compensate for the motion of the coded block. The predicted image signal 11 is generated according to a prediction method (second prediction method) such as 264. The second prediction unit 202 uses the input image signal 10 and the reference image signal 17 from the frame memory to generate the prediction image signal 11B.

FIG. 28 schematically shows the configuration of the second prediction unit 202. As shown in FIG. 28, the second prediction unit 202 includes a motion information acquisition unit 205 that generates motion information 21 using the input image signal 10 and the reference image signal 17, and the reference image signal 17 and the motion information 21. The motion compensating unit 113 (FIG. 1) is provided to generate the predicted image signal 11A by using the above. Based on the input image signal 10 and the reference image signal 17, the motion information acquisition unit 205 obtains a motion vector to be assigned to the coded block by, for example, block matching. As the matching evaluation standard, a value obtained by accumulating the difference between the input image signal 10 and the interpolated image after matching is used for each pixel.

The motion information acquisition unit 205 may determine the optimum motion vector by using the value obtained by converting the difference between the predicted image signal 11 and the input image signal 10. Further, the optimum motion vector may be determined in consideration of the magnitude of the motion vector and the sign amount of the motion vector and the reference frame number, or by using Equation 1. The matching method may be executed based on the search range information provided from the outside of the image coding apparatus, or may be executed hierarchically for each pixel accuracy. Further, the motion information given by the coding control unit 150 may be used as the output 21 of the motion information acquisition unit 205 without performing the search process.

The prediction unit 101 of FIG. 27 further includes a prediction method changeover switch 203 that selects and outputs either the prediction image signal 11A from the first prediction unit 101 or the prediction image signal 11B from the second prediction unit 202. There is. For example, the prediction method changeover switch 203 uses the input image signal 10 for each of the prediction image signals 11A and 11B to obtain the coding cost according to, for example, Equation 1, and the prediction image so that the coding cost becomes smaller. Either one of the signals 11A and 11B is selected and output as the predicted image signal 11. Further, the prediction method changeover switch 203 indicates whether the output prediction image signal 11 is output from either the first prediction unit 101 or the second prediction unit 202 together with the motion information 18 and the selection block information 31. Information 32 is also output. The output motion information 18 is encoded by the variable length coding unit 204 and then multiplexed with the coded data 14.

FIG. 29 schematically shows the configuration of the variable length coding unit 204. The variable-length coding unit 204 shown in FIG. 29 includes a motion information coding unit 217 in addition to the configuration of the variable-length coding unit 104 shown in FIG. Further, unlike the selection block coding unit 116 of FIG. 18, the selection block coding unit 216 of FIG. 29 encodes the prediction switching information 32 to generate the coded data 14D. When the first prediction unit 101 executes the prediction process, the selection block coding unit 216 further encodes the available block information 30 and the selection block information 31. The coded available block information 30 and the selected block information 31 are included in the coded data 14D. When the second prediction unit 202 executes the prediction process, the motion information coding unit 217 encodes the motion information 18 and generates the encoded data 14E. The selection block coding unit 216 and the motion information coding unit 217 each perform prediction processing based on the prediction switching information 32 indicating whether or not the prediction image is generated by the motion compensation prediction using the motion information of the selection block. 1 It is determined which of the prediction unit 101 and the second prediction unit 202 has been executed.

The multiplexing unit 117 receives the coded data 14A, B, D, and E from the parameter coding unit 114, the conversion coefficient coding unit 115, the selection block coding unit 216, and the motion information coding unit, and receives the coded data. 14 A, B, D, E are multiplexed.

30A and 30B show examples of macroblock layer syntax according to the present embodiment, respectively. The available_block_num shown in FIG. 30A indicates the number of available blocks, and when this is a value greater than 1, the selection block encoding unit 216 encodes the selection block information 31. Further, the stds_flag is a flag indicating whether or not the motion information of the selected block is used as the motion information of the coded block in the motion compensation prediction, that is, the prediction method changeover switch 203 is the first prediction unit 101 and the second prediction unit. It is a flag indicating which of 202 is selected. When the number of available blocks is larger than 1 and the stds_flag is 1, it indicates that the motion information of the selected block is used for the motion compensation prediction. Further, when stds_flag is 0, H.H. Similar to 264, the difference value obtained by directly or predicting the information of the motion information 18 is encoded. Further, stds_idx indicates the selected block information, and the code table according to the number of available blocks is as described above.

FIG. 30A shows the syntax when encoding the selected block information after mb_type. Encode stds_flag and stds_idx only if the mode indicated by mb_type is the specified size or the specified mode. For example, the motion information of the selected block can be used when the block sizes are 64 × 64, 32 × 32, 16 × 16, or in the direct mode, stds_flag and stds_idx are encoded.

FIG. 30B shows the syntax when encoding the selected block information before mb_type. For example, if stds_flag is 1, mb_type does not need to be encoded. If stds_flag is 0, mb_type is encoded.

As described above, the image coding apparatus according to the second embodiment uses the first prediction unit 101 according to the first embodiment and a prediction method such as H.264 so that the coding cost is reduced. The input image signal is compressed and encoded by selectively switching between the second prediction unit 202 and the second prediction unit 202. Therefore, in the image coding apparatus according to the second embodiment, the coding efficiency is improved as compared with the image coding apparatus of the first embodiment.

(Third Embodiment)
FIG. 31 schematically shows an image decoding apparatus according to a third embodiment. As shown in FIG. 31, this image decoding device includes an image decoding unit 300, a decoding control unit 350, and an output buffer 308. The image decoding unit 300 is controlled by the decoding control unit 350. The image decoding device according to the third embodiment corresponds to the image coding device according to the first embodiment. That is, the decoding process by the image decoding device of FIG. 31 has a complementary relationship with the coding process of the image coding process of FIG. The image decoding device of FIG. 31 may be realized by hardware such as an LSI chip, or may be realized by causing a computer to execute an image decoding program.

The image decoding device of FIG. 31 includes a coded sequence decoding unit 301, an inverse quantization / inverse conversion unit 302, an adder 303, a frame memory 304, a prediction unit 305, a motion information memory 306, and an available block acquisition unit 307. I have. In the image decoding unit 300, the coded data 80 from the storage system or the transmission system (not shown) is input to the coded sequence decoding unit 301. The coded data 80 corresponds to, for example, the coded data 14 transmitted in a multiplexed state from the image coding device of FIG.

In the present embodiment, the pixel block (for example, macroblock) to be decoded is simply referred to as the decoding target block. Further, the image frame including the decoding target block is referred to as a decoding target frame.

In the coded sequence decoding unit 301, decoding by parsing is performed for each frame or field based on the syntax. Specifically, the coded sequence decoding unit 301 sequentially performs variable length decoding of the coded strings of each syntax, and obtains prediction information such as conversion coefficient information 33, selection block information 61, and block size information and prediction mode information. Decodes the coding parameters related to the decryption target block, including.

In the present embodiment, the decoding parameter includes a conversion coefficient 33, selection block information 61, and prediction information, and includes all parameters required for decoding such as information on the conversion coefficient and information on quantization. Prediction information, information on conversion coefficients, and information on quantization are input to the decoding control unit 350 as control information 71. The decoding control unit 350 provides each unit of the image decoding unit 300 with decoding control information 70 including prediction information and parameters necessary for decoding such as quantization parameters.

Further, the coded sequence decoding unit 301 simultaneously decodes the coded data 80 to obtain the prediction information and the selected block information 61, as will be described later. The motion information 38 including the motion vector and the reference frame number does not have to be decoded.

The conversion coefficient 33 decoded by the coded sequence decoding unit 301 is sent to the inverse quantization / inverse conversion unit 302. Various information about the quantization decoded by the coded sequence decoding unit 301, that is, the quantization parameter and the quantization matrix are given to the decoding control unit 350, and are dequantized and inversely converted at the time of dequantization. It is loaded into unit 302. The inverse quantization / inverse transform unit 302 inversely quantizes the conversion coefficient 33 according to the loaded information regarding the quantization, and then performs an inverse transform process (for example, inverse discrete cosine transform) to obtain the prediction error signal 34. obtain. The inverse conversion process by the inverse quantization / inverse conversion unit 302 of FIG. 31 is the inverse transformation of the conversion process by the conversion / quantization unit of FIG. For example, when the wavelet transform is performed by the image coding apparatus (FIG. 1), the inverse quantization / inverse transform unit 302 executes the corresponding inverse quantization and inverse wavelet transform.

The prediction error signal 34 restored by the inverse quantization / inverse transformation unit 302 is input to the adder 303. The adder 303 adds the prediction error signal 34 and the prediction image signal 35 generated by the prediction unit 305 described later to generate the decoded image signal 36. The generated decoded image signal 36 is output from the image decoding unit 300, temporarily stored in the output buffer 308, and then output according to the output timing managed by the decoding control unit 350. Further, the decoded image signal 36 is stored in the frame memory 304 as a reference image signal 37. The reference image signal 37 is sequentially read from the frame memory 304 frame by frame or field by field, and is input to the prediction unit 305.

The usable block acquisition unit 307 receives the reference motion information 39 from the motion information memory 306 described later, and outputs the available block information 60. The operation of the available block acquisition unit 307 is the same as that of the available block acquisition unit 109 (FIG. 1) described in the first embodiment.

The motion information memory 306 receives motion information 38 from the prediction unit 305 and temporarily stores it as reference motion information 39. The motion information memory 306 temporarily stores the motion information 38 output from the prediction unit 305 as the reference motion information 39. FIG. 4 shows an example of the motion information memory 306. The motion information memory 306 holds a plurality of motion information frames 26 having different coding times. The motion information 38 or the group of motion information 38 for which decoding has been completed is stored as reference motion information 39 in the motion information frame 26 corresponding to the decoding time. In the motion information frame 26, the reference motion information 39 is stored, for example, in units of 4 × 4 pixel blocks. The reference motion information 39 held by the motion information memory 306 is read and referenced by the prediction unit 305 when generating the motion information 38 of the decoding target block.

Next, the motion reference block and the available block according to the present embodiment will be described. The motion reference block is a candidate block selected from the already decoded region according to a method predetermined by the image coding device and the image decoding device described above. FIG. 8A shows an example of the available blocks. In FIG. 8A, a total of nine motion reference blocks are arranged, including four motion reference blocks in the decoding target frame and five motion reference blocks in the reference frame. The motion reference blocks A, B, C, and D in the decoding target frame of FIG. 8A are blocks adjacent to the left, top, upper right, and upper left of the decoding target block. In the present embodiment, the motion reference block selected from the decoding target frames including the decoding target block is referred to as a spatial direction motion reference block. Further, the motion reference block TA in the reference frame is a pixel block in the reference frame at the same position as the decoding target block, and the pixel blocks TB, TC, TD, and TE in contact with the motion reference block TA move. Selected as a reference block. The motion reference block selected from the pixel blocks in the reference frame is referred to as a motion reference block in the time direction. Further, the frame in which the time direction movement reference block is located is referred to as a movement reference frame.

The spatial movement reference block is not limited to the example shown in FIG. 8A, and as shown in FIG. 8B, the pixel block to which the pixels a, b, c, and d adjacent to the decoding target block belong is used as the spatial movement reference block. It may be selected. In this case, the relative positions (dx, dy) of the pixels a, b, c, and d with respect to the upper left pixel in the decoding target block are shown in FIG. 8C.

Further, as shown in FIG. 8D, all the pixel blocks A1 to A4, B1, B2, C, and D adjacent to the decoding target block may be selected as the spatial movement reference blocks. In FIG. 8D, the number of spatial movement reference blocks is eight.

Further, the time direction movement reference blocks TA to TE may partially overlap each other as shown in FIG. 8E, or may be separated from each other as shown in FIG. 8F. Further, the time direction motion reference block does not necessarily have to be located in the block at the Collocation position and its surroundings, and may be a pixel block at any position within the motion reference frame. For example, using the motion information of a block that has already been decoded adjacent to the block to be decoded, the reference block pointed to by the motion vector included in the motion information is selected as the center of the motion reference block (for example, block TA). It doesn't matter. Further, the reference blocks in the time direction do not have to be arranged at equal intervals.

In the method of selecting the motion reference block as described above, if both the image decoding device and the image decoding device share information on the number and position of the motion reference blocks in the spatial direction and the temporal direction, the motion reference block Can be selected from any number and position. Further, the size of the motion reference block does not necessarily have to be the same size as the decoding target block. For example, as shown in FIG. 8D, the size of the motion reference block may be larger, smaller, or arbitrary size than the size of the decoding target block. Further, the shape of the motion reference block is not limited to a square shape, and may be a rectangular shape.

Next, the available blocks will be described. The usable block is a pixel block selected from the motion reference blocks, and is a pixel block to which motion information can be applied to the decoding target block. The available blocks have different motion information from each other. The available blocks are selected, for example, by executing the available block determination process shown in FIG. 9 for a total of nine motion reference blocks in the decoding target frame and the reference frame as shown in FIG. 8A. .. FIG. 10 shows the result of executing the available block determination process shown in FIG. In FIG. 10, shaded pixel blocks indicate unusable blocks, and white-painted blocks indicate usable blocks. That is, two of the spatial direction movement reference blocks and two of the temporal direction movement reference blocks, a total of four, are determined as usable blocks. The motion information selection unit 314 in the prediction unit 305 can use the optimum one of these available blocks arranged in the temporal and spatial directions according to the selection block information 61 received from the selection block decoding unit 323. Select a block as the selection block.

Next, the usable block acquisition unit 307 will be described. The available block acquisition unit 307 has the same function as the available block acquisition unit 109 of the first embodiment, acquires reference motion information 39 from the motion information memory 306, and uses the available blocks or uses for each motion reference block. The available block information 60, which is information indicating an impossible block, is output.

The operation of the usable block acquisition unit 307 will be described with reference to the flowchart of FIG. First, the available block acquisition unit 307 determines whether or not the motion reference block (index p) has motion information (step S801). That is, in step S801, it is determined whether or not at least one small pixel block in the motion reference block p has motion information. When it is determined that the motion reference block p does not have motion information, that is, the temporal motion reference block is a block in the I slice without motion information, or all the small blocks in the temporal motion reference block. If the pixel block is intra-predictively decoded, the process proceeds to step S805. In step S805, the motion reference block p is determined to be an unusable block.

When it is determined in step S801 that the motion reference block p has motion information, the available block acquisition unit 307 selects a motion reference block q (referred to as an available block q) that has already been determined to be an available block. (Step S802). Here, q is a value smaller than p. Subsequently, the available block acquisition unit 307 compares the motion information of the motion reference block p with the motion information of the available block q for all q, and the motion reference block p is the available block q. It is determined whether or not the motion information is the same as that of (S803). If the motion reference block p has the same motion vector as the available block q, the process proceeds to step S805, and in step S805, the motion reference block p is referred to as an unavailable block by the available block acquisition unit 307. It is judged. When the motion reference block p has motion information different from that of all the available blocks q, in step S804, the motion reference block p is determined to be an available block by the available block acquisition unit 307.

By executing the above-mentioned available block determination process for all motion reference blocks, it is determined whether the usable block or the unavailable block is available for each motion reference block, and the available block information 60 is generated. An example of the available block information 60 is shown in FIG. The available block information 60 includes the index p and availability of the motion reference block, as shown in FIG. In FIG. 11, the available block information 60 indicates that the motion reference block having the index p of 0, 1, 5 and 8 is selected as the available block, and the number of available blocks is 4.

In step S801 of FIG. 9, when at least one of the blocks in the time direction motion reference block p is an intra-prediction encoded block, the available block acquisition unit 307 uses the motion reference block p. It may be determined as an impossible block. That is, the process may proceed to step S802 only when all the blocks in the time direction movement reference block p are encoded by the inter-prediction.

12A to 12E show an example in which the motion information 38 of the motion reference block p and the motion information 38 of the available block q are determined to be the same in the comparison of the motion information 38 in step S803. 12A to 12E show a plurality of shaded blocks and two whitewashed blocks, respectively. In FIGS. 12A to 12E, for the sake of simplicity, it is assumed that the motion information 38 of these two white-painted blocks is compared without considering the shaded blocks. It is assumed that one of the two whitewashed blocks is the motion reference block p and the other is the motion reference block q (available block q) that has already been determined to be available. Unless otherwise specified, either of the two white blocks may be the motion reference block p.

FIG. 12A shows an example in which both the motion reference block p and the available block q are spatially oriented blocks. In the example of FIG. 12A, if the motion information 38 of the blocks A and B is the same, it is determined that the motion information 38 is the same. At this time, the sizes of blocks A and B do not have to be the same.

FIG. 12B shows an example in which one of the motion reference block p and the available block q is the block A in the spatial direction and the other is the block TB in the time direction. In FIG. 12B, there is one block having motion information in the block TB in the time direction. If the motion information 38 of the block TB in the time direction and the motion information 38 of the block A in the spatial direction are the same, it is determined that the motion information 38 is the same. At this time, the sizes of blocks A and TB do not have to be the same.

FIG. 12C shows another example in which one of the motion reference block p and the available block q is the block A in the spatial direction and the other is the block TB in the time direction. FIG. 12C shows a case where the block TB in the time direction is divided into a plurality of small blocks, and there are a plurality of small blocks having motion information 38. In the example of FIG. 12C, if all the blocks having the motion information 38 have the same motion information 38 and the motion information 38 is the same as the motion information 38 of the block A in the spatial direction, the motion information 38 is the same. Is determined. At this time, the sizes of blocks A and TB do not have to be the same.

FIG. 12D shows an example in which both the motion reference block p and the available block q are blocks in the time direction. In this case, if the motion information 38 of the blocks TB and TE is the same, it is determined that the motion information 38 is the same.

FIG. 12E shows another example in which the motion reference block p and the available block q are both blocks in the time direction. FIG. 12E shows a case where the blocks TB and TE in the time direction are each divided into a plurality of small blocks, and each of them has a plurality of small blocks having motion information 38. In this case, the motion information 38 is compared for each small block in the block, and if the motion information 38 is the same for all the small blocks, the motion information 38 of the block TB and the motion information 38 of the block TE are the same. It is determined that there is.

FIG. 12F shows yet another example in which the motion reference block p and the available block q are both blocks in the time direction. FIG. 12F shows a case where the block TE in the time direction is divided into a plurality of small blocks, and the block TE has a plurality of small blocks having motion information 38. When all the motion information 38 of the block TE is the same motion information 38 and is the same as the motion information 38 of the block TD, it is determined that the motion information 38 of the block TD and the TE is the same.

In this way, in step S803, it is determined whether or not the motion information 38 of the motion reference block p and the motion information 38 of the available block q are the same. In the examples of FIGS. 12A to 12F, the number of available blocks q to be compared with the motion reference block p has been described as 1, but when the number of available blocks q is 2 or more, the motion information of the motion reference block p has been described. 38 may be compared with the motion information 38 of each available block q. Further, when scaling described later is applied, the motion information 38 after scaling becomes the motion information 38 described above.

The determination that the motion information of the motion reference block p and the motion information of the available block q are the same is not limited to the case where the motion vectors included in the motion information completely match. For example, if the norm of the difference between the two motion vectors is within a predetermined range, the motion information of the motion reference block p and the motion information of the available block q may be regarded as substantially the same.

FIG. 32 is a block diagram showing the coded sequence decoding unit 301 in more detail. As shown in FIG. 32, the coded sequence decoding unit 301 decodes the separation unit 320 that separates the coded data 80 into syntax units, the conversion coefficient decoding unit 322 that decodes the conversion coefficient, and the selected block information. It includes a selection block decoding unit 323 for decoding, and a parameter decoding unit 321 for decoding parameters related to the predicted block size and quantization.

The parameter decoding unit 321 receives the coded data 80A including the parameters related to the predicted block size and the quantization from the separation unit, decodes the coded data 80A, and generates the control information 71. The conversion coefficient decoding unit 322 receives the encoded conversion coefficient 80B from the separation unit 320, decodes the encoded conversion coefficient 80B, and obtains the conversion coefficient information 33. The selection block decoding unit 323 inputs the coded data 80C and the available block information 60 related to the selection block, and outputs the selection block information 61. The input available block information 60 indicates the availability of each motion reference block, as shown in FIG.

Next, the prediction unit 305 will be described in detail with reference to FIG. 33.
As shown in FIG. 33, the prediction unit 305 includes a motion information selection unit 314 and a motion compensation unit 313, and the motion information selection unit 314 includes a spatial direction motion information acquisition unit 310, a time direction motion information acquisition unit 311 and motion information. A changeover switch 312 is provided. The prediction unit 305 basically has the same configuration and function as the prediction unit 101 described in the first embodiment.

The prediction unit 305 takes the available block information 60, the selected block information 61, the reference motion information 39, and the reference image signal 37 as inputs, and outputs the prediction image signal 35 and the motion information 38. The spatial direction motion information acquisition unit 310 and the time direction motion information acquisition unit 311 have the same functions as the spatial direction motion information acquisition unit 110 and the time direction motion information acquisition unit 111, respectively, described in the first embodiment. The spatial direction motion information acquisition unit 310 uses the available block information 60 and the reference motion information 39 to generate motion information 38A including motion information and indexes of each available block located in the spatial direction. The time direction motion information acquisition unit 311 uses the available block information 60 and the reference motion information 39 to move information (or a group of motion information) including motion information and indexes of each available block located in the time direction. Generate 38B.

In the motion information changeover switch 312, 1 from the motion information 38A from the spatial direction motion information acquisition unit 310 and the motion information (or motion information group) 38B from the time direction motion information acquisition unit 311 according to the selection block information 61. Select one to obtain motion information 38. The selected motion information 38 is sent to the motion compensation unit 313 and the motion information memory 306. The motion compensation unit 313 performs motion compensation prediction in the same manner as the motion compensation unit 113 described in the first embodiment according to the selected motion information 38, and generates a predicted image signal 35.

Since the motion vector scaling function of the motion compensation unit 313 is the same as that described in the first embodiment, the description thereof will be omitted.

FIG. 22 shows the syntax structure of the image decoding unit 300. As shown in FIG. 22, the syntax mainly includes three parts: high level syntax 901, slice level syntax 904 and macroblock level syntax 907. The high-level syntax 901 holds the syntax information of the upper layer above the slice. The slice level syntax 904 holds the information required for each slice, and the macroblock level syntax 907 holds the data required for each macroblock shown in FIGS. 7A to 7D.

Each part contains more detailed syntax. High-level syntax 901 includes sequence and picture-level syntax such as sequence parameter set syntax 902 and picture parameter set syntax 903. The slice level syntax 904 includes a slice header syntax 905, a slice data syntax 906, and the like. Further, the macroblock level syntax 907 includes a macroblock layer syntax 908, a macroblock prediction syntax 909, and the like.

23A and 23B show examples of macroblock layer syntax. Available_block_num shown in FIGS. 23A and 23B indicates the number of available blocks, and if this is a value greater than 1, decoding of the selected block information is required. Further, stds_idx indicates the selected block information, and stds_idx is encoded using the above-mentioned code table according to the number of available blocks.

FIG. 23A shows the syntax when decoding the selected block information after mb_type. Decrypt stds_idx when the prediction mode indicated by mb_type is a specified size or a specified mode (TARGET_MODE) and available_block_num is a value greater than 1. For example, the motion information of the selected block can be used when the block size is 64 × 64 pixels, 32 × 32 pixels, 16 × 16 pixels, or when the direct mode is used, stds_idx is encoded.

FIG. 23B shows the syntax when decoding the selected block information before mb_type. Decrypt stds_idx if available_block_num is greater than 1. If available_block_num is 0, then H. Since the conventional motion compensation represented by 264 is performed, the mb_type is encoded.

It is possible to insert a syntax element not specified in the present invention between the rows of the table shown in FIGS. 23A and 23B, and other descriptions regarding conditional branching may be included. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Further, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used. Further, each syntax element described in the macroblock layer syntax may be changed as specified in the macroblock data syntax described later.

As described above, the image decoding apparatus according to the present embodiment decodes the image encoded by the image coding apparatus according to the first embodiment described above. Therefore, the image decoding according to the present embodiment can reproduce a high-quality decoded image from relatively small coded data.

(Fourth Embodiment)
FIG. 34 schematically shows an image decoding apparatus according to a fourth embodiment. As shown in FIG. 34, the image decoding device includes an image decoding unit 400, a decoding control unit 350, and an output buffer 308. The image decoding device according to the fourth embodiment corresponds to the image coding device according to the second embodiment. In the fourth embodiment, parts and operations different from those in the third embodiment will be mainly described. As shown in FIG. 34, the image decoding unit 400 according to the present embodiment is different from the third embodiment in the coded sequence decoding unit 401 and the prediction unit 405.

The prediction unit 405 of the present embodiment includes a prediction method (first prediction method) that compensates for movement using the movement information of the selected block, and H.K. The predicted image signal 35 is generated by selectively switching between a prediction method (second prediction method) for motion compensation using one motion vector for a block to be decoded, such as 264.

FIG. 35 is a block diagram showing the coded sequence decoding unit 401 in more detail. The coded sequence decoding unit 401 shown in FIG. 35 includes a motion information decoding unit 424 in addition to the configuration of the coded sequence decoding unit 301 shown in FIG. 32. Further, unlike the selection block decoding unit 323 shown in FIG. 32, the selection block decoding unit 423 shown in FIG. 35 decodes the coded data 80C relating to the selection block to obtain the prediction switching information 62. The prediction switching information 62 indicates whether the prediction unit 101 in the image coding apparatus of FIG. 1 used the first or second prediction method. When the prediction switching information 62 indicates that the prediction unit 101 has used the first prediction method, that is, when the decoding target block is encoded by the first prediction method, the selection block decoding unit 423 encodes the blocks. The selected block information 61 in the data 80C is decoded to obtain the selected block information 61. When the prediction switching information 62 indicates that the prediction unit 101 has used the second prediction method, that is, when the decoding target block is encoded by the second prediction method, the selection block decoding unit 423 has the selection block information. The motion information decoding unit 424 decodes the encoded motion information 80D without decoding the motion information 80D, and obtains the motion information 40.

FIG. 36 is a block diagram showing the prediction unit 405 in more detail. The prediction unit 405 shown in FIG. 34 includes a first prediction unit 305, a second prediction unit 410, and a prediction method changeover switch 411. The second prediction unit 410 uses the motion information 40 and the reference image signal 37 decoded by the coded sequence decoding unit 401 to perform motion compensation prediction similar to the motion compensation unit 313 of FIG. 33, and predicts the motion compensation image signal. Generate 35B. The first prediction unit 305 is the same as the prediction unit 305 described in the third embodiment, and generates the prediction image signal 35B. Further, the prediction method changeover switch 411 selects either one of the prediction image signal 35B from the second prediction unit 410 and the prediction image signal 35A from the first prediction unit 305 based on the prediction changeover information 62. It is output as a predicted image signal 35 of the prediction unit 405. At the same time, the prediction method changeover switch 411 sends the motion information used by the selected first prediction unit 305 or the second prediction unit 410 to the motion information memory 306 as motion information 38.

Next, the syntax structure of the present embodiment will be mainly described as being different from the third embodiment.

30A and 30B show examples of macroblock layer syntax according to the present embodiment, respectively. The available_block_num shown in FIG. 30A indicates the number of available blocks, and when this is a value greater than 1, the selected block decoding unit 423 decodes the selected block information in the coded data 80C. Further, the stds_flag is a flag indicating whether or not the motion information of the selected block is used as the motion information of the decoding target block in the motion compensation prediction, that is, the prediction method changeover switch 411 is the first prediction unit 305 and the second prediction unit. It is a flag indicating which of 410 is selected. When the number of available blocks is larger than 1 and the stds_flag is 1, it indicates that the motion information of the selected block is used for the motion compensation prediction. Further, when stds_flag is 0, H.H. Similar to 264, the difference value obtained by directly or predicting the motion information is encoded. Further, stds_idx indicates the selected block information, and the code table according to the number of available blocks is as described above.

FIG. 30A shows the syntax when decoding the selected block information after mb_type. Decrypt stds_flag and stds_idx only when the prediction mode indicated by mb_type is the specified block size or the specified mode. For example, when the block size is 64 × 64, 32 × 32, 16 × 16, or in the direct mode, stds_flag and stds_idx are decoded.

FIG. 30B shows the syntax when decoding the selected block information before mb_type. For example, if stds_flag is 1, mb_type does not need to be decrypted. If stds_flag is 0, mb_type is decrypted.

As described above, the image decoding apparatus according to the present embodiment decodes the image encoded by the image coding apparatus according to the second embodiment described above. Therefore, the image decoding according to the present embodiment can reproduce a high-quality decoded image from relatively small coded data.

The present invention is not limited to the above-described embodiment as it is, and at the implementation stage, the components can be modified and embodied within a range that does not deviate from the gist thereof. In addition, various inventions can be formed by an appropriate combination of the plurality of components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components over different embodiments may be combined as appropriate.

As an example of this, the same effect can be obtained by modifying the above-described first to fourth embodiments as follows.

(1) In the first to fourth embodiments, the processing target frame is divided into rectangular blocks such as a 16 × 16 pixel block, and the pixel block on the upper left of the screen as shown in FIG. 4 moves toward the pixel block on the lower right. Although the case of encoding or decoding in order is described as an example, the coding or decoding order is not limited to this example. For example, the coding or decoding order may be an order from the lower right to the upper left of the screen, or an order from the upper right to the lower left. Further, the coding or decoding order may be an order from the central portion of the screen to the peripheral portion in a spiral shape, or may be an order from the peripheral portion to the central portion of the screen.

(2) In the first to fourth embodiments, a case where the luminance signal and the color difference signal are not divided and limited to one color signal component is described as an example. However, different prediction processes may be used for the luminance signal and the color difference signal, or the same prediction process may be used. When different prediction processes are used, the prediction method selected for the color difference signal is encoded / decoded in the same manner as the luminance signal.

In addition, it goes without saying that it can be carried out in the same manner even if various prejudices are applied without departing from the gist of the present invention.

The image coding / decoding method according to the present invention has industrial applicability because the coding efficiency can be improved.

10 ... Input image signal, 11 ... Predicted image signal, 12 ... Prediction error image signal, 13 ... Quantization conversion coefficient, 14 ... Coded data, 15 ... Decoding prediction error signal, 16 ... Locally decoded image signal, 17 ... Reference image Signal, 18 ... motion information, 20 ... bit stream, 21 ... motion information, 25, 26 ... information frame, 30 ... available block information, 31 ... selected block information, 32 ... prediction switching information, 33 ... conversion coefficient information, 34 ... Prediction error signal, 35 ... Predicted image signal, 36 ... Decoded image signal, 37 ... Reference image signal, 38 ... Motion information, 39 ... Reference motion information, 40 ... Motion information, 50 ... Coding control information, 51 ... Feedback information , 60 ... Available block information, 61 ... Selected block information, 62 ... Prediction switching information, 70 ... Decoding control information, 71 ... Control information, 80 ... Coded data, 100 ... Image coding unit, 101 ... Prediction unit, 102 ... subtractor, 103 ... conversion / quantization unit, 104 ... variable length coding unit, 105 ... inverse quantization / inverse conversion unit, 106 ... adder, 107 ... frame memory, 108 ... information memory, 109 ... available Block acquisition unit, 110 ... Spatial direction movement information acquisition unit, 111 ... Time direction movement information acquisition unit, 112 ... Information changeover switch, 113 ... Motion compensation unit, 114 ... Parameter coding unit, 115 ... Conversion coefficient coding unit, 116 ... selection block coding unit, 117 ... multiplexing unit, 118 ... motion information selection unit, 120 ... output buffer, 150 ... coding control unit, 200 ... image coding unit, 201 ... prediction unit, 202 ... second prediction unit , 203 ... Prediction method changeover switch, 204 ... Variable length coding unit, 205 ... Motion information acquisition unit, 216 ... Selected block coding unit, 217 ... Motion information coding unit, 300 ... Image decoding unit, 301 ... Coding Column decoding unit, 301 ... Code sequence decoding unit, 302 ... Inverse quantization / inverse conversion unit, 303 ... Adder, 304 ... Frame memory, 305 ... Prediction unit, 306 ... Information memory, 307 ... Available block acquisition unit, 308 ... Output buffer, 310 ... Spatial direction motion information acquisition unit, 311 ... Time direction motion information acquisition unit 312 ... Motion information changeover switch, 313 ... Motion compensation unit, 314 ... Information selection unit, 320 ... Separation unit, 321 ... Parameters Decoding unit, 322 ... Conversion coefficient decoding unit, 323 ... Selected block decoding unit, 350 ... Decoding control unit, 400 ... Image decoding unit, 401 ... Coded string decoding unit, 405 ... Prediction unit, 410 ... 2nd prediction unit, 411 ... Prediction method changeover switch, 423 ... Selected block decoding unit, 424 ... Information decoding unit , 901 ... High level syntax, 902 ... Sequence parameter set syntax, 903 ... Picture parameter set syntax, 904 ... Slice level syntax, 905 ... Slice header syntax, 906 ... Slice data syntax, 907 ... Macroblock level syntax, 908 ... Macroblock Layer syntax, 909 ... Macroblock prediction syntax.

Claims (3)

A first coding unit that encodes mode information regarding the prediction mode of the target block,
Depending on the mode information indicating predetermined information, whether or not a plurality of candidate blocks having a predetermined positional relationship with respect to the target block can be used is determined in a predetermined order according to the positional relationship. Judgment unit to judge according to
A selection unit that selects a selection block from the candidate blocks determined to be available,
A second coding unit that encodes the identification information that identifies the selected block, and
A prediction unit that generates a prediction image of the target block based on the motion information corresponding to the selection block, and
With
The plurality of candidate blocks include a block adjacent to the upper left of the target block and a block adjacent to the target block.
The determination unit determines the block adjacent to the upper left of the target block after the determination of the block adjacent to the target block, and corresponds to the candidate block for which the candidate block is already determined to be available. An image encoding device that determines that the candidate block can be used when it has motion information that does not match the motion information. Encoding the mode information about the prediction mode of the target block and
Depending on the mode information indicating predetermined information, whether or not a plurality of candidate blocks having a predetermined positional relationship with respect to the target block can be used is determined in a predetermined order according to the positional relationship. Judging according to
To select a selection block from the candidate blocks determined to be available,
Encoding the identification information that identifies the selected block, and
To generate a predicted image of the target block based on the motion information corresponding to the selected block,
With
The plurality of candidate blocks include a block adjacent to the upper left of the target block and a block adjacent to the target block.
The determination corresponds to a candidate block for which the candidate block is already determined to be available by determining the block adjacent to the upper left of the target block after the determination of the block adjacent to the target block. An image coding method for determining that the candidate block can be used when it has motion information that does not match the motion information to be performed. A means of encoding mode information about the prediction mode of the target block,
Depending on the mode information indicating predetermined information, whether or not a plurality of candidate blocks having a predetermined positional relationship with respect to the target block can be used is determined in a predetermined order according to the positional relationship. Means to judge according to
A means for selecting a selected block from the candidate blocks determined to be available,
A program for operating a computer as a means for encoding identification information for identifying the selected block and as a means for generating a predicted image of the target block based on motion information corresponding to the selected block.
The plurality of candidate blocks include a block adjacent to the upper left of the target block and a block adjacent to the target block.
The determination means determines the block adjacent to the upper left of the target block after the determination of the block adjacent to the target block, and corresponds to the candidate block for which the candidate block is already determined to be available. A program that determines that the candidate block can be used when it has motion information that does not match the motion information to be performed.

Images