JP5893178B2 - Video encoding method and video decoding method - Google Patents

Description

  Embodiments described herein relate generally to a motion information compression method, a video encoding method, and a video decoding method in video encoding and decoding.

  In recent years, image coding methods that have greatly improved coding efficiency have been jointly developed by ITU-T and ISO / IEC in accordance with ITU-T Rec. H.264 and ISO / IEC 14496-10 (hereinafter referred to as H.264). Recommended). In H.264, prediction processing, conversion processing, and entropy encoding processing are performed in units of rectangular blocks (for example, 16 × 16 pixel block units, 8 × 8 pixel block units, etc.). In the prediction process, motion compensation is performed on a rectangular block to be encoded (encoding target block) with reference to an already encoded frame (reference frame) to perform prediction in the time direction. In such motion compensation, it is necessary to encode motion information including a motion vector as spatial shift information between an encoding target block and a block referred to in a reference frame and send the encoded motion information to the decoding side. Furthermore, when motion compensation is performed using a plurality of reference frames, it is necessary to encode a reference frame number together with motion information. For this reason, the amount of codes related to motion information and reference frame numbers may increase. In addition, there is a motion information prediction method for deriving predicted motion information of an encoding target block with reference to motion information stored in a motion information memory of a reference frame (Patent Document 1 and Non-Patent Document 2). May increase the capacity of the motion information memory for storing.

  As an example of a method for reducing the capacity of the motion information memory, (Non-Patent Document 2) derives representative motion information in a predetermined block and stores only the representative motion information in the motion information memory.

Patent No. 4020789 J. Jung et al, “Temporal MV predictor modification for MV-Comp, Skip, Direct and Merge schemes”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 Document, JCTVC-D164, January 20110. Yeping Su et al, “CE9: Reduced resolution storage of motion vector data”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 Document, JCTVC-D072, January 20110.

  However, when the method for deriving the predicted motion information shown in Non-Patent Document 1 is different from the method for deriving the representative motion information shown in Non-Patent Document 2, in order to reduce the temporal correlation of the predicted motion information, There is a problem that the amount is increased.

  The problem to be solved by the present invention is made to solve the above-mentioned problems. A moving picture encoding apparatus and a moving picture decoding apparatus including a motion information compression apparatus capable of improving encoding efficiency are provided. Is to provide.

  According to the embodiment, the moving image encoding method is a method of dividing an input image signal into pixel blocks and performing inter prediction on these divided pixel blocks. This method includes selecting predicted motion information from a motion information buffer that holds motion information in an encoded area, and predicting motion information of a block to be encoded using the predicted motion information. Further, this method obtains representative motion information according to first information indicating a method for selecting the predicted motion information from among a plurality of motion information in the region where encoding is completed, and obtains only the representative motion information. Including.

1 is a block diagram schematically showing the configuration of an image encoding device according to a first embodiment. Explanatory drawing of the prediction encoding order of a pixel block. Explanatory drawing of an example of pixel block size. Explanatory drawing of another example of pixel block size. Explanatory drawing of another example of pixel block size. Explanatory drawing of an example of the pixel block in a coding tree unit. Explanatory drawing of another example of the pixel block in a coding tree unit. Explanatory drawing of another example of the pixel block in a coding tree unit. Explanatory drawing of another example of the pixel block in a coding tree unit. The block diagram which shows schematically the structure of the entropy encoding part of FIG. FIG. 2 is an explanatory diagram schematically showing a configuration of a motion information memory in FIG. 1. Explanatory drawing of an example of the inter prediction process which the inter prediction part of FIG. 1 performs. Explanatory drawing of another example of the inter prediction process which the inter prediction part of FIG. 1 performs. Explanatory drawing of an example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing of another example of a prediction unit. Explanatory drawing which shows skip mode, merge mode, and inter mode. The block diagram which shows schematically the structure of the motion information encoding part of FIG. Explanatory drawing which shows the example of the position of the prediction motion information candidate with respect to the encoding target prediction unit. Explanatory drawing which shows another example of the position of a prediction motion information candidate with respect to the encoding target prediction unit. Explanatory drawing which shows the example of the list | wrist which shows the relationship between the block position of a some prediction motion information candidate, and index Mvpidx. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding prediction unit is 32x32. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding target prediction unit is 32x16. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding target prediction unit is 16x32. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding target prediction unit is 16x16. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding target prediction unit is 16x8. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding target prediction unit is 8x16. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit when the size of the encoding prediction unit is 32x32. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit when the size of the encoding prediction unit is 32x16. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding prediction unit is 16x32. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit in case the size of the encoding prediction unit is 16x16. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit when the size of the encoding prediction unit is 16x8. Explanatory drawing which shows another example of the reference motion information acquisition position which shows the center of a prediction unit when the size of the encoding prediction unit is 8x16. Explanatory drawing regarding the spatial direction reference motion information memory 501 and the time direction reference motion information memory 502. The flowchart which shows an example of operation | movement of the motion information compression part of FIG. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit when the size of the encoding prediction unit is 32x32. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit when the size of the encoding target prediction unit is 32x16. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit when the size of the encoding target prediction unit is 16x32. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left end of a prediction unit when the size of the encoding target prediction unit is 16x16. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the left upper end of a prediction unit when the size of the encoding prediction unit is 16x8. Explanatory drawing which shows the example of the reference motion information acquisition position which shows the upper left upper end of a prediction unit in case the size of the encoding target prediction unit is 8x16. Explanatory drawing which shows the example of a representative motion information position. Explanatory drawing which shows another example of a representative motion information position. Explanatory drawing which shows the example of the center of the prediction unit in each prediction size. Explanatory drawing which shows the example of a representative motion information position at the time of setting the gravity center of the some reference motion information acquisition position for every motion information compression block as a representative motion information position. Explanatory drawing which shows another example of the representative motion information position at the time of setting the gravity center of the some reference motion information acquisition position for every motion information compression block as a representative motion information position. Explanatory drawing which shows the example of a representative motion information position. Explanatory drawing which shows another example of a representative motion information position. It is a figure which shows the syntax structure according to one Embodiment. It is a figure which shows an example of the sequence parameter set syntax according to one Embodiment. It is a figure which shows another example of the sequence parameter set syntax according to one Embodiment. It is a figure which shows an example of the prediction unit syntax according to one Embodiment. The block diagram which shows roughly the image decoding apparatus which concerns on 2nd Embodiment. FIG. 26 is a block diagram schematically showing an entropy decoding unit in FIG. 25. FIG. 27 is a block diagram schematically showing a motion information decoding unit in FIG. 26.

Hereinafter, with reference to the drawings, a video encoding device and a video decoding device according to each embodiment will be described in detail. In the following description, the term “image” can be appropriately read as terms such as “video”, “pixel”, “image signal”, and “image data”. Moreover, in the following embodiment, the same number is attached | subjected about what performs the same operation | movement, and repeated description is abbreviate | omitted.
(First embodiment)
The first embodiment relates to an image encoding device. A moving image decoding apparatus corresponding to the image encoding apparatus according to the present embodiment will be described in the second embodiment. This image encoding device can be realized by hardware such as an LSI (Large-Scale Integration) chip, a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array). The image encoding apparatus can also be realized by causing a computer to execute an image encoding program.

  As illustrated in FIG. 1, the image encoding device 100 according to the present embodiment includes a subtraction unit 101, an orthogonal transformation unit 102, a quantization unit 103, an inverse quantization unit 104, an inverse orthogonal transformation unit 105, an addition unit 106, and a reference. An image memory 107, an inter prediction unit 108, a motion information compression unit 109, a motion information memory 110, and an entropy coding unit 112 are included. The encoding control unit 114 and the output buffer 113 are usually installed outside the image encoding apparatus 100.

  The image coding apparatus 100 in FIG. 1 divides each frame, each field, or each slice constituting the input image signal 151 into a plurality of pixel blocks, performs predictive coding on the divided pixel blocks, and performs coding. The converted data 163 is output. In the following description, for the sake of simplicity, it is assumed that pixel blocks are predictively encoded from the upper left to the lower right as shown in FIG. 2A. In FIG. 2A, the encoded pixel block p is located on the left side and the upper side of the encoding target pixel block c in the encoding processing target frame f.

  Here, the pixel block refers to a unit for processing an image such as an M × N size block (N and M are natural numbers), a coding unit, a macro block, a sub block, and one pixel. In the following description, the pixel block is basically used in the meaning of the coding unit. However, the pixel block can be interpreted in the above-described meaning by appropriately replacing the description. The coding unit is typically a 16 × 16 pixel block shown in FIG. 2B, for example, but may be a 32 × 32 pixel block shown in FIG. 2C or a 64 × 64 pixel block shown in FIG. 2D. It may be an 8 × 8 pixel block or a 4 × 4 pixel block. Also, the coding unit need not necessarily be square. Hereinafter, the encoding target block or coding unit of the input image signal 151 may be referred to as a “prediction target block”. The coding unit is not limited to a pixel block such as a coding unit, and a frame, a field, a slice, or a combination thereof can be used.

  3A to 3D are diagrams illustrating specific examples of coding units. FIG. 3A shows an example when the size of the coding unit is 64 × 64 (N = 32). Here, N represents the size of the reference coding unit. The size when divided is defined as N, and the case where it is not divided is defined as 2N. The coding tree unit has a quadtree structure, and when divided, the four pixel blocks are indexed in the Z-scan order. FIG. 3B shows an example in which the 64 × 64 pixel block in FIG. 3A is divided into quadtrees. The numbers shown in the figure represent the Z scan order. Further, it is possible to further perform quadtree division within the index of one quadtree of the coding unit. The depth of division is defined by Depth. That is, FIG. 3A shows an example in which Depth = 0. FIG. 3C shows an example of a 32 × 32 (N = 16) size coding tree unit in the case of Depth = 1. A unit having the largest coding tree unit is called a large coding tree unit or tree block, and as shown in FIG. 2A, input image signals are encoded in this unit in raster scan order.

  The image coding apparatus 100 in FIG. 1 performs inter prediction (also referred to as inter-screen prediction, inter-frame prediction, motion compensation prediction, or the like) for a pixel block based on a coding parameter input from the coding control unit 114. Intra prediction (also referred to as intra-screen prediction, intra-frame prediction, etc.) not shown is performed to generate a predicted image signal 159. The image encoding apparatus 100 performs orthogonal transform and quantization on the prediction error signal 152 between the pixel block (input image signal 151) and the predicted image signal 159, performs entropy encoding, and generates encoded data 163. Output.

  The image encoding apparatus 100 in FIG. 1 performs encoding by selectively applying a plurality of prediction modes having different block sizes and generation methods of the predicted image signal 159. The generation method of the predicted image signal 159 can be broadly divided into two types: intra prediction in which prediction is performed in the encoding target frame and inter prediction in which prediction is performed using one or a plurality of reference frames that are temporally different. is there.

Hereinafter, each element included in the image encoding device 100 of FIG. 1 will be described.
The subtraction unit 101 subtracts the corresponding prediction image signal 159 from the encoding target block of the input image signal 151 to obtain a prediction error signal 152. The subtraction unit 101 inputs the prediction error signal 152 to the orthogonal transformation unit 102.

  The orthogonal transform unit 102 performs orthogonal transform such as discrete cosine transform (DCT) on the prediction error signal 152 from the subtraction unit 101 to obtain a transform coefficient 153. The orthogonal transform unit 102 outputs the transform coefficient 153 to the quantization unit 103.

  The quantization unit 103 performs quantization on the transform coefficient 153 from the orthogonal transform unit 102 to obtain a quantized transform coefficient 154. Specifically, the quantization unit 103 performs quantization according to quantization information such as a quantization parameter and a quantization matrix specified by the encoding control unit 114. The quantization parameter indicates the fineness of quantization. Although the quantization matrix is used for weighting the fineness of quantization for each component of the transform coefficient, the use / nonuse of the quantization matrix is not an essential part of the embodiment of the present invention. The quantization unit 103 outputs the quantized transform coefficient 154 to the entropy encoding unit 112 and the inverse quantization unit 104.

  The entropy encoding unit 112 includes a quantized transform coefficient 154 from the quantization unit 103, motion information 160 from the inter prediction unit 108, prediction information 165 specified by the encoding control unit 114, and reference from the encoding control unit 114. Entropy encoding (for example, Huffman encoding, arithmetic encoding, etc.) is performed on various encoding parameters such as position information 164 and quantization information to generate encoded data 163. The encoding parameter is a parameter necessary for decoding such as prediction information 165, information on transform coefficients, information on quantization, and the like. For example, the encoding control unit 114 has an internal memory (not shown), the encoding parameters are held in this memory, and the encoding parameters of the adjacent already encoded pixel blocks are encoded when encoding the prediction target block. Use.

  Specifically, the entropy encoding unit 112 includes a parameter encoding unit 401, a transform coefficient encoding unit 402, a motion information encoding unit 403, and a multiplexing unit 404, as shown in FIG. The parameter encoding unit 401 encodes encoding parameters such as the prediction information 165 received from the encoding control unit 114 to generate encoded data 451A. The transform coefficient encoding unit 402 encodes the quantized transform coefficient 154 received from the quantization unit 103 to generate encoded data 451B.

  The motion information encoding unit 403 encodes the motion information 160 received from the inter prediction unit 108 with reference to the reference motion information 166 received from the motion information memory 110 and the reference position information 164 received from the encoding control unit 114. Thus, encoded data 451C is generated. Details of the motion information encoding unit 403 will be described later.

  The multiplexing unit 404 multiplexes the encoded data 451A, 451B, and 451C to generate encoded data 163. The generated encoded data 163 includes all parameters necessary for decoding such as information on transform coefficients and information on quantization, along with motion information 160 and prediction information 165.

  The encoded data 163 generated by the entropy encoding unit 112 is temporarily accumulated in the output buffer 113 through multiplexing, for example, and output as encoded data 163 according to an appropriate output timing managed by the encoding control unit 114. The The encoded data 163 is output to, for example, a storage system (storage medium) or a transmission system (communication line) not shown.

  The inverse quantization unit 104 performs inverse quantization on the quantized transform coefficient 154 from the quantization unit 103 to obtain a restored transform coefficient 155. Specifically, the inverse quantization unit 104 performs inverse quantization according to the quantization information used in the quantization unit 103. The quantization information used in the quantization unit 103 is loaded from the internal memory of the encoding control unit 114. The inverse quantization unit 104 outputs the restored transform coefficient 155 to the inverse orthogonal transform unit 105.

  The inverse orthogonal transform unit 105 performs an inverse orthogonal transform corresponding to the orthogonal transform performed in the orthogonal transform unit 102 such as an inverse discrete cosine transform on the restored transform coefficient 155 from the inverse quantization unit 104, A restored prediction error signal 156 is obtained. The inverse orthogonal transform unit 105 outputs the restored prediction error signal 156 to the addition unit 106.

  The adder 106 adds the restored prediction error signal 156 and the corresponding predicted image signal 159 to generate a local decoded image signal 157. The decoded image signal 157 is subjected to a deblocking filter and a Wiener filter (not shown) and is input to the reference image memory 107.

  The reference image memory 107 stores the filtered image signal 158 after local decoding in the memory, and is referred to as the reference image signal 158 when the inter prediction unit 108 generates a prediction image as necessary.

  The inter prediction unit 108 performs inter prediction using the reference image signal 158 stored in the reference image memory 107. Specifically, the inter prediction unit 108 performs block matching processing between the prediction target block and the reference image signal 158 to derive a motion shift amount (motion vector). The inter prediction unit 108 performs motion compensation (interpolation processing in the case of decimal precision motion) based on the motion vector to generate an inter prediction image. H. With H.264, interpolation processing up to 1/4 pixel accuracy is possible. The derived motion vector is entropy encoded as part of the motion information 160.

  The motion information memory 110 includes a motion information compression unit 109, appropriately compresses the motion information 160 to reduce the amount of information, and temporarily stores it as reference motion information 166. As shown in FIG. 5, the motion information memory 110 is held in units of frames (or slices), and the spatial direction reference motion information memory 501 that stores the motion information 160 on the same frame as the reference motion information 166, and already It further includes a time direction reference motion information memory 502 that stores motion information 160 of the frame that has been encoded as reference motion information 166. A plurality of temporal direction reference motion information memories 502 may be provided depending on the number of reference frames used for prediction by the encoding target frame.

  Further, the spatial direction reference motion information memory 501 and the temporal direction reference motion information memory 502 may be logically divided into physically identical memories. Furthermore, the spatial direction reference motion information memory 501 holds only the spatial direction motion information necessary for the frame that is currently encoded, and sequentially compresses the spatial direction motion information that is no longer needed for reference, thereby generating a temporal direction reference motion. It may be stored in the information memory 502.

  The reference motion information 166 is held in the spatial direction reference motion information memory 501 and the temporal direction reference motion information memory 502 in a predetermined area unit (for example, 4 × 4 pixel block unit). The reference motion information 166 further includes information indicating whether the region is encoded by inter prediction described later or encoded by intra prediction described later. Also, the coding unit (or prediction unit) is H.264. As in the skip mode, the direct mode or the merge mode described later in H.264, the motion vector value in the motion information 160 is not encoded, and the motion information 160 predicted from the encoded region is used for the interpolating. Even when predicted, the motion information of the coding unit (or prediction unit) is retained as the reference motion information 166.

  When the encoding process of the frame or slice to be encoded is completed, the handling of the spatial direction reference motion information memory 501 of the frame is changed as the temporal direction reference motion information memory 502 used for the next frame to be encoded. The At this time, in order to reduce the memory capacity of the time direction reference motion information memory 502, the motion information 160 compressed by the motion information compression unit 109 described later is stored in the time direction reference motion information memory 502.

  The prediction information 165 follows the prediction mode controlled by the encoding control unit 114, and as described above, inter prediction or intra prediction or inter prediction (not shown) can be selected to generate the predicted image signal 159. A plurality of modes can be further selected for each of the inter prediction and the inter prediction. The encoding control unit 114 determines one of a plurality of prediction modes of intra prediction and inter prediction as the optimal prediction mode, and sets the prediction information 165.

  For example, the encoding control unit 114 determines an optimal prediction mode using a cost function represented by the following formula (1).

  In Equation (1) (hereinafter referred to as simple encoding cost), OH indicates a code amount relating to prediction information 160 (for example, motion vector information and prediction block size information), and SAD is a prediction target block and a prediction image signal 159. The difference absolute value sum (ie, the cumulative sum of the absolute values of the prediction error signal 152) is shown. Further, λ represents a Lagrange undetermined multiplier determined based on the value of quantization information (quantization parameter), and K represents an encoding cost. When Expression (1) is used, the prediction mode that minimizes the coding cost K is determined as the optimum prediction mode from the viewpoint of the generated code amount and the prediction error. As a modification of Equation (1), the encoding cost may be estimated from OH alone or SAD alone, or the encoding cost may be estimated using a value obtained by subjecting SAD to Hadamard transform or an approximation thereof.

  It is also possible to determine an optimal prediction mode by using a temporary encoding unit (not shown). For example, the encoding control unit 114 determines an optimal prediction mode using a cost function expressed by the following formula (2).

  In Equation (2), D represents the sum of square errors (ie, encoding distortion) between the prediction target block and the local decoded image, and R represents the prediction between the prediction target block and the prediction image signal 159 in the prediction mode. An error amount indicates a code amount estimated by provisional encoding, and J indicates an encoding cost. In order to derive the encoding cost J (hereinafter referred to as the detailed encoding cost) of Equation (2), provisional encoding processing and local decoding processing are required for each prediction mode, so that the circuit scale or the amount of calculation increases. . On the other hand, since the encoding cost J is derived based on more accurate encoding distortion and code amount, it is easy to determine the optimal prediction mode with high accuracy and maintain high encoding efficiency. As a modification of Equation (2), the encoding cost may be estimated from only R or D, or the encoding cost may be estimated using an approximate value of R or D. These costs may be used hierarchically. The encoding control unit 114 performs determination using Expression (1) or Expression (2) based on information obtained in advance regarding the prediction target block (prediction mode of surrounding pixel blocks, results of image analysis, and the like). The number of prediction mode candidates may be narrowed down in advance.

  As a modification of the present embodiment, the number of prediction mode candidates can be further reduced while maintaining encoding performance by performing two-stage mode determination combining Formula (1) and Formula (2). It becomes. Here, unlike the formula (2), the simple encoding cost represented by the formula (1) does not require a local decoding process, and can be calculated at high speed. In the moving picture coding apparatus according to the present embodiment, H.264 is used. Since the number of prediction modes is large even when compared with H.264, mode determination using the detailed coding cost is not realistic. Therefore, as a first step, mode determination using the simple coding cost is performed on the prediction modes available in the pixel block, and prediction mode candidates are derived.

  Here, the number of prediction mode candidates is changed using the property that the correlation between the simple coding cost and the detailed coding cost increases as the value of the quantization parameter that determines the roughness of quantization increases.

Next, the prediction process of the image coding apparatus 100 will be described.
Although not shown in the figure, the image coding apparatus 100 in FIG. 1 has a plurality of prediction modes, and the prediction image signal 159 generation method and the motion compensation block size are different in each prediction mode. Specifically, the prediction unit 108 generates the predicted image signal 159 broadly, and can be broadly divided into intra prediction (intraframe) that generates a predicted image using the reference image signal 158 of the encoding target frame (or field). Prediction) and inter prediction (interframe prediction) for generating a prediction image using the reference image signal 158 of one or more encoded reference frames (or reference fields). The prediction unit 108 selectively switches between intra prediction and inter prediction, and generates a prediction image signal 159 of the encoding target block.

  FIG. 6A illustrates an example of inter prediction. Inter prediction is typically performed in units of prediction units, and can have different motion information 160 in units of prediction units. In inter prediction, as shown in FIG. 6A, a pixel block in a reference frame that has already been encoded (for example, an encoded frame one frame before), and a prediction unit to be encoded A predicted image signal 159 is generated from the block 601 at the same position using the reference image signal 158 of the block 602 at a position spatially shifted according to the motion vector included in the motion information 160. That is, in the generation of the predicted image signal 159, the reference image signal 158 of the block 602 in the reference frame specified by the position (coordinates) of the encoding target block and the motion vector included in the motion information 160 is used.

  In inter prediction, motion compensation with small pixel accuracy (for example, 1/2 pixel accuracy or 1/4 pixel accuracy) is possible, and by performing a filtering process on the reference image signal 158, an interpolated pixel value is generated. Is done. For example, H.M. In H.264, it is possible to perform interpolation processing up to 1/4 pixel accuracy on the luminance signal. The interpolation process is described in H.264. In addition to the filtering defined by H.264, it can be executed by using arbitrary filtering.

  In inter prediction, not only an example in which a reference frame one frame before as shown in FIG. 6A is used, but any encoded reference frame may be used as shown in FIG. 6B. When the reference image signals 158 of a plurality of reference frames having different time positions are held, information indicating which time position the reference image signal 159 has generated the predicted image signal 159 is represented by a reference frame number. The reference frame number is included in the motion information 160. The reference frame number can be changed in region units (picture, slice, block unit, etc.). That is, a different reference frame can be used for each prediction unit. As an example, when the encoded reference frame one frame before is used for prediction, the reference frame number of this region is set to 0, and when the encoded reference frame two frames before is used for prediction, The reference frame number of this area is set to 1. As another example, when the reference image signal 158 for only one frame is held in the reference image memory 107 (the number of reference frames held is only one), the reference frame number is always 0. Is set.

  Furthermore, in inter prediction, a size suitable for a block to be encoded can be selected and used from the sizes of a plurality of prediction units prepared in advance. For example, it is possible to perform motion compensation for each prediction unit obtained by dividing a coding tree unit as shown in FIGS. 7A to 7G. Further, it is possible to perform motion compensation for each prediction unit obtained by dividing the rectangle other than the rectangles as shown in FIGS. 7F and 7G.

  As described above, the motion information 160 of the encoded pixel block (for example, 4 × 4 pixel block) in the encoding target frame used for the inter prediction is held as the reference motion information 166, so that the input image signal According to the local nature of 151, the optimal motion compensation block shape and motion vector, reference frame number can be used. Further, the coding unit and the prediction unit can be arbitrarily combined. When the coding tree unit is a 64 × 64 pixel block, the coding tree unit is further divided into four for each of the four coding tree units (32 × 32 pixel block) obtained by dividing the 64 × 64 pixel block. Thus, 64 × 64 pixel blocks to 16 × 16 pixel blocks can be used hierarchically. Similarly, 64 × 64 pixel blocks to 8 × 8 pixel blocks can be used hierarchically. Here, if the prediction unit is obtained by dividing the coding tree unit into four, it is possible to execute a hierarchical motion compensation process from a 64 × 64 pixel block to a 4 × 4 pixel block. .

  Also, in inter prediction, bi-directional prediction using two types of motion compensation can be performed on the encoding target pixel block. H. In H.264, two types of motion compensation are performed on the encoding target pixel block, and a weighted average of the two types of predicted image signals is obtained to obtain a new predicted image signal (not shown). Two types of motion compensation in bidirectional prediction are referred to as list 0 prediction and list 1 prediction, respectively.

<Explanation of skip mode, merge mode, and inter mode>
The image encoding apparatus 100 according to the present embodiment uses a plurality of prediction modes with different encoding processes shown in FIG. The skip mode in the figure is a mode in which only syntax related to a predicted motion information position 954 described later is encoded, and other syntax is not encoded. The merge mode is a mode in which only the syntax relating to the predicted motion information position 954 and the transform coefficient information 153 are encoded, and other syntaxes are not encoded. The inter mode is a mode for encoding syntax related to the predicted motion information position 954, differential motion information 953, which will be described later, and transform coefficient information 153. These modes are switched by prediction information 165 controlled by the encoding control unit 114.

<Motion Information Encoding Unit 403>
Hereinafter, the motion information encoding unit 403 will be described with reference to FIG.

  The motion information encoding unit 403 includes a reference motion vector acquisition unit 901, a prediction motion vector selection switch (also referred to as a prediction motion information selection switch) 902, a subtraction unit 903, a difference motion information encoding unit 904, and a prediction motion information position encoding. A unit 905 and a multiplexing unit 906.

  The reference motion vector acquisition unit 901 receives at least the reference motion information 166 and the reference position information 164 and generates at least one predicted motion information candidate (also referred to as a predicted motion vector candidate) 951 (951A, 951B,...). . 10 and 11 show an example of the position of the predicted motion information candidate 951 with respect to the target prediction unit. FIG. 10 shows the position of the prediction unit spatially adjacent to the target prediction unit. AX (X = 0 to nA-1) is a prediction unit adjacent to the left with respect to the target prediction unit, and BY (Y = 0 to nB-1) is a predicate adjacent to the target prediction unit. C, D, and E indicate prediction units adjacent to the target prediction unit at the upper right, upper left, and lower left, respectively. FIG. 11 shows the position of the prediction unit in the already encoded reference frame with respect to the encoding target prediction unit. Col in FIG. 11 indicates a prediction unit in the reference frame and at the same position as the encoding target prediction unit. FIG. 12 shows an example of a list indicating the relationship between the block positions of the plurality of predicted motion information candidates 951 and the index Mvpidx. When Mvpidx is 0 to 2, a predicted motion vector candidate 951 is located in the spatial direction, and Mvpidx is 3 is a predicted motion vector candidate 951 is located in the temporal direction. Prediction unit position A is an inter prediction among AX shown in FIG. 10, that is, a prediction unit having reference motion information 166, and a position where the value of X is the smallest is taken as prediction unit position A. Prediction unit position B is inter prediction among BY shown in FIG. 10, that is, a prediction unit having reference motion information 166, and the position with the smallest Y value is designated as prediction unit position A. To do. When the prediction unit position C is not inter prediction, the reference motion information 166 of the prediction unit position D is replaced with the reference motion information 166 of the prediction unit position C. When the prediction unit positions C and D are not inter prediction, the reference motion information 166 of the prediction unit position E is replaced with the reference motion information 166 of the prediction unit position C.

  When the size of the encoding target prediction unit is larger than the minimum prediction unit, the prediction unit position Col may hold a plurality of reference motion information 166 in the temporal direction reference motion information memory 502. In this case, the reference motion information 166 in the prediction unit at the position Col is acquired according to the reference position information 164. Hereinafter, the acquisition position of the reference motion information 166 in the prediction unit at the position Col is referred to as a reference motion information acquisition position. 13A to 13F show an example of the reference motion information acquisition position when the reference position information 164 indicates the center of the prediction unit at the position Col, for each size (32 × 32 to 16 × 16) of the encoding target prediction unit. Each block in the figure indicates a 4 × 4 prediction unit, and a circle indicates the position of the 4 × 4 prediction unit acquired as the predicted motion information candidate 951. Another example of the reference motion information acquisition position is shown in FIGS. 14A to F, since there are no 4x4 prediction units at the positions of the circles, a predetermined method such as an average value or median value of the reference motion information 166 in four 4x4 prediction units adjacent to the circles, A predicted motion information candidate 951 is generated. As yet another example of the reference motion information acquisition position, the reference motion information 166 of the 4 × 4 prediction unit located at the upper left end of the prediction unit at the position Col may be used as the predicted motion information candidate 951. In addition to the above examples, the predicted motion information candidate 951 may be generated using any position and method as long as it is a predetermined method.

  When the reference motion information 166 does not exist, the motion information 160 having a zero vector is output as the predicted motion information candidate 951.

  As described above, at least one predicted motion information candidate 951 is output from the reference motion block. When the reference frame number included in the prediction motion information candidate 951 is different from the reference frame number of the encoding target prediction unit, the reference frame number included in the prediction motion information candidate 951 and the encoding target prediction You may scale according to the reference frame number of the unit.

  The predicted motion information selection switch 902 selects one of a plurality of predicted motion information candidates 951 in response to a command from the encoding control unit 114 and outputs predicted motion information 952. The predicted motion information selection switch 902 may output predicted motion information position information 954 described later. The selection may be made by using an evaluation function such as Equation (1) or (2). The subtraction unit 903 subtracts the predicted motion vector information 952 from the motion information 160, and outputs the difference motion information 953 to the difference motion information encoding unit 904. The difference motion information encoding unit 904 encodes the difference motion information 953 and outputs encoded data 960A. In the skip mode and the merge mode, the difference motion information encoding unit 904 does not need to encode the difference motion information 953.

  The predicted motion information position encoding unit 905 encodes predicted motion information position information 954 (Mvpidx) indicating which predicted motion information candidate 951 has been selected from the list shown in FIG. 12, and outputs encoded data 960B. To do. The predicted motion information position information 954 is encoded by using equal length encoding or variable length encoding generated from the total number of predicted motion information candidates 951. Variable length coding may be performed using the correlation with adjacent blocks. Further, when there are overlapping information in a plurality of motion vector predictor candidates 951, a code table is created from the total number of motion vector predictor candidates 951 from which the motion vector predictor candidates 951 that are duplicated are deleted, and the motion information position information 954 is encoded. It doesn't matter. Further, when the total number of predicted motion information candidates 951 is one type, the predicted motion information candidate 951 is determined as the predicted motion information 952, and therefore it is not necessary to encode the predicted motion information position information 954.

  In addition, in each of the skip mode, the merge mode, and the inter mode, the derivation method of the prediction motion information candidate 951 does not need to be the same, and the derivation method of the prediction motion information candidate 951 may be set independently. In this embodiment, description will be made assuming that the derivation method of the prediction motion information candidate 951 in the skip mode and the inter mode is the same, and the derivation method of the prediction motion information candidate 951 in the merge mode is different.

<Details of Motion Information Compression Unit 109>
First, the motion information compression process will be described with reference to FIG. In FIG. 15, the reference motion information 166 in the spatial direction reference motion information memory 501 is compressed and stored in the temporal direction reference motion information memory 502. The spatial direction reference motion information memory 501 stores the reference motion information 166 held at the representative motion information position for each motion information compression block (16 × 16 pixel block in the figure) in the temporal direction reference motion information memory 502. When the above-described motion information encoding process is performed, the reference motion information 166 held at the above-described reference motion information acquisition position is set as the predicted motion information candidate 951. At this time, the reference motion information 166 held at the reference motion information acquisition position may be set as the predicted motion information candidate 951, assuming that the motion information compression block virtually has the same reference motion information 166. No (the same predicted motion information candidate 951 is derived)
Next, the motion information compression unit 109 will be described with reference to the flowchart shown in FIG. The motion information compression unit 109 compresses the motion information 160 and stores the motion information 160 in the temporal direction reference motion information memory 502 when the encoding process of the frame (or any unit such as a slice or a coding unit) is completed. .

  First, the reference position information 164 is acquired from the encoding control unit 114 (step S1601), and the frame is divided into motion information compression blocks which are compression units of the motion information 160 (step S1602). The motion information compression block is a pixel block larger than a unit (typically a 4 × 4 pixel block) in which the motion information 160 is held by the motion compensation process, and is typically a 16 × 16 pixel block. The motion information compression block may be a 64 × 64 pixel block, a 32 × 32 pixel block, an 8 × 8 pixel block, a rectangular pixel block, or a pixel region having an arbitrary shape.

  Next, a representative motion information position is generated according to the reference position information 164 (step S1603). As an example of generating the representative motion information position, when the motion information compression block is a 16 × 16 pixel block, the reference motion information acquisition position when the size of the prediction unit shown in FIGS. 13D, 14D, and 17D is 16 × 16 is representative. Let it be the motion information position. Next, the generated reference motion information 166 of the representative motion information position is set as representative motion information (step S1604), and the representative motion information is stored in the time direction reference motion information memory (step S1605). The above steps S1604 to S1605 are executed for all motion information compression blocks.

  Assuming that the unit in which the motion information 160 is held is an MxM block and the size of the motion information compression block is NxN (N is a multiple of M), the motion information compression processing is executed, so that the capacity of the reference motion information memory is (MxM ) / (N × N).

<Another embodiment of representative motion information position>
As another example of generating the representative motion information position, the center position of a plurality of reference motion information acquisition positions may be used as the representative motion information position. 18A and 18B show representative motion information positions for each motion compression block having a size of 16 × 16. 18A shows the representative motion information position when the reference motion information acquisition position is the position shown in FIG. 13D. Similarly, FIG. 18B shows the representative motion information when the reference motion information acquisition position is the position shown in FIG. 17D. Each position is shown. The circles in FIGS. 18A and 18B indicate the reference motion information acquisition positions when the prediction unit is a 16 × 16 block, and the center position (also referred to as the center of gravity position) of the four reference motion information acquisition positions. Representative motion information positions indicated by crosses are arranged.

  As still another example of generating the representative motion information position, the reference motion information acquisition position for each size of the plurality of prediction units is included as the reference position information 164, and the representative motion information position is generated from the plurality of reference motion information acquisition positions. It doesn't matter.

  As an example of generating the representative motion information position, the reference motion information acquisition position for each size of the plurality of prediction units is included as the reference position information 164, and the representative motion information position is generated from the plurality of reference motion information acquisition positions. I do not care. FIG. 19 shows the centers of the prediction units (reference motion information acquisition positions) in each size of the prediction unit size of 16 × 16 or more when the tree block is a 64 × 64 pixel block.

  As another example of generating the representative motion information position, the representative motion information position may be set using a reference motion information acquisition position arranged for each motion information compression block. FIG. 20A shows an example in which the center of gravity of a plurality of reference motion information acquisition positions for each motion information compression block is set as the representative motion information position. If the position of the center of gravity does not match the position of the 4x4 block, the nearest 4x4 block may be used as the representative motion information position, or the reference motion vector 166 of the center of gravity position may be obtained using an interpolation method such as bilinear interpolation. It may be generated.

  FIG. 20B shows an example in which one of a plurality of reference motion information acquisition positions is selected for each motion information compression block and set as a representative motion information position.

  Further, FIGS. 21A and 21B further illustrate an example in which the reference motion information acquisition position is made the same in each motion information compression block in the tree block. Since it is the same representative motion information position in all the motion information compression blocks, it is not necessary to switch the representative motion information position according to the position in the tree block. Further, the representative motion information position may be at any position such as the upper left end or the upper right end in the motion information compression block, in addition to FIGS. 21A and 21B.

  As an example of generating the representative motion information position, the representative motion information position may be indicated by using BlkIdx indicating the 4 × 4 block position in the motion information compression block in the Z scan order. When the size of the motion information compression block is 16 × 16, the representative motion information position shown in FIG. 21A corresponds to the position of BlkIdx = 12. Further, the representative motion information position shown in FIG. 21B corresponds to the position of BlkIdx = 15.

  As another example of the motion information compression process, the reference frame number may be included in the motion information compression process in order to reduce the memory capacity related to the reference frame number. In this case, the reference frame number held at the representative motion information position is stored in the memory capacity related to the reference frame number. Therefore, the spatial direction reference motion information memory 501 and the spatial direction reference motion information memory 502 shown in FIG. 5 store the reference frame number in addition to the motion vector information.

  As yet another example of the motion information compression processing, when the reference frame number is not included in the motion information compression processing, the motion vector information in the motion information at the representative motion information position is subjected to scaling processing using the reference frame number. Thus, it may be stored in the motion information memory 110. A typical example of the scaling process is a linear scaling process based on a reference frame number of zero. In this case, when the reference frame number is a value other than zero, linear scaling processing is performed so that the motion vector information refers to the reference frame corresponding to the reference frame number zero. The reference for the scaling process described above may be a value other than zero for the reference frame number. When division occurs when performing the above-described linear scaling processing, the division processing may be tabulated in advance and the division may be realized by subtracting the table each time.

  When the size of the motion information compression block is other than 16 × 16 blocks, the representative motion information position is generated using the same processing as described above. For example, when the size of the motion information compression block is 64 × 64, the reference motion information acquisition position when the size of the prediction unit is 64 × 64 is set as the representative motion information position. In yet another example, the representative motion information position obtained by scaling the representative motion information position in the motion information compression block size 16 × 16 block shown in FIGS. 21A and 21B etc. in the horizontal direction and the vertical direction according to the size of the motion information compression block. It does not matter as the information position.

  If the reference motion information does not exist because the representative motion information position is outside the picture or slice, the position where the reference motion information can be acquired within the motion information compression block, such as the upper left corner of the motion information compression block, is set as the new representative motion. It may be replaced as an information position. Also, when the representative motion information position is an area to which intra prediction is applied and no reference motion information exists, the same processing may be executed and replaced with a new representative motion information position.

<Syntax configuration>
Hereinafter, the syntax used by the image coding apparatus 100 in FIG. 1 will be described.
The syntax indicates the structure of encoded data (for example, encoded data 163 in FIG. 1) when the image encoding apparatus encodes moving image data. When decoding the encoded data, the moving picture decoding apparatus interprets the syntax with reference to the same syntax structure. FIG. 22 shows an example of syntax 2200 used by the moving picture encoding apparatus of FIG.

  The syntax 2200 includes three parts: a high level syntax 2201, a slice level syntax 2202, and a coding tree level syntax 2203. The high level syntax 2201 includes syntax information of a layer higher than the slice. A slice refers to a rectangular area or a continuous area included in a frame or a field. The slice level syntax 2202 includes information necessary for decoding each slice. The coding tree level syntax 2203 includes information necessary for decoding each coding tree (ie, each coding tree unit). Each of these parts includes more detailed syntax.

  High level syntax 2201 includes sequence and picture level syntax, such as sequence parameter set syntax 2204 and picture parameter set syntax 2205. The slice level syntax 2202 includes a slice header syntax 2206, a slice data syntax 2207, and the like. The coding tree level syntax 2203 includes a coding tree unit syntax 2208, a transform unit syntax 2209, a prediction unit syntax 2210, and the like.

  The coding tree unit syntax 2208 may have a quadtree structure. Specifically, the coding tree unit syntax 2208 can be recursively called as a syntax element of the coding tree unit syntax 2208. That is, one coding tree unit can be subdivided with a quadtree. In addition, the coding tree unit syntax 2208 includes a transform unit syntax 2209 and a pre-dye cushion unit syntax 2210. The transform unit syntax 2209 and the predy cushion unit syntax 2210 are invoked at each coding tree unit syntax 2208 at the extreme end of the quadtree. The Predy cushion unit syntax 2210 describes information related to prediction, and the transform unit syntax 2209 describes information related to inverse orthogonal transformation and quantization.

FIG. 23 illustrates a sequence parameter set syntax 2204 according to this embodiment. The motion_vector_buffer_comp_flag shown in FIG. 23A and FIG. 23B is a syntax indicating whether the motion information compression according to the present embodiment is valid / invalid for the sequence. When motion_vector_buffer_comp_flag is 0, the motion information compression according to the present embodiment is invalid for the sequence. Accordingly, the process of the motion information compression unit shown in FIG. 1 is skipped. As an example, when motion_vector_buffer_comp_flag is 1, motion information compression according to the present mobile is effective for the sequence. Motion_vector_buffer_comp_ratio_log2 shown in FIG. 23 and FIG. 23B is information indicating a unit of motion information compression processing, and is shown when motion_vector_buffer_comp_flag is 1. motion_vector_buffer_comp_ratio_log2 indicates, for example, information on the size of the motion information compression block according to the present embodiment, and motion_vector_buffer_comp_ratio_log2 is 2 ^{(motion_vector_buffer_ratio_comput_ratio_complex_size_motion_comput_block_compression_com block_size)} . An example in which the minimum unit of motion compensation is a 4 × 4 pixel block, that is, the reference motion information memory is held in 4 × 4 pixel block units is shown below. When motion_vector_buffer_comp_ratio_log2 is 1, the size of the motion information compression block according to the present embodiment is an 8 × 8 pixel block. Similarly, when motion_vector_buffer_comp_ratio_log2 is 2, the size of the motion information compression block according to the present embodiment is a 16 × 16 pixel block. Motion_vector_buffer_comp_position shown in FIG. 23B is information indicating a representative motion information position in the motion information compression block, and is shown when motion_vector_buffer_comp_flag is 1. motion_vector_buffer_comp_position indicates, for example, the reference motion information position in the motion information compression block as shown in FIGS. 21A and 21B, or the reference motion information position for each motion information compression block as shown in FIGS. 20A and 20B. It doesn't matter. Further, it may be at the center of a plurality of blocks.

  In addition, as another example, a layer lower than a motion_vector_buffer_comp_flag, a motion_vector_buffer_comp_ratio_log2, a motion_vector_buffer_comp_position, a local unit of a picture tree, a coding tree unit, a coding tree unit, a coding tree unit, a coding tree unit The validity / invalidity of such prediction may be defined.

  FIG. 24 shows an example of the prediction unit syntax. The skip_flag in the figure is a flag indicating whether or not the prediction mode of the coding unit to which the prediction unit syntax belongs is the skip mode. When skip_flag is 1, it indicates that syntaxes other than the predicted motion information position information 954 (coding unit syntax, prediction unit syntax, transform unit syntax) are not encoded. NumMVPCand (L0) and NumMVPCand (L1) indicate the numbers of predicted motion information candidates 951 in list 0 prediction and list 1 prediction, respectively. When the motion vector predictor candidate 951 exists (NumMVPCand (LX)> 0, X = 0 or 1), mvp_idx_lX indicating the motion vector predictor position information 954 is encoded.

  When skip_flag is 0, it indicates that the prediction mode of the coding unit to which the prediction unit syntax belongs is not the skip mode. NumMergeCandidates indicates the number of predicted motion information candidates 951 derived in FIG. When the motion information candidate 951 exists (NumMergeCandidates> 0), merge_flag, which is a flag indicating whether the prediction unit is in the merge mode, is encoded. merge_flag indicates that the prediction unit is in the merge mode when the value is 1, and indicates that the prediction unit uses the inter mode when the value is 0. When merge_flag is 1 and there are two or more predicted motion information candidates 951 (NumMergeCandidates> 1), merge_idx, which is the predicted motion information 952 indicating which block of the predicted motion information candidates 951 is to be merged, is encoded. .

  When merge_flag is 1, prediction unit syntax other than merge_flag and merge_idx need not be encoded.

  When merge_flag is 0, it indicates that the prediction unit is in the inter mode. In the inter mode, in the case of mvd_lX (X = 0 or 1) indicating the difference motion vector information included in the difference motion information 953, the reference frame number ref_idx_lX, and the B slice, the prediction unit is unidirectional prediction (list 0 or list 1). Or inter_pred_idc indicating whether the prediction is bi-directional prediction. Similarly to the skip mode, NumMVPCand (L0) and NumMVPCand (L1) are acquired, and when there is a predicted motion information candidate 951 (NumMVPCand (LX)> 0, X = 0 or 1), predicted motion information position information 954. Mvp_idx_lX indicating is encoded.

  The above is the syntax configuration according to the present embodiment.

(Second Embodiment)
The second embodiment relates to a moving picture decoding apparatus. The video encoding device corresponding to the video decoding device according to the present embodiment is as described in the first embodiment. That is, the moving picture decoding apparatus according to the present embodiment decodes encoded data generated by, for example, the moving picture encoding apparatus according to the first embodiment.

  As shown in FIG. 25, the video decoding apparatus according to the present embodiment includes an entropy decoding unit 2501, an inverse quantization unit 2502, an inverse orthogonal transform unit 2503, an addition unit 2504, a reference image memory 2505, and an inter prediction unit 2506. A reference motion information memory 2507, a reference motion information compression unit 2508, and a decoding control unit 2510.

  25 decodes the encoded data 2550, stores the decoded image signal 2554 in the output buffer 2511, and outputs it as an output image. The encoded data 2550 is output from, for example, the moving image encoding device shown in FIG. 1, and is input to the moving image decoding device 2500 via a storage system or transmission system (not shown).

  The entropy decoding unit 2501 performs decoding based on the syntax for decoding the encoded data 2550. The entropy decoding unit 2501 sequentially entropy-decodes the code string of each syntax and reproduces the encoding parameters of the encoding target block such as motion information 2559 and quantization transform coefficient 2551. An encoding parameter is a parameter required for decoding, such as prediction information, information on transform coefficients, information on quantization, and the like.

  Specifically, the entropy decoding unit 2501 includes a separation unit 2601, a parameter decoding unit 2602, a transform coefficient decoding unit 2603, and a motion information decoding unit 2604 as shown in FIG. The separation unit 2601 separates the encoded data 2550, the encoded data 2651A related to the parameters to the parameter decoding unit 2602, the encoded data 2651B related to the transform coefficient to the transform coefficient decoding unit 2603, and the encoded data 2651C related to the motion information to the motion information. The data are output to the decoding unit 2604, respectively. The parameter decoding unit 2602 decodes the encoding parameter 2570 such as prediction information, outputs the encoding parameter 2570, and outputs it to the decoding control unit 2510. The transform coefficient decoding unit 2603 receives the encoded data 2651B, decodes the transform coefficient information 2551, and outputs it to the inverse quantization unit 2502.

  The motion information decoding unit 2604 receives the encoded data 2651C from the separation unit 2601, the reference position information 2560 from the decoding control unit 2510, and the reference motion information 2558 from the reference motion information memory 2507, and outputs the motion information 2559. The output motion information 2559 is input to the inter prediction unit 2506.

  As shown in FIG. 27, the motion information decoding unit 2604 includes a separation unit 2701, a difference motion information decoding unit 2702, a predicted motion information position decoding unit 2503, a reference motion information acquisition unit 2704, a predicted motion information selection switch 2705, and An adding unit 2706 is included.

  The encoded data 2651C related to the motion information is input to the separation unit 2701, and is separated into the encoded data 2751 related to the difference motion information and the encoded data 2752 related to the predicted motion information position. The difference motion information encoding unit 2702 receives the encoded data 2751 related to the difference motion information, and decodes the difference motion information 2753. The difference motion information 2753 is added to predicted motion information 2756, which will be described later, in an adder 2706, and motion information 2759 is output. The predicted motion information position decoding unit 2703 receives encoded data 2752 related to the predicted motion information position, and decodes the predicted motion information position 2754.

  The predicted motion information position 2754 is input to the predicted motion information selection switch 2705, and the predicted motion information 2756 is selected from the predicted motion information candidates 2755. The predicted motion information position information 2560 is decoded using equal length decoding or variable length decoding generated from the number of predicted motion information candidates 2755. You may perform variable length decoding using the correlation with an adjacent block. Furthermore, when overlapping with a plurality of prediction motion information candidates 2755, the prediction motion information position information 2560 may be decoded from a code table generated from the total number of prediction motion information candidates 2755 from which the overlap is deleted. Also, when the total number of predicted motion information candidates 2755 is one, the predicted motion information candidate 2755 is determined as the predicted motion information 2556, and therefore it is not necessary to decode the predicted motion information position information 2754.

  The reference motion information acquisition unit 2704 has the same configuration and processing content as the reference motion information acquisition unit 901 described in the first embodiment.

  The reference motion information acquisition unit 2704 receives the reference motion information 2558 and the reference position information 2560 and generates at least one predicted motion information candidate 2755 (2755A, 2755B,...). 10 and 11 show an example of the position of the predicted motion information candidate 2755 with respect to the decoding target prediction unit. FIG. 10 shows the position of the prediction unit spatially adjacent to the decoding target prediction unit. AX (X = 0 to nA-1) is a prediction unit adjacent to the left with respect to the target prediction unit, and BY (Y = 0 to nB-1) is a predicate adjacent to the target prediction unit. C, D, and E indicate prediction units adjacent to the decoding target prediction unit at the upper right, upper left, and lower left, respectively. FIG. 11 shows the position of the prediction unit in the reference frame that has already been decoded with respect to the decoding target prediction unit. Col in the figure indicates a prediction unit in the reference frame and at the same position as the decoding target prediction unit. FIG. 12 shows an example of a list indicating the relationship between the block positions of a plurality of motion estimation candidate candidates 2755 and the index Mvpidx. When Mvpidx is 0 to 2, a predicted motion information candidate 2755 located in the spatial direction and Mvpidx 3 indicates a measured motion vector candidate 2755 located in the temporal direction. Prediction unit position A is an inter prediction among AX shown in FIG. 10, that is, a prediction unit having reference motion information 2558, and a position having the smallest X value is defined as a prediction unit position A. Prediction unit position B is inter prediction among BY shown in FIG. 10, that is, a prediction unit having reference motion information 2558, and the position with the smallest Y value is designated as prediction unit position A. To do. When the prediction unit position C is not inter prediction, the reference motion information 2558 of the prediction unit position D is replaced with the reference motion information 2558 of the prediction unit position C. When the prediction unit positions C and D are not inter prediction, the reference motion information 2558 of the prediction unit position E is replaced with the reference motion information 2558 of the prediction unit position C.

  When the size of the decoding target prediction unit is larger than the minimum prediction unit, the prediction unit position Col may hold a plurality of reference motion information 2558 in the temporal direction reference motion information memory 2507. In this case, reference motion information 2558 in the prediction unit at the position Col is acquired according to the reference position information 2560. Hereinafter, the acquisition position of the reference motion information 2558 in the prediction unit at the position Col is referred to as a reference motion information acquisition position. 13A to 13F show an example of the reference motion information acquisition position when the reference position information 2560 indicates the center of the prediction unit at the position Col, for each size (32 × 32 to 16 × 16) of the decoding target prediction unit. Each block in the figure represents a 4 × 4 prediction unit, and a circle represents the position of the 4 × 4 prediction unit acquired as the predicted motion information candidate 2755. Another example of the reference motion information acquisition position is shown in FIGS. In FIGS. 14A to F, since there are no 4 × 4 prediction units at the positions of the circles, a predetermined method such as an average value or a median value of the reference motion information 2558 in the four x4 prediction units adjacent to the circles, A predicted motion information candidate 2755 is generated. As yet another example of the reference motion information acquisition position, the reference motion information 2558 of the 4 × 4 prediction unit located at the upper left end of the prediction unit at the position Col may be used as the predicted motion information candidate 2755. In addition to the above examples, the predicted motion information candidate 2755 may be generated using any position and method as long as it is a predetermined method.

  When the reference motion information 2558 does not exist, motion information 2559 having a zero vector is output as a predicted motion information candidate 2755.

  Thus, at least one predicted motion information candidate 2755 is output from the reference motion block. When the reference frame number of the prediction motion information candidate 2755 and the reference frame number of the decoding target prediction unit are different, the reference frame number of the prediction motion information candidate 2755 and the decoding target prediction unit You may scale according to the reference frame number. The predicted motion information selection switch 2705 selects one from a plurality of predicted motion information candidates 2755 according to the predicted motion information position 2754, and outputs predicted motion information 952.

  The inverse quantization unit 2502 performs inverse quantization on the quantized transform coefficient 2551 from the entropy decoding unit 2501 to obtain a restored transform coefficient 2552. Specifically, the inverse quantization unit 2502 performs inverse quantization according to the information regarding quantization decoded by the entropy decoding unit 2501. The inverse quantization unit 2502 outputs the restored transform coefficient 2552 to the inverse orthogonal transform unit 2503.

  The inverse orthogonal transform unit 2503 performs an inverse orthogonal transform corresponding to the orthogonal transform performed on the encoding side on the restored transform coefficient 2552 from the inverse quantization unit 2502 to obtain a restored prediction error signal 2553. The inverse orthogonal transform unit 2503 inputs the restored prediction error signal 2553 to the addition unit 2504.

  The adding unit 2504 adds the restored prediction error signal 2553 and the corresponding predicted image signal 2556 to generate a decoded image signal 2554. The decoded image signal 2554 is subjected to a deblocking filter and a Wiener filter (not shown), temporarily stored in the output buffer 2511 for the output image, and also stored in the reference image memory 2505 for the reference image signal 2555. The The decoded image signal 2554 stored in the reference image memory 2505 is referred to as a reference image signal 2555 by the inter prediction unit 2506 in units of frames or fields as necessary. The decoded image signal 2554 temporarily stored in the output buffer 2511 is output according to the output timing managed by the decoding control unit 2510.

  The inter prediction unit 2506 performs inter prediction using the reference image signal 2555 stored in the reference image memory 2505. Specifically, the inter prediction unit 2506 obtains motion information 2559 including a motion shift amount (motion vector) between the prediction target block and the reference image signal 2555 from the entropy decoding unit 2501, and uses this motion vector as the motion vector. Based on this, interpolation processing (motion compensation) is performed to generate an inter prediction image. Since the generation of the inter prediction image is the same as that of the first embodiment, the description thereof is omitted.

  The decoding control unit 2510 controls each element of the video decoding device in FIG. Specifically, the decoding control unit 2510 outputs reference position information 2560, which will be described later, to the entropy decoding unit 2501, and performs various controls for decoding processing including the above-described operation.

<Explanation of skip mode, merge mode, and inter mode>
The image decoding apparatus 2500 according to the present embodiment uses a plurality of prediction modes with different decoding processes shown in FIG. The skip mode in the figure is a mode in which only syntax related to a predicted motion information position 2754 described later is decoded, and other syntax is not decoded. The merge mode is a mode in which only the syntax related to the predicted motion information position 2754 and the transform coefficient information 2551 are decoded, and the other syntaxes are not decoded. The inter mode is a mode for decoding syntax related to the predicted motion information position 2754, differential motion information 2753 described later, and transform coefficient information 2551. These modes are switched by prediction information 2571 controlled by the decoding control unit 2510.

  The moving picture decoding apparatus in FIG. 25 uses the same or similar syntax as the syntax described in FIG.

<Details of Motion Information Compression Unit 2508>
Next, the motion information compression unit 2508 will be described with reference to the flowchart shown in FIG. The motion information compression unit 2508 compresses the motion information 2559 and stores the motion information 2559 in the temporal direction reference motion information memory 502 when the decoding process of the frame (or any unit such as a slice or a coding unit) is completed. .

  First, reference position information 2560 is acquired from the decoding control unit 2510 (step S1601), and the frame is divided into motion information compression blocks that are compression units of the motion information 2559 (step S1602). The motion information compression block is a pixel block larger than a unit (typically a 4 × 4 pixel block) in which motion information 2559 is held by the motion compensation process, and is typically a 16 × 16 pixel block. The motion information compression block may be a 32 × 32 pixel block, an 8 × 8 pixel block, a rectangular pixel block, or a pixel region having an arbitrary shape.

  Next, a representative motion information position is generated according to the reference position information 2560 (step S1603). As an example of generating the representative motion information position, when the motion information compression block is a 16 × 16 pixel block, the reference motion information acquisition position when the size of the prediction unit shown in FIGS. 13D, 14D, and 17D is 16 × 16 is representative. Let it be the motion information position. Next, the generated reference motion information 2558 of the representative motion information position is set as representative motion information (step S1605), and the representative motion information is stored in the time direction reference motion information memory (step S1606). The above steps S1604 to S1605 are executed for all motion information compression blocks.

  Assuming that the unit in which the motion information 2559 is held is an MxM block and the size of the motion information compression block is NxN (N is a multiple of M), the motion information compression processing is executed, so that the capacity of the reference motion information memory is (MxM ) / (N × N).

<Another embodiment of representative motion information position>
As another example of generating the representative motion information position, the center position of a plurality of reference motion information acquisition positions may be used as the representative motion information position. 18A and 18B show representative motion information positions for each motion compression block having a size of 16 × 16. 18A shows the representative motion information position when the reference motion information acquisition position is the position shown in FIG. 13D. Similarly, FIG. 18B shows the representative motion information when the reference motion information acquisition position is the position shown in FIG. 17D. Each position is shown. The circles in FIGS. 18A and 18B indicate reference motion information acquisition positions when the prediction unit is 16 × 16, and are representative motions indicated by crosses at the center positions of the four reference motion information acquisition positions. The information position is arranged.

  As yet another example of generating the representative motion information position, the reference motion information acquisition position for each size of a plurality of prediction units is used as reference position information 2560, and the representative motion information position is generated from the plurality of reference motion information acquisition positions. It doesn't matter. FIG. 19 shows the centers of the prediction units (reference motion information acquisition positions) in each size of the prediction unit size of 16 × 16 or more when the tree block is a 64 × 64 pixel block.

  As another example of generating the representative motion information position, the representative motion information position may be set using a reference motion information acquisition position arranged for each motion information compression block. FIG. 20A shows an example in which the center of gravity of a plurality of reference motion information acquisition positions for each motion information compression block is set as the representative motion information position. If the position of the center of gravity does not match the position of the 4x4 block, the nearest 4x4 block may be used as the representative motion information position, or the reference motion vector 166 of the center of gravity position may be obtained using an interpolation method such as bilinear interpolation. It may be generated.

  FIG. 20B shows an example in which one of a plurality of reference motion information acquisition positions is selected for each motion information compression block and set as a representative motion information position.

  Furthermore, FIGS. 21A and 21B further illustrate an example in which the reference motion information acquisition position is made the same in each motion information compression block in the tree block. Since it is the same representative motion information position in all the motion information compression blocks, it is not necessary to switch the representative motion information position according to the position in the tree block. Further, the representative motion information position may be at any position such as the upper left end or upper right end in the motion information compression block other than FIGS. 21A and 21B.

  As an example of generating the representative motion information position, the representative motion information position may be indicated by using BlkIdx indicating the 4 × 4 block position in the motion information compression block in the Z scan order. When the size of the motion information compression block is 16 × 16, the representative motion information position shown in FIG. 21A corresponds to the position of BlkIdx = 12. Further, the representative motion information position shown in FIG. 21B corresponds to the position of BlkIdx = 15.

  As another example of the motion information compression process, the reference frame number may be included in the motion information compression process in order to reduce the memory capacity related to the reference frame number. In this case, the reference frame number held at the representative motion information position is stored in the memory capacity related to the reference frame number. Therefore, the spatial direction reference motion information memory 501 and the spatial direction reference motion information memory 502 shown in FIG. 5 store the reference frame number in addition to the motion vector information.

  As yet another example of the motion information compression processing, when the reference frame number is not included in the motion information compression processing, the motion vector information in the motion information at the representative motion information position is subjected to scaling processing using the reference frame number. Thus, it may be stored in the motion information memory 110. A typical example of the scaling process is a linear scaling process based on a reference frame number of zero. In this case, when the reference frame number is a value other than zero, linear scaling processing is performed so that the motion vector information refers to the reference frame corresponding to the reference frame number zero. The reference for the scaling process described above may be a value other than zero for the reference frame number. When division occurs when performing the above-described linear scaling processing, the division processing may be tabulated in advance and the division may be realized by subtracting the table each time.

  When the size of the motion information compression block is other than 16 × 16 blocks, the representative motion information position is generated using the same processing as described above. For example, when the size of the motion information compression block is 64 × 64, the reference motion information acquisition position when the size of the prediction unit is 64 × 64 is set as the representative motion information position. In yet another example, the representative motion information position obtained by scaling the representative motion information position in the motion information compression block size 16 × 16 block shown in FIGS. 21A and 21B etc. in the horizontal direction and the vertical direction according to the size of the motion information compression block. It does not matter as the information position.

  If the reference motion information does not exist because the representative motion information position is outside the picture or slice, the position where the reference motion information can be acquired within the motion information compression block, such as the upper left corner of the motion information compression block, is set as the new representative motion. It may be replaced as an information position. Also, when the representative motion information position is an area to which intra prediction is applied and no reference motion information exists, the same processing may be executed and replaced with a new representative motion information position.

Hereinafter, modifications of each embodiment will be listed and introduced.
In the first and second embodiments, an example is described in which a frame is divided into rectangular blocks of 16 × 16 pixel size and the like, and encoding / decoding is sequentially performed from the upper left block to the lower right side ( (See FIG. 2A). However, the encoding order and the decoding order are not limited to this example. For example, encoding and decoding may be performed sequentially from the lower right to the upper left, or encoding and decoding may be performed so as to draw a spiral from the center of the screen toward the screen end. Furthermore, encoding and decoding may be performed in order from the upper right to the lower left, or encoding and decoding may be performed so as to draw a spiral from the screen edge toward the center of the screen.

  In the first and second embodiments, the description has been given by exemplifying the prediction target block size such as the 4 × 4 pixel block, the 8 × 8 pixel block, and the 16 × 16 pixel block, but the prediction target block is a uniform block It does not have to be a shape. For example, the prediction target block (prediction unit) size may be a 16 × 8 pixel block, an 8 × 16 pixel block, an 8 × 4 pixel block, a 4 × 8 pixel block, or the like. Also, it is not necessary to unify all the block sizes within one coding tree unit, and a plurality of different block sizes may be mixed. When a plurality of different block sizes are mixed in one coding tree unit, the amount of codes for encoding or decoding the division information increases as the number of divisions increases. Therefore, it is desirable to select the block size in consideration of the balance between the code amount of the division information and the quality of the locally decoded image or the decoded image.

  In the first and second embodiments, for simplification, the luminance signal and the color difference signal are not distinguished from each other, and a comprehensive description is described regarding the color signal component. However, when the prediction process is different between the luminance signal and the color difference signal, the same or different prediction methods may be used. If different prediction methods are used between the luminance signal and the chrominance signal, the prediction method selected for the chrominance signal can be encoded or decoded in the same manner as the luminance signal.

  In the first and second embodiments, for simplification, the luminance signal and the color difference signal are not distinguished from each other, and a comprehensive description is described regarding the color signal component. However, when the orthogonal transformation process is different between the luminance signal and the color difference signal, the same or different orthogonal transformation methods may be used. If different orthogonal transformation methods are used between the luminance signal and the color difference signal, the orthogonal transformation method selected for the color difference signal can be encoded or decoded in the same manner as the luminance signal.

  In the first and second embodiments, syntax elements not defined in the embodiment can be inserted between the rows of the table shown in the syntax configuration, and other conditional branch descriptions are included. It does not matter. Alternatively, the syntax table can be divided and integrated into a plurality of tables. Moreover, it is not always necessary to use the same term, and it may be arbitrarily changed depending on the form to be used.

  As described above, each embodiment can realize highly efficient orthogonal transform and inverse orthogonal transform while alleviating the difficulty in hardware implementation and software implementation. Therefore, according to each embodiment, the encoding efficiency is improved, and the subjective image quality is also improved.

The instructions shown in the processing procedure shown in the above embodiment can be executed based on a program that is software. A general-purpose computer system stores this program in advance, and by reading this program, it is also possible to obtain the same effects as those obtained by the video encoding device and video decoding device of the above-described embodiment. is there. The instructions described in the above-described embodiments are, as programs that can be executed by a computer, magnetic disks (flexible disks, hard disks, etc.), optical disks (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD). ± R, DVD ± RW, etc.), semiconductor memory, or a similar recording medium. As long as the recording medium is readable by the computer or the embedded system, the storage format may be any form. If the computer reads the program from the recording medium and causes the CPU to execute instructions described in the program based on the program, the computer is similar to the video encoding device and video decoding device of the above-described embodiment. Operation can be realized. Of course, when the computer acquires or reads the program, it may be acquired or read through a network.
In addition, the OS (operating system), database management software, MW (middleware) such as a network, etc. running on the computer based on the instructions of the program installed in the computer or embedded system from the recording medium implement this embodiment. A part of each process for performing may be executed.
Furthermore, the recording medium in the embodiment of the present invention is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium in which a program transmitted via a LAN or the Internet is downloaded and stored or temporarily stored. Further, the program for realizing the processing of each of the above embodiments may be stored on a computer (server) connected to a network such as the Internet and downloaded to the computer (client) via the network.
Further, the number of recording media is not limited to one, and even when the processing in this embodiment is executed from a plurality of media, it is included in the recording medium in the embodiment of the present invention, and the configuration of the media is any configuration. Good.

Note that the computer or the embedded system in the embodiment of the present invention is for executing each process in the present embodiment based on a program stored in a recording medium, and is an apparatus composed of one of a personal computer, a microcomputer, and the like. Any configuration such as a system in which a plurality of devices are network-connected may be used.
Further, the computer in the embodiment of the present invention is not limited to a personal computer, but includes an arithmetic processing device, a microcomputer, and the like included in an information processing device, and a device capable of realizing the functions in the embodiment of the present invention by a program, The device is a general term.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 100 ... Image coding apparatus, 101 ... Subtraction part, 102 ... Orthogonal transformation part, 103 ... Quantization part, 104, 2502 ... Inverse quantization part, 105, 2503 ... Inverse orthogonal transformation part, 106, 2504, 2706 ... Addition part 107, 2505 ... reference image memory, 108, 2506 ... inter prediction unit, 109 ... motion information compression unit, 110 ... motion information memory, 112 ... entropy coding unit, 113 ... output buffer, 114 ... coding control unit, 401 ... parameter encoding unit, 402 ... transform coefficient encoding unit, 403 ... motion information encoding unit, 404 ... multiplexing unit, 901 ... reference motion vector acquisition unit, 902 ... prediction motion vector selection switch, 903 ... subtraction unit, 904 ... differential motion information encoding unit, 905 ... predictive motion information position encoding unit, 906 ... multiplexing unit, 2500 ... moving picture decoding device, 250 ... entropy decoding unit, 2507 ... reference motion information memory, 2508 ... reference motion information compression unit, 2510 ... decoding control unit, 2601, 2701 ... separation unit, 2602 ... parameter decoding unit, 2603 ... transform coefficient decoding unit, 2604 ... motion information decoding unit, 2702 ... differential motion information decoding unit, 2503 ... predicted motion information position decoding unit, 2704 ... reference motion information acquisition unit, 2705 ... predicted motion information selection switch.

Claims (3)

In a moving image encoding method for performing inter prediction on a pixel block divided from an input image signal,
Perform scaling processing according to the reference frame information included in the predicted motion information candidate,
Based on the first information encoded got out of a plurality of motion information in the completed area, select the predicted motion information from the predicted motion information candidates, sign-current block using the predicted motion information A video encoding method for predicting motion information of
The moving image coding method, wherein the first information includes second information for specifying a position in the coding target block for selecting the predicted motion information. In a video encoding device that performs inter prediction on a pixel block divided from an input image signal,
Performs scaling processing in accordance with information of the reference frame included in the prediction motion information candidates, based on the first information obtained from a plurality of motion information of the encoding is completed within regions predicted from the predicted motion information candidate motion a prediction unit which selects the information, predicting motion information of sign-current block using the predicted motion information,
Comprising
The moving image encoding apparatus, wherein the first information includes second information for specifying a position in the encoding target block for selecting the predicted motion information.   A computer program for causing a computer to execute a moving image encoding method for performing inter prediction on a pixel block divided from an input image signal,
  A procedure for performing scaling processing according to the information of the reference frame included in the predicted motion information candidate,
  First information obtained from a plurality of pieces of motion information in a region where encoding has been completed, and includes second information for specifying a position in a block to be encoded for selecting the predicted motion information. A step of selecting prediction motion information from the prediction motion information candidates based on one information, and predicting motion information of the coding target block using the prediction motion information;
A computer program for causing the computer to execute.

Images