JP5946980B1 - Image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image coding technique for reducing a code amount. In inter prediction in which motion search is performed using a block obtained by dividing an input image and a reference image, a motion vector of the block is selected from motion vectors of surrounding blocks, predicted, and a difference vector is calculated and coded. In this method, adaptive and efficient video compression is performed by changing the number or position of candidate blocks for selecting a prediction vector in units of blocks or frames. [Selection] Figure 1

Description

  The present invention relates to an image encoding method for encoding an image, an image encoding device, an image decoding method for decoding encoded image data, and an image decoding device.

  The H.264 / AVC (Advanced Video Coding) standard has been established as a method for recording and transmitting image and audio information as digital data. ISO / IEC MPEG and ITU-T VCEG have established JCT-VC (Joint Collaborative Team on Video Coding) in order to achieve compression ratios exceeding this, and are considering the next generation method called HEVC (High Efficiency Video Coding). I'm starting.

  As one of the technical candidates for HEVC, a method called motion vector competition has been proposed. This method proposes a prediction vector selection method for a motion prediction mode in an encoding target macroblock. In H.264 / AVC, the prediction vector is obtained by performing a median operation on the X-direction component and the Y-direction component from the motion vectors of the encoded macroblocks above, left, and upper right of the encoding target macroblock. It was.

  On the other hand, in motion vector competition, the motion vectors at the same position as the encoding target macroblocks of the top, left, and nearest reference frames are compared, and the closest one to the motion vector of the encoding target macroblock is selected as the prediction vector. Transmitted as a flag.

I. Amonou, et. Al. "Description of video coding technology proposal by France Telecom, NTT, NTT DOCOMO, Panasonic and Technicolor", Doc. JCTVC-A114, Joint Collaborative Team on Video Coding (JCT-VC), April 2010.

  However, since the motion vector competition method transmits a flag, if the result of the median operation and the prediction vector are the same, the amount of code increases by the amount of the flag.

  Also, it is not effective for an encoding mode that uses all motion vectors.

  Furthermore, when the block to be referenced does not have a motion vector, the processing when it is outside the frame region and the flag description method are not efficient, and the flag is recorded by selecting two from three candidates. However, only candidate blocks in a narrow range can be selected, there is a problem that the prediction error is large, the code amount is large, and the coding efficiency is low.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image encoding technique and an image decoding technique that can realize high encoding efficiency by appropriate motion prediction.

  The present application includes various means for solving the above-mentioned problems. One of them is as follows.

  An image encoding device that encodes an input image, a motion search unit that performs a motion search using an encoding target block obtained by dividing the input image and a reference image, and a motion vector of the encoding target block, An adaptive PMV selector that generates a motion vector predictor from motion vectors of candidate blocks consisting of some or all of the blocks around the current block, and a difference vector is calculated from the motion vector of the current block and the motion vector predictor. A PMV predictive encoding unit for encoding, wherein the adaptive PMV selection unit includes first means for changing a number or position of candidate blocks to be used or setting a motion vector of the candidate block to be used as a zero vector. An image encoding device.

  According to the present invention, encoding efficiency can be improved.

1 is a block diagram of an image encoding device according to Embodiment 1. FIG. It is a block diagram of the image decoding apparatus which concerns on Example 2. It is explanatory drawing of the candidate block which selects a motion vector as PMV (predictive motion vector) which concerns on Example 1. FIG. It is explanatory drawing about the outline | summary of the scene analysis which concerns on Example 1. FIG. FIG. 3 is an image encoding flowchart used in the image encoding apparatus according to the first embodiment. It is a flowchart of the image decoding method which concerns on Example 2. 10 is an explanatory diagram of a data recording medium according to Embodiment 3. FIG. It is a figure which shows the PMV selection determination flow per frame in the adaptive PMV selection part which concerns on Example 1. FIG. It is a figure which shows the PMV selection determination flow per block in the adaptive PMV selection part which concerns on Example 1. FIG. It is a figure which shows the adjustment flow of the candidate block which performs PMV selection per block in the adaptive PMV selection part which concerns on Example 1. FIG. It is a figure which shows the example of the description method as a flag of the vector selected concerning Example 1. FIG.

  Hereinafter, examples will be described with reference to the drawings.

  Moreover, in each drawing, the component to which the same code | symbol was attached | subjected shall have the same function.

  In addition, the expression “motion vector competition” in each description and each drawing of the present specification indicates a conventional encoding method, and the expression “adaptive motion vector selection” indicates a new encoding method according to the present invention. “PMV selection” means to select one of motion vectors of candidate blocks and generate a predicted motion vector (Predicted Motion Vector: PMV) from the selected motion vector (selecting median or the like as it is) And new generation by combining a plurality of motion vectors, etc.).

  Further, “0 vec” or “0 vector” in each description and each drawing of this specification indicates that a value of each component is a vector of 0, or conversion into such a vector and setting.

  In addition, “non-referenceable” in each description and each drawing of the present specification indicates that block information cannot be acquired. As a specific example, the position of the candidate block is outside the range of the screen. Conversely, “can be referred to” means that block information can be acquired.

  In addition, “block information” in each description and each drawing of this specification includes a vector and a reference frame number.

  In addition, the expression “residual component” in each description and each drawing of the present specification includes the same meaning as “prediction error”.

  In addition, the expression “region” in each description and each drawing of this specification includes the same meaning as “image”.

  In addition, the expression “transmitted with a flag” in each description and each drawing of the present specification also includes the meaning of “transmitted in a flag”.

  In addition, the expression “encoding mode” in each description and each drawing of the present specification includes a combination of a type of intra prediction / inter prediction and a block size to be applied.

First, a first embodiment will be described with reference to the drawings.
FIG. 1 is a block diagram of an image encoding apparatus according to the first embodiment.

  The image coding apparatus includes a block division unit 101, a prediction error calculation unit 102, an intra prediction unit 104, a PMV prediction coding unit 107, an adaptive PMV selection unit 106, a motion search unit 105, a decoded image memory 103, a DCT conversion unit 108, A quantization unit 109, an entropy encoding unit 110, an inverse quantization unit 111, an inverse DCT conversion unit 112, a mode selection unit 113, and a data output unit 114 are provided.

  The operation of each component of the image encoding device will be described in detail below.

  The operation of each component of the image encoding device may be an autonomous operation of each component as described below, for example. Alternatively, the computer control unit may be realized in cooperation with software stored in the computer storage unit.

  First, the block dividing unit 101 inputs an original image to be encoded, and divides it into blocks that are encoding units. The size of the block to be divided here is a power-of-two size such as 16 × 16 pixels called Coding Unit (CU) or Prediction Unit (PU) (a block of m pixels × n pixels, a power of m, n = 2) , Eg 128 × 128, 64 × 64, 32 × 32, 16 × 16, 16 × 8, 8 × 16, 8 × 8, 4 × 4), but the size is variable and of different sizes May be next to each other. Hereinafter, a case where blocks of the same size continue will be described as an example, and a vector selection method when the sizes are different will be described separately.

  The prediction error calculation unit 102 calculates the difference component between the generated prediction image and the original image, and outputs it. As prediction methods, there are generally intra (intra-frame) prediction and inter (inter-frame) prediction. Intra prediction is performed by the intra prediction unit 104, and inter prediction is performed by the inter prediction unit 120 including the PMV prediction encoding unit 107, the adaptive PMV selection unit 106, and the motion search unit 105. Intra prediction uses the information of the same frame encoded before the encoding target block, and inter prediction uses the information of the previous or subsequent frame encoded before the encoding target frame as the reproduction time. Use. These are stored in the decoded image memory 103.

  Here, although only one intra prediction unit 104, motion prediction unit, and decoded image memory 103 are shown for explanation, they may be provided for each encoding mode and for each frame.

  The intra prediction unit 104 predicts pixels of the encoding target block using information of the same frame encoded before the encoding target block. As an intra prediction method, an existing method may be used.

  For the motion prediction unit 120, first, the motion search unit 105 searches for a motion vector (Motion Vector: MV) using the reference frame and the encoding target block stored in the decoded image memory 103. The accuracy of the pixels may be an integer or a decimal, and 1/4 pixel accuracy is preferable by interpolation of the reference frame by a filter, but 1/2 or 1/8 may be used. In this embodiment, an existing method is used for the motion search method.

  Next, the adaptive PMV selection unit 106 uses the surrounding block in the same frame of the encoding target block and the same position of the reference frame and the information of the surrounding block to adaptively determine the PMV by temporal and spatial discrimination. Select In this embodiment, the PMV is adaptively discriminated with both temporal discrimination and spatial discrimination, but only one of them may be used. Details of the method, the candidate surrounding blocks, and the PMV selection method will be described later.

  An example of a candidate block for selecting a PMV will be described with reference to FIG. In the existing motion vector competition method, as in 301, the left reference block (A) and the upper reference block (L0 [0]) closest to the upper block (B) in the same encoding target frame as the encoding target block. The block (Co-located Block) (S) at the same position as the current block to be encoded is taken as a candidate, arranged in order as A, B, S, and if there is the same vector among these, it is excluded from the candidate Thus, the one closest to the vector of the encoding target block is selected and indicated by a flag. Although the nearest forward reference frame is adopted in this embodiment, a backward reference frame may be used as long as it is an encoded frame. Further, not only one frame but also a plurality of encoded frames may be referred to.

On the other hand, in the adaptive motion vector selection method used in the adaptive PMV selection unit of the present embodiment, not only the left side and the upper side of the encoding target block as in 302, but also the upper right (C) and upper left (D) are included in the candidates. In addition, blocks surrounding the block S of the reference frame can be included in the candidates. That is, (a) number, (b) spatial position (block in a frame) and (c) temporal position (block in another frame) of candidate blocks used by the adaptive PMV selection unit can be changed. Like that.
Details of the selection method will be described later.

  Further, in the adaptive motion vector selection method of the present embodiment, when the size of the candidate block is different as in 303, if the block at the position to be referred to first does not have a motion vector or cannot be referenced, the neighboring blocks are sequentially It is checked whether the block has a motion vector, and if it is available, it is a candidate for PMV selection. Specifically, if B1 cannot be referenced in 303, B2, B3, and B4 are sequentially checked. Similarly, when A1 cannot be referred to, A2, A3 and A4 are checked. When S1 cannot be referred to, S2, S3 and S4 are checked. Scanning may be performed in order from the top left of the encoding target frame. For block B, left to right, bottom to top, for block A, top to bottom, right to left, for block S, normal raster scan order, left to right, top to bottom It becomes the order of.

  Next, the PMV predictive encoding unit 107 calculates a differential vector (Differential Motion Vector: DMV) by subtracting PMV from MV and encodes it. In addition, motion prediction is performed from the reference frame using MV to create a prediction block.

  In the DCT conversion unit 108, DCT conversion is performed on the residual component input from the prediction error calculation unit 102 and converted into a coefficient. Then, it is sent to the quantization part 109 and each transform coefficient is quantized. The quantized coefficients are sent to the entropy coding unit 110, information compression using a variable length code or the like is performed according to a certain order, and the data is sent to the mode selection unit 113. On the other hand, the quantized coefficient is also sent to the inverse quantization unit 111, returned to the transform coefficient by inverse quantization, and further restored to the residual component by the inverse DCT transform unit 112. The residual component is added with information from the intra prediction unit 104 and the series of motion prediction units, returns to the decoded image, and is stored in the decoded image memory 103. The decoded image is input to the mode selection unit 113 and used for mode selection.

  Next, the mode selection unit 113 selects a mode with the highest coding efficiency for the block based on information on a plurality of coding modes. For each coding mode, an error (coding error) between the original image and the decoded image and a code amount generated when the coding mode is used are calculated, and an appropriate coding mode is selected from the relationship between the two. For example, an existing method such as a Rate-Distortion optimization method may be used as a method for selecting an encoding mode with the highest encoding efficiency. This method is a method for calculating the coding error between the code amount and the original image after decoding for all modes for each macroblock, and selecting the best mode according to the cost calculation formula.

  Finally, the data output unit 114 outputs the selected code to create an encoded stream, and the image is encoded.

  Next, an image coding flow used in the image coding apparatus will be described with reference to FIG.

  First, step 501 is processing for inputting an original image to be encoded from the decoded image memory 103 and dividing it into blocks. Here, the size of each block is variable. The aforementioned CU and PU may be used. Step 501 is executed by the block dividing unit 101.

  Next, in step 502, intra prediction is performed on the block of the original image acquired in step 501. An existing method may be used for intra prediction, and prediction is performed for a plurality of modes according to each intra prediction mode. The processing in step 502 is executed in the intra prediction unit 104.

  Next, in step 503, residual components are acquired for pixels of the intra-predicted coding block, and DCT transform processing, quantization processing, and entropy coding processing of the residual components are performed to calculate encoded data. . Furthermore, entropy decoding processing, inverse quantization, and inverse DCT conversion processing are performed on the encoded data to create a decoded image. As a result, the encoded data and the decoded image are output. The process of step 503 is performed in this order in the prediction error calculation unit 102, the DCT conversion unit 108, the quantization unit 109, and the entropy encoding unit 110.

  In step 504, scene analysis is performed. The outline of the scene analysis will be described with reference to FIG. In a moving image sequence, there may be a change in image characteristics even within one GOP (Group of Pictures). For example, there may be a scene change in a movie or drama, a camera change in a television broadcast, or the like. FIG. 4 shows a case where switching between scene 1 and scene 2 occurs between the P2 frame and the B3 frame. Here, in the existing motion vector competition method, the P4 and B3 frames refer to the motion vector of the P2 frame, but an effective motion vector cannot be acquired because the scenes are different. Therefore, in this embodiment, a plurality of decoded images are analyzed to find a scene change without using a flag, or as a result of scene analysis on the encoder side, the detected scene change position is transmitted to the decoder by a flag, When the conversion target frame and the reference frame belong to different scenes, it is possible to perform a process of not making the block a PMV candidate block. Such scene analysis processing may be performed in units of frames or in units of blocks. This process is not shown as a clear block, but is executed before the motion search by the motion search unit 105 in the inter prediction unit 120.

  Next, in step 505, motion search is performed for each inter prediction mode. Inter prediction modes include motion prediction modes with different block sizes, Skip mode, Direct mode, and the like. An existing method may be used as the motion search method. This step 505 is executed by the motion search unit 105 in the inter prediction unit 120.

  Next, in step 506, it is determined whether or not PMV selection is performed in units of frames. If PMV selection is not performed, the process proceeds to step 510; otherwise, the process proceeds to step 507. Subsequently, in step 507, it is determined whether or not PMV selection is performed in block units. If PMV selection is not performed, the process proceeds to step 510; otherwise, the process proceeds to step 508. Further, in step 508, PMV candidate blocks are adjusted to determine which blocks are included in candidate blocks and which blocks are not included depending on conditions. These processes can be switched in units of frames and blocks, and such switching may be transmitted to the decoding device side by a flag, or the same switching condition may be held in the decoding device.

  Details of PMV selection determination in frame units will be described with reference to FIG. Step 801 determines whether or not to perform PMV selection by determining whether or not a scene change has occurred. As described above, the scene analysis process may be performed on the decoder side as described above to determine whether to perform the PMV selection process without transmitting the flag, or the determination on the encoder side may be determined on the decoder side by the flag. You may judge it. When PMV selection is not performed, a specific method, such as a method for obtaining PMV by median calculation or a method in which the left or upper block vector is set to PMV, is determined in advance.

  As another example, in step 802, it is determined whether to perform PMV selection by determining the picture type. For example, if the encoding target frame is a P picture, motion vector prediction by PMV selection is performed. Otherwise, other means such as median calculation are used. It may be a B picture instead of a P picture. You may provide the flag which discriminate | determines how to judge at the time of which picture.

  Details of PMV selection determination in block units will be described with reference to FIG. Step 901 determines whether or not to perform PMV selection by determining the type of encoding mode of the encoding target block. For example, PMV selection may be performed only for a specific block size, or PMV selection may not be applied to Skip mode or Direct mode.

  As another example, in step 902, it is determined whether or not to perform PMV selection by determining the type of encoding mode of the candidate block. For example, if all candidate blocks are in the intra coding mode, PMV selection is not performed.

  With the above method, it is possible to switch whether or not PMV selection is performed in units of frames or blocks.

  Details of a method for adjusting candidate blocks for performing PMV selection in units of blocks will be described with reference to FIG. In this case, when PMV selection is performed, processing such as exclusion from candidate blocks or addition of other candidate blocks is performed depending on whether or not the encoding mode and vector of the candidate block are present and whether or not reference is possible.

  First, in step 1001, one candidate block of the encoding target frame, for example, one block not checked yet is selected from A and B in FIG. This is block X.

  Next, in step 1002, it is checked whether the block X can be referred to. If reference is impossible, the process proceeds to step 1003, and if reference is possible, the process proceeds to step 1004. Unreferable refers to a state in which block information cannot be acquired because the block position is out of the screen range, etc., and referable means that block information can be acquired. The block information includes a vector and a reference frame number. Is included.

  In step 1003, the block X is set to 0 vector or excluded from the candidate blocks. Which one to use is predetermined. For example, in step 1003, the zero vector is set. A “0 vector” is a vector in which the value of each component is 0.

  In step 1004, it is checked whether or not the block X has a vector. If it is an intra-coded block, it has no vector. If it has no vector, the process proceeds to step 1005. If it has a vector, the process proceeds to step 1006.

  In step 1005, the block X is set to 0 vector or excluded from the candidate blocks. Which one to use is predetermined. For example, in step 1005, the block is excluded from the candidate blocks.

  In step 1006, it is checked whether the reference frame numbers of the block X and the encoding target block match. For example, when the block to be encoded is unidirectional prediction and the block X is bidirectional prediction, either L0 or L1 may match. Further, when the encoding target block is bidirectional prediction and the block X is unidirectional prediction, there is a mismatch. If they do not match, the process proceeds to step 1007, and if they match, the process proceeds to step 1008.

  In step 1007, the block X is set to 0 vector or excluded from the candidate blocks. Which one to use is predetermined. For example, in step 1007, the block is excluded from the candidate blocks.

  In step 1008, it is determined whether there is another candidate block to be checked in the encoding target frame. If there is another block to be checked, the process returns to step 1001. Otherwise, go to step 1009.

  In step 1009, one candidate block of the reference frame, for example, one block not checked yet is selected from S in FIG. This is block Y.

  In step 1010, it is checked whether the block Y has a vector. If it is an intra-coded block, it has no vector. If it has no vector, the process proceeds to step 1011. If it has a vector, the process proceeds to step 1012.

  In step 1011, the block Y is set to 0 vector or excluded from candidate blocks. Which one to use is predetermined. For example, in step 1011, it is excluded from the candidate blocks.

  In step 1012, it is checked whether the reference frame numbers of the block Y and the encoding target block match. For example, when the block to be encoded is unidirectional prediction and the block Y is bidirectional prediction, either L0 or L1 may match. Further, when the encoding target block is bidirectional prediction and the block Y is unidirectional prediction, there is a mismatch. If they do not match, the process proceeds to step 1013, and if they match, the process proceeds to step 1014.

  In step 1013, the block Y is set to 0 vector or excluded from the candidate blocks. Which one to use is predetermined. For example, in step 1013, the block is excluded from the candidate blocks.

  In step 1014, it is checked whether the encoding target block or the block Y is in the bidirectional prediction mode. If either is the bidirectional prediction mode, the process proceeds to step 1015; otherwise, the process proceeds to step 1016.

  In step 1015, the block Y is set to 0 vector or excluded from the candidate blocks. Which one to use is predetermined. For example, in step 1015, it is excluded from the candidate blocks.

  In step 1016, it is determined whether there is another candidate block to be checked in the reference frame. If there is another block to be checked, the process returns to step 1009.

  Here, in steps 1003, 1005, 1007, 1011, 1013, and 1015 corresponding to each condition, a process of setting the block X or the block Y to 0 vector or excluding from the candidate block was performed. On the other hand, for example, blocks at positions C and D in FIG. 3 may be added to the candidate blocks as candidate blocks. For example, among S, A, and B, if the candidate block is 0, C and D are added to the candidate block, and if it is 1, C is added to the candidate block. C, D, and other blocks may be added to the candidate block in advance, and a vector may be selected therefrom.

  For example, when the picture of L0 [0] is intra-coded, S may be excluded from the candidate blocks and C may be added instead. Alternatively, block C may be used when block A cannot be referenced.

As to whether or not to perform PMV selection as described above, or as a candidate block adjustment method, a method predetermined by a flag may be switched for each frame or block. That is, there can be the following methods.
・ Unify the method to any method ・ Indicates which method can be switched by frame and selected by flag ・ Indicates which method can be switched by block and selected by flag The conditions shown in FIG. There are the following, which may change the order of determination.
・ Whether the candidate block cannot be referenced ・ Whether the candidate block has a motion vector ・ Whether the candidate block and the encoding target block are different in encoding type (for example, the candidate block is encoded by bidirectional prediction (B prediction)) Target block is one-way prediction (P prediction))
-Whether the reference frame number of the candidate block and the encoding target block match-Whether the candidate block and the encoding target block are bidirectional prediction The methods for adjusting the candidate block include the following.
・ Exclude candidate block from candidate block ・ Process vector of candidate block as 0 vector ・ Add another block in encoding target frame to candidate block ・ Add another block in reference frame to candidate block 3. Add another block shown at 303 in the process to the candidate block. Process only when the encoding type matches, exclude it from the candidate block. Process only when the reference frame number matches, otherwise process from the candidate block. Exclude / Apply the vector that matches the reference frame number to the other vector and perform both processing. Subsequently, in step 509, the PMV is selected and the difference vector is calculated and encoded. The PMV selects a vector closest to the vector of the encoding target block from the candidate block vectors. A vector with the smallest code amount is selected when each component of the vector is encoded. The selected vector is transmitted as a flag. In this case, the candidate blocks are arranged in order, and if it is 1 bit, the vector of the first candidate block is selected when it is 0, and the vector of the next candidate block is selected when it is 1. For example, when the A, B, and S blocks match, the candidate blocks are ordered in the order of A, B, and S. If the same vector is found, it is excluded and the two blocks from the top are arranged and compared. Will be.

  The case where the selected vector is indicated by 2 bits is shown below. Assume that there are four candidate blocks A, B, S, and C shown in FIG. Similar to the above example, a vector closest to the vector of the encoding target block is selected, and the code is transmitted in the form of A: 0, B: 01, C: 10, S: 11. For example, if any of these is excluded from the candidate blocks, the order of the blocks is increased. The number of bits indicating the selected vector may be switched in units of frames or blocks. In that case, a flag indicating switching is transmitted separately.

  FIG. 11 shows an example of a method for describing the code of the vector selected. Let P be the candidate block number of the current previous encoding target block, and N be the number of candidate blocks. Let X be the candidate block number of the current encoding target block. If the current encoding candidate block number is P, 0 is set as the code C. Otherwise, output 1 first, and then continue to output (X-P-1)% N. On the contrary, when the code C is 0, the candidate block number X of the current encoding target block is the same as the previous candidate block number P. On the other hand, when the first bit of the code C is 1, X = (P + C ^ + 1)% N with the remaining code excluding the first bit as C ^. In this way, the code amount can be shortened when the same candidate block number is selected successively.

  Note that the processing from step 506 to step 509 is executed by the adaptive PMV selection unit 106.

Next, in step 510, PMV calculation and vector encoding are performed. Here, the PMV is not selected, and the PMV is calculated by a normal median operation. In this method, median operation is performed on each component of the left A, upper B, upper right C, or upper left D vector to determine PMV. After the PMV is determined, a difference vector from the vector of the encoding target block is calculated and encoded.
The processing in step 510 is executed by the PMV predictive encoding 107.

  In step 511, motion prediction is performed for each inter prediction mode, a residual component is obtained for the pixels of the encoded block, DCT transform processing, quantization processing, and entropy encoding processing of the residual component are performed, and encoded data Is calculated. Each process so far included in step 511 is executed by DCT transform section 108, quantization section 109, and entropy coding section 110. Furthermore, entropy decoding processing, inverse quantization, and inverse DCT conversion processing are performed from the encoded data to create a decoded image. As a result, the encoded data and the decoded image are output. Each process included in step 511 so far is performed by the inverse quantization unit 111 and the inverse DCT transform unit 112.

  Next, in step 512, the image encoding results for the respective encoding modes are compared according to the processing results of step 503 and step 511, and the encoding mode to be output for the block is determined. The process in step 512 is executed by the mode selection unit 113.

  Next, in step 513, the encoded data in the encoding mode selected in step 512 is output as an encoded stream. The process of step 513 is executed by the data output unit 114.

  As described above, the encoding process according to the first embodiment is performed.

  According to the image coding apparatus and the image coding method according to the first embodiment described above, inter prediction with higher accuracy and less code amount than the existing coding method is possible, and the compression efficiency is higher than that of the existing method. A high image encoding device and image encoding method can be realized.

  In addition, the image coding apparatus and the image coding method according to the first embodiment can be applied to a recording apparatus, a mobile phone, a digital camera, and the like using these.

  According to the image encoding device and the image encoding method according to the first embodiment described above, it is possible to reduce the code amount of the encoded data and prevent deterioration of the image quality of the decoded image when the encoded data is decoded. It becomes possible. That is, a high compression rate and better image quality can be realized.

  Next, FIG. 2 shows an example of a block diagram of an image decoding apparatus according to the second embodiment.

  The image decoding apparatus according to the present embodiment includes a stream analysis unit 201, a mode determination unit 202, a coefficient analysis unit 203, an intra prediction synthesis unit 204, an adaptive PMV selection decoding unit 205, a motion prediction synthesis unit 206, an inverse quantization unit 207, and an inverse. A DCT conversion unit 208, a decoded image memory 209, and an image output unit 210 are provided.

  The operation of each component of the image decoding device will be described in detail below. The operation of each component of the image decoding device may be an autonomous operation of each component as described below, for example. Further, for example, it may be realized by cooperating with software stored in a control unit or storage unit of a computer.

  First, the stream analysis unit 201 analyzes the input encoded stream. Here, the stream analysis unit 201 also performs data extraction processing from packets and information acquisition processing of various headers and flags. Further, each block is processed.

  At this time, the encoded stream input to the stream analysis unit 201 is an encoded stream generated by the image encoding device and the image encoding method according to the first embodiment. Since the generation method is as shown in the first embodiment, the description is omitted. It may be an encoded stream read from the data recording medium shown in the third embodiment. The recording method will be described later.

  Next, the mode determination unit 202 determines the encoding mode designated by a flag or the like for each block. In the following decoding process, a process corresponding to the encoding mode of the determination result is performed. The process for each encoding mode will be described below.

  First, when the coding mode is intra coding, the intra prediction combining unit 204 combines the intra prediction and the predicted image. A conventional method may be used for this method. Here, the intra prediction combining unit 204 outputs the combined predicted image.

  When the encoding mode is encoding by inter prediction between frames, motion prediction and prediction image synthesis are performed. First, the adaptive PMV selective decoding unit 205 adaptively selects a PMV and combines it with the decoded difference vector to restore a motion vector. The PMV selection method is the same as that described in the first embodiment because the encoder and decoder operate so as to select the same PMV.

  Subsequently, the motion prediction synthesis unit 206 performs motion prediction using the reference frame for the pixels in the block using the restored motion vector. Here, the motion prediction synthesis unit 206 outputs the synthesized predicted image.

  On the other hand, the coefficient analysis unit 203 analyzes the encoded data of each macroblock included in the input encoded stream, and outputs the encoded data of the residual component. At this time, processing corresponding to the encoding mode of the determination result of the mode determination unit 202 is performed.

  The inverse quantization unit 207 performs inverse quantization processing on the encoded data, and sends this to the inverse DCT transform unit 208. The inverse DCT transform unit 208 performs inverse DCT transform on the inversely quantized coefficient to restore a residual component.

  The residual component of each coding mode restored as described above is added to the prediction image output from the intra prediction synthesis unit 204 or the motion prediction synthesis unit 206, and returns to the decoded image, and is stored in the decoded image memory 209. Stored. The decoded image memory 209 stores information on the currently decoded frame and information on previously decoded frames, and is referred to by the intra prediction synthesis unit 204 and the motion prediction synthesis unit 206.

  The last decoded image is output by the image output unit 210, and the image is decoded.

  Next, the flow of the image decoding method in the image decoding apparatus according to the second embodiment will be described with reference to FIG.

  First, in step 601, an encoded stream to be decoded is acquired. Next, in step 602, the encoding mode flag and the encoded data included in the encoded stream acquired in step 601 are analyzed. Steps 601 and 602 are executed by the stream analysis unit 201.

  Next, in step 603, the encoding mode for one encoding unit (block unit, pixel unit, etc.) included in the encoded data is determined using the encoding mode flag analyzed in step 602. If the intra coding mode is selected, the process proceeds to step 604. If the intra coding mode is selected, the process proceeds to step 605. This step 603 is executed by the mode determination unit 202.

  In step 604, intra prediction is performed according to the method specified by the encoding mode. For this, an existing intra prediction method may be used. This step 604 is executed by the intra prediction synthesis unit 204.

  In step 605, it is determined whether or not PMV selection is to be performed for the frame in accordance with various frame conditions and flag information. The determination as to whether or not to perform PMV selection for each frame is the same as the method described in the first embodiment because the encoder and the decoder operate so as to perform the same processing. If PMV selection is not performed, the process proceeds to step 609. Otherwise, the process proceeds to step 606. In step 606, it is determined whether or not PMV selection is to be performed for the block according to the encoding mode, conditions for each block, and flag information. The determination as to whether or not to perform PMV selection in block units is the same as the method described in the first embodiment because the encoder and decoder operate so as to perform the same processing. If PMV selection is not performed, the process proceeds to step 609; otherwise, the process proceeds to step 607.

  In step 607, the PMV candidate block is adjusted. Here, it is determined which block is included in the candidate block and which block is not included depending on the condition. The adjustment of the PMV candidate block is the same as the method described in the first embodiment because the encoder and the decoder operate so as to perform the same processing.

  In step 608, PMV selection and vector decoding are performed. The PMV selection method is the same as the method described in the first embodiment because the encoder and the decoder operate so as to perform the same processing. In the vector decoding, the PMV and the decoded difference vector are combined to restore the motion vector of the encoding target block.

  Next, in step 609, PMV calculation and vector decoding are performed. The PMV calculation method is the same as the method described in the first embodiment because the encoder and the decoder operate so as to perform the same processing. In the vector decoding, the PMV and the decoded difference vector are combined to restore the motion vector of the encoding target block. These step 605, step 606, step 607, step 608 and step 609 are executed by the adaptive PMV selective decoding unit 205.

  In step 610, inter prediction is performed using the restored vector and the reference frame, and a predicted image is synthesized. According to the determined coding mode, if the intra prediction synthesis unit 204 and the motion prediction synthesis unit 206 of the image decoding apparatus in FIG. 2 perform the prediction image synthesis process in each corresponding coding mode. Good. This step 610 is executed by the motion prediction synthesis unit 206.

  In step 611, the predicted image and the residual component are combined to generate a decoded image. First, the encoded data is analyzed, the inverse quantization process and the inverse DCT transform process are performed on the encoded data, and the residual component for the one encoding unit is decoded. A predicted image is synthesized with this to generate a decoded image, which is stored in a memory. Finally, in step 612, the generated decoded image is output. Each processing of step 611 is executed by the coefficient analysis unit 203, the inverse quantization unit 207, the inverse DCT conversion unit 208, the decoded image memory 209, and the image output unit 210.

  In this embodiment, besides the example, an encoded stream that is defined by subdividing each encoding mode using a block size used in the encoding mode as a parameter may be set as a decoding target stream.

  According to the above-described image decoding apparatus and image decoding method according to the second embodiment, inter prediction with higher accuracy and less code amount than the existing encoding method is possible, and the image has higher compression efficiency than the existing method. A decoding device and an image decoding method can be realized.

  In addition, the image decoding apparatus and the image decoding method according to the second embodiment can be applied to a reproduction apparatus, a mobile phone, a digital camera, and the like using these.

  According to the image decoding apparatus and the image decoding method according to the second embodiment described above, it is possible to decode encoded data with a small code amount with higher image quality.

  Next, FIG. 7 shows a data recording medium in which an encoded stream generated by the image encoding device or the image encoding method according to the first embodiment is stored.

  The encoded stream according to the present embodiment is an encoded stream generated by the image encoding device or the image encoding method according to the first embodiment. Since the generation method is as shown in the first embodiment, the description is omitted.

  Here, the encoded stream according to the present embodiment is recorded as a data string 702 on the data recording medium 701. The data string 702 is recorded as an encoded stream according to a predetermined grammar. The following description assumes that a part of the H.264 / AVC standard has been changed.

  First, in H.264 / AVC, a stream is composed of a sequence parameter set 703, a picture parameter set 704, and slices 705, 706, and 707. Hereinafter, a case where one image (picture) is stored in one slice will be described.

  Each slice includes information 708 related to each block. Within the block-related information 708, there is an area for recording each coding mode for each block, and this is used as a coding mode flag 709.

  Since the data recording medium according to the third embodiment described above is recorded with the encoded stream having higher compression efficiency than the existing method, many images can be recorded.

  It should be noted that any of the embodiments described above, such as each figure and each method, may be combined.

DESCRIPTION OF SYMBOLS 101 ... Block division part, 102 ... Prediction error calculation part, 103 ... Decoded image memory, 104 ... Intra prediction part, 105 ... Motion search part, 106 ... Adaptive PMV selection part, 107 ... PMV prediction encoding part, 108 ... DCT conversion 109: Quantization unit, 110 ... Entropy encoding unit, 111 ... Inverse quantization unit, 112 ... Inverse DCT transform unit, 113 ... Mode selection unit, 114 ... Data output unit, 120 ... Inter prediction unit 201 ... Stream analysis 202: Mode determination unit, 203 ... Coefficient analysis unit, 204 ... Intra prediction synthesis unit, 205 ... Adaptive PMV selection decoding unit, 206 ... Motion prediction synthesis unit, 207 ... Inverse quantization unit, 208 ... Inverse DCT conversion unit, 209 ... Decoded image memory, 210 ... Image output unit

Claims (1)

An image decoding method for decoding an encoded stream obtained by encoding an image,
In the decoding target block constituting the image,
According to the encoding mode information obtained from the encoded stream, a motion vector is selected from the motion vectors of a plurality of candidate blocks including blocks around the decoding target block, and the motion vector is used to select the decoding target block. A first step of restoring a motion vector;
A second step of generating a predicted image using the motion vector of the decoding target block and a reference image,
The motion vector selection process in the first step is:
Candidate blocks from a first block that is in the same frame as the decoding target block and is adjacent to the decoding target block, and a second block that is in the same position as the decoding target block in a different frame from the decoding target block Is a process for selecting
In the motion vector selection process,
In the first block, a comparison process for comparing motion vectors of a block located on the left side of the target block and a block located on the upper side of the target block is performed. Exclude the block above the block from the candidate block,
If the first block includes a non-referenceable block, the non-referenceable block is excluded from the motion vector candidate blocks, and another block is added to the candidate block.
The number of blocks to be added to the candidate block is changed according to the number of blocks already selected as candidate blocks,
Before the comparison process, if there are a plurality of blocks having available motion vectors located on the left side of the target block in the first block, the available motion vectors located on the left side of the target block An image decoding method comprising: performing a process of selecting a motion vector included in one of a plurality of blocks having a motion vector as a target of the comparison process.

Images