JP2005252921A - Image encoding method and apparatus, recording medium, and image decoding apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoding apparatus, image encoding method, recording medium, and image decoding apparatus in which encoding efficiency is not reduced even in the case of a video image having a luminance change between pictures like flashing or fade in compression-coding of a moving image signal. <P>SOLUTION: An image encoding apparatus 200 comprises an input image memory 101, a subtraction section 102, a selector 103, an orthogonal transform/quantization section 104, a variable length encoding section 105, an inverse quantization/inverse orthogonal transform section 106, an addition section 107, a reference image memory 108, a motion detection section 109, a motion compensation section 110, a code amount estimation section (1) 211, a code amount estimation section (2) 212, a reliability calculation section 201, an encoding mode determination section 213. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image encoding method, an image encoding device, a recording medium, and an image decoding device, and more particularly to an image encoding device that compresses and encodes moving image data with high efficiency.

  In recent years, with the development of multimedia applications, it has become common to digitize and handle all media information such as images, sounds, and texts in a unified manner. However, since a digitized image has an enormous amount of data, an information compression technique that compresses image data with high efficiency is indispensable for storage and transmission. As standards for such information compression techniques, for example, there are MPEG-1, MPEG-2, and MPEG-4 standardized by ISO (International Organization for Standardization). In any of these compression methods, input image data is divided into a plurality of blocks, and for each block, intra-frame coding (coding mode without motion compensation) and inter-screen prediction coding (prediction by motion compensation) are performed. The main feature is that encoding is performed by selecting an encoding mode to be performed.

  FIG. 2 is a block diagram showing a configuration of a conventional image encoding device 100. The image coding apparatus 100 includes an input image memory 101, a subtraction unit 102, a selector 103, an orthogonal transform / quantization unit 104, a variable length coding unit 105, an inverse quantization / inverse orthogonal transform unit 106, an addition unit 107, a reference image. A memory 108, a motion detection unit 109, a motion compensation unit 110, a code amount estimation unit (1) 111, a code amount estimation unit (2) 112, and a coding mode determination unit 113 are provided.

  The input image memory 101 is a memory for storing input image data until the time of encoding. The memory needs to have a sufficient capacity for at least a delay caused by changing the encoding order. The input image data is divided into a plurality of blocks at the time of encoding, and encoded in units of blocks.

  The subtractor 102 inputs the block-unit input image data output from the input image memory 101 to the positive input terminal, and inputs the block-unit motion compensated image data output from the motion compensator 110 to the negative input terminal. Subtraction is performed, and prediction error image data in units of blocks is output as the subtraction result.

  The selector 103 has a function of switching the a terminal and the b terminal at the input terminal or the output terminal, and switches the terminal to be connected according to the encoding mode determined by the encoding mode determination unit 113. The a terminal is selected when the encoding mode received from the encoding mode determination unit 113 is intra-picture encoding, and the b terminal is selected when inter-picture predictive encoding by motion compensation.

  The orthogonal transform / quantization unit 104 performs block-unit input image data (intra-screen encoding) or prediction error image data (inter-screen prediction encoding) output from the selector 103 by orthogonal transform such as discrete cosine transform. The frequency component is converted, and the frequency component is further quantized to generate a quantized frequency component.

  The variable length coding unit 105 uses the quantized frequency component output from the orthogonal transform / quantization unit 104 as the coding mode determined by the coding mode determination unit 113 or the motion detected by the motion detection unit 109. Variable length encoding is performed together with information such as vectors, and encoded image data is output.

The inverse quantization / inverse orthogonal transform unit 106 performs inverse quantization on the quantized frequency component output from the orthogonal transform / quantization unit 104, and further performs inverse orthogonal transform to perform decoding. At this time, when the encoding mode determined by the encoding mode determination unit 113 is intra-screen encoding, a decoded image for input image data in units of blocks is generated.
On the other hand, when the encoding mode determined by the encoding mode determination unit 113 is inter-frame predictive encoding based on motion compensation, decoding is performed on prediction error image data in units of blocks, and then the adder 107 The decoded image data is generated by adding the block-unit motion compensated image data output from the compensation unit 110.

  The reference image memory 108 stores decoded image data in units of blocks generated by the inverse quantization / inverse orthogonal transform unit 106 at the time of intra-screen encoding, and block units generated by the addition unit 107 at the time of inter-screen prediction encoding. This memory is required to store at least a reference image used at the time of encoding. The decoded image data stored in the reference image memory 108 is used as a reference image when screen predictive coding by motion compensation is performed on subsequent input images.

  The motion detection unit 109 detects a similar block region from the specified range of the reference image data stored in the reference image memory 108 with respect to the input image data in units of blocks output from the input image memory 101, A motion vector corresponding to the amount of movement is output.

  The motion compensation unit 110 generates motion compensated image data in units of blocks from the reference image for inter-frame predictive coding stored in the reference image memory 108 using the motion vector detected by the motion detection unit 109. .

  The code amount estimation unit (1) 111 estimates the code amount of encoded image data generated when intra-screen encoding is performed on input image data in units of blocks output from the input image memory 101. .

  The code amount estimation unit (2) 112 encodes encoded image data generated when inter-frame prediction encoding by motion compensation is performed on the prediction error image data in block units output from the subtraction unit 102. Estimate the amount.

  The encoding mode determination unit 113 performs inter-screen prediction encoding based on the code amount at the time of intra-frame encoding estimated by the code amount estimation unit (1) 111 and the motion compensation estimated by the code amount estimation unit (2) 112. An encoding mode (intra-screen encoding or inter-screen predictive encoding) is determined based on the code amount at the time. The coding mode determined here is sent as a control signal of the selector 103 and simultaneously sent to the variable length coding unit 105 and is variable length coded together with information such as quantized frequency components and motion vectors.

  In the image encoding apparatus having the above-described configuration, the correlation of video with respect to the time direction is usually high, and it is more efficient to select inter-frame prediction encoding by motion compensation for most blocks. However, for images with scene changes or flashing between screens when predictive coding is performed, or for images where changes between screens are severe, the correlation between the screens is low, and the inter-screen predictive coding by motion compensation is lower It may be more efficient to select encoding. For this reason, it can be said that switching the coding mode (intra-screen coding and inter-screen predictive coding) in units of blocks is very useful for improving the coding efficiency.

  As a typical coding mode switching method used conventionally, there is a method shown in FIG. 3 (see, for example, Non-Patent Document 1). The parameter VAROR in FIG. 3 is an evaluation value calculated by Equation 1 by the code amount estimation unit (1) 111 that is a component of the image encoding device 100 in FIG. This corresponds to the estimated code amount of the encoded image data generated when the image data is converted.

(Expression 1) VAROR = (1 / N) * Σ {Y1 (x, y) ^ 2} − {(1 / N) * ΣY1 (x, y)} ^ 2
Here, Y1 (x, y) is the value of luminance data at the pixel position (x, y) of the input image data in block units, and N is the number of luminance data included in the block.

  On the other hand, the parameter VAR in FIG. 3 is an evaluation value calculated by the code amount estimation unit (2) 112 according to Equation 2, and is generated when the prediction error image data in units of blocks is subjected to inter-frame prediction encoding. This corresponds to the estimated code amount of image data.

(Expression 2) VAR = (1 / N) * Σ {Y2 (x, y)} ^ 2
Here, Y2 (x, y) is the value of the luminance data at the pixel position (x, y) of the prediction error image data in block units, and N is the number of luminance data included in the block.

  As described above, the parameter VAROR calculated by the code amount estimation unit (1) 111 and the parameter VAR calculated by the code amount estimation unit (2) 112 are input to the encoding mode determination unit 113 and are shown in FIG. The encoding mode is determined based on the conditions shown. That is, in FIG. 3, when the plot position of the parameters (VAR, VAROR) is included in the region (a), intra-screen coding is selected, and when it is included in the region (b), inter-screen predictive coding is performed. select. Note that when the plot position is on the boundary, the inter-frame prediction encoding is selected.

Further, as another conventional switching method, there is a method shown in FIG. 4 (see, for example, Patent Document 1). In FIG. 4, the encoding mode is determined using parameters VAR and VAROR calculated by the same method as in the conventional example shown in FIG. The parameter TH in the figure is a threshold value that is determined based on the frequency distribution after encoding after obtaining the frequency distribution for the VAR of each block at the time of encoding, and this threshold value is the number of blocks for which intra-frame encoding is selected. It is adaptively changed so as to be constant. These processes are performed for the purpose of preventing image degradation for blocks for which predictive encoding has not been performed correctly.
Japanese Patent No. 3300199 ISO-IEC / JTC1 / SC29 / WG11 N0328 Test Model 3 (TM3)

  However, in inter-picture predictive coding using motion compensation, if there is a luminance change such as flushing or fading between the image data to be coded and the image data referenced during predictive coding, motion detection is performed. The reliability of the motion vector itself obtained by the above is low, and even if the estimated code amount at the time of inter-screen predictive coding is smaller than the estimated code amount at the time of intra-screen coding, visual coding distortion is conspicuous.

  In order to solve such a problem, an image encoding method according to the present invention divides input image data representing an input image into a plurality of blocks, and inter-screen encoding or intra-frame prediction code by motion compensation for each block. An image encoding method for generating encoded image data by selecting one of the encoding modes for encoding, and estimating a code amount of encoded image data when the block is encoded in a screen Step 1, a code amount estimation step 2 for estimating a code amount of encoded image data when the block is subjected to inter prediction encoding by motion compensation, and a reliability of inter prediction encoding by the motion compensation is calculated. Based on the reliability calculation step, the code amount estimated in the code amount estimation steps 1 and 2, and the reliability calculated in the reliability calculation step, the code of the block Characterized in that a coding mode determining step of determining the mode.

  The image encoding apparatus according to the present invention divides input image data representing an input image into a plurality of blocks, and encodes either one of intra-frame encoding or inter-frame predictive encoding by motion compensation for each block. Is an image encoding device that generates encoded image data by selecting a code amount estimation unit 1 that estimates the amount of encoded image data when the block is intra-coded, and moves the block Code amount estimation means 2 for estimating the code amount of encoded image data when inter-picture prediction encoding is performed by compensation, reliability calculation means for calculating the reliability of inter-picture prediction encoding by motion compensation, and the code Coding mode determining means for determining the coding mode of the block based on the code amount estimated by the quantity estimating means 1 and 2 and the reliability calculated by the reliability calculating means. It is characterized in.

  As described above, according to the present invention, it is possible to suppress deterioration in image quality due to erroneous detection of motion vectors in consideration of the reliability of motion detection that greatly affects the coding efficiency. .

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment)
FIG. 1 is a block diagram showing a functional configuration of an image encoding device 200 according to an embodiment of the present invention. In this figure, since the same components as those of the conventional image encoding device 100 shown in FIG. 2 have already been described, the same reference numerals are given and description thereof is omitted. Similarly in the subsequent drawings, the already described components are denoted by the same reference numerals, and the description thereof is omitted. The image coding apparatus 200 includes an input image memory 101, a subtraction unit 102, a selector 103, an orthogonal transform / quantization unit 104, a variable length coding unit 105, an inverse quantization / inverse orthogonal transform unit 106, an addition unit 107, and a reference image. A memory 108, a motion detection unit 109, a motion compensation unit 110, a code amount estimation unit (1) 211, a code amount estimation unit (2) 212, a reliability calculation unit 201, and an encoding mode determination unit 213 are provided.

  The code amount estimation unit (1) 211 estimates the code amount of encoded image data generated when intra-screen encoding is performed on input image data in units of blocks output from the input image memory 101. .

  The code amount estimation unit (2) 212 encodes encoded image data generated when inter-frame prediction encoding is performed on the prediction error image data in units of blocks output from the subtraction unit 102 by motion compensation. Estimate the amount.

  The reliability calculation unit 201 is based on the input image data stored in the input image memory 101 and the reference image data stored in the reference image memory 108, and the reliability of the motion detection process performed by the motion detection unit 109. Is calculated.

  The encoding mode determination unit 213 performs the intra-screen coding estimated by the code amount estimation unit (1) 211 and the inter-frame prediction encoding based on the motion compensation estimated by the code amount estimation unit (2) 212. The coding mode (intra-screen coding or inter-screen prediction coding) is determined on the basis of the amount of codes and the reliability of the motion detection process calculated by the reliability calculation unit 201.

  Hereinafter, each of the code amount estimation unit (1) 211, the code amount estimation unit (2) 212, the reliability calculation unit 201, and the encoding mode determination unit 213 will be described in detail.

  The code amount estimation unit (1) 211 calculates the statistic for the input image data in units of blocks output from the input image memory 101, and generates encoded image data when the data is encoded on the screen. Is estimated. For example, the code amount B1 can be calculated by Equations 3 to 5 using the luminance and color difference data constituting the input image data in block units.

(Formula 3) BY1 = Σ {Y1 (x, y) ^ 2} − (1 / NY) * {ΣY1 (x, y)} ^ 2
(Formula 4) BC1 = Σ {C1 (x, y) ^ 2} − (1 / NC) * {ΣC1 (x, y)} ^ 2
(Formula 5) B1 = KY * BY1 + KC * BC1
Here, BY1 and BC1 are estimated code amounts for luminance and chrominance data, respectively, and Y1 (x, y) and C1 (x, y) are luminance and chrominance data at the pixel position (x, y) of the input image data in block units. , NY and NC are the numbers of luminance and color difference data included in the block, and KY and KC are weighting factors for adjusting the importance of the luminance and color difference data. For example, when emphasizing the code amount of luminance data, KY> KC may be set. When estimating the code amount considering only luminance data, KC = 0 may be set.

  Further, since the estimated code amount calculated by the equation 5 at the time of the intra-picture encoding considers only the dispersion of the pixel data, that is, the AC component, the code amount corresponding to the DC component of the frequency component is added. Thus, it is possible to perform estimation with higher accuracy.

  Further, in Equations 3 and 4, variance is used as the statistic of the pixel data, but the amount of calculation can be reduced by using the absolute difference value as in Equations 6 and 7.

(Expression 6) BY1 = Σ | Y1 (x, y) − (1 / NY) * ΣY1 (x, y) |
(Expression 7) BC1 = Σ | C1 (x, y) − (1 / NC) * ΣC1 (x, y) |
Furthermore, the statistic used for code amount estimation is not limited to the above, and other statistic values may be used as long as the parameters correlate with the code amount.

  The code amount estimation unit (2) 212 calculates a statistic for the block-unit prediction error image data output from the subtraction unit 102, and generates the data when inter-frame prediction encoding is performed using motion compensation. The code amount B2 of the encoded image data to be performed is estimated. For example, the code amount B2 can be calculated by Equations 8 to 10 using the luminance and color difference data constituting the prediction error image data in block units.

(Formula 8) BY2 = Σ {Y2 (x, y) ^ 2} − (1 / NY) * {ΣY2 (x, y)} ^ 2
(Formula 9) BC2 = Σ {C2 (x, y) ^ 2} − (1 / NC) * {ΣC2 (x, y)} ^ 2
(Formula 10) B2 = KY * BY2 + KC * BC2
Here, BY2 and BC2 are the estimated code amounts for the luminance and chrominance data, respectively, Y2 (x, y) and C2 (x, y) are the luminance and chrominance at the pixel position (x, y) of the prediction error image data in block units. Data values, NY and NC are the numbers of luminance and color difference data included in the block, and KY and KC are weighting factors for adjusting the importance of the luminance and color difference data. In this case as well, if importance is attached to the code amount of the luminance data, KY> KC may be set. If the code amount considering only the luminance data is estimated, KC = 0 may be set.

  Further, in the inter-picture predictive encoding based on motion compensation, since a motion vector component is also encoded at the time of variable length encoding, the code amount corresponding to the motion vector component is added to the code amount B2 calculated by Expression 10. Thus, it is possible to perform estimation with higher accuracy.

  In Expressions 8 and 9, variance is used as the statistic of the pixel data. However, the calculation amount can be reduced by using the absolute difference as in Expressions 11 and 12.

(Expression 11) BY2 = Σ | Y2 (x, y) − (1 / NY) * ΣY2 (x, y) |
(Expression 12) BC2 = Σ | C2 (x, y) − (1 / NC) * ΣC2 (x, y) |
Furthermore, the statistic used for code amount estimation is not limited to the above, and other statistic values may be used as long as the parameters correlate with the code amount.

  The reliability calculation unit 201 calculates the reliability as an index indicating the probability of the motion detection process executed by the motion detection unit 109. Usually, in the motion detection process, as shown in FIG. 5, block matching using luminance data is performed between input image data in block units to be encoded and reference image data used for prediction. However, in the block matching as described above, for example, when there is a luminance change between screens, the motion detection fails and the possibility of selecting an incorrect motion vector increases. In addition, in a video with such a luminance change, color difference data often changes, and it is difficult to detect an accurate motion vector. The above-described erroneous detection of motion vectors becomes more prominent as the dynamic range (blurring width) of the luminance data is smaller, particularly in the input image data in block units to be encoded. Therefore, when motion detection is performed by block matching using an evaluation value as shown in FIG. 5, an index (confidence of motion detection) Conf indicating whether or not the motion vector is correct is expressed as shown in Equation 13, for example. Can do.

(Formula 13) if (DY <RY) Conf = 0
else Conf = K * (DY-RY)
Here, DY represents the luminance change amount between the screens of the input image data and the reference image data, RY represents the dynamic range of the luminance value in the input image data in units of blocks, and K is a coefficient for adjusting the importance of Conf. In Expression 13, the reliability Conf of motion detection indicates that the closer to 0, the higher the reliability.

  Note that the calculation formula for the reliability is not limited to the formula 13, and any other calculation formula may be used as long as it is an index indicating the reliability of motion detection.

  In the coding mode determination unit 213, the code amount B1 at the time of the intra-frame coding estimated by the code amount estimation unit (1) 211, and the inter-frame prediction coding by the motion compensation estimated by the code amount estimation unit (2) 212. The encoding mode is determined according to Equation 14 using the time code amount B2 and the motion detection reliability Conf calculated by the reliability calculation unit 201.

(Formula 14) if (B1 <B2 + Conf) Intra-screen coding
Else Motion Compensation Interframe Predictive Coding Note that the coding mode is not limited to the condition of Expression 14, but can be determined by other comparison operations using B1, B2, and Conf.

  The image encoding apparatus according to the present invention is useful as an image encoding apparatus that compresses and encodes moving image data with high efficiency.

The block diagram which shows the structure of the image coding apparatus in embodiment of this invention. The block diagram which shows the structure of the conventional image coding apparatus. Explanatory drawing about the conventional encoding mode determination method Explanatory drawing about the conventional encoding mode determination method Explanatory drawing about block matching in motion vector detection

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image coding apparatus 101 Input image memory 102 Subtraction part 103 Selector 104 Orthogonal transformation / quantization part 105 Variable length coding part 106 Inverse quantization / inverse orthogonal transformation part 107 Addition part 108 Reference image memory 109 Motion detection part 110 Motion compensation Part 111 Code amount estimation part (1)
112 Code amount estimation unit (2)
113 Coding mode determination unit 200 Image coding device 201 Reliability calculation unit 211 Code amount estimation unit (1)
212 Code amount estimation unit (2)
213 Coding mode determination unit

Claims (9)

The input image data representing the input image is divided into a plurality of blocks, and encoded image data is generated by selecting one of the encoding modes of intra-frame encoding or inter-frame prediction encoding by motion compensation for each block. A code amount estimation step 1 for estimating a code amount of encoded image data when the block is subjected to intra-frame encoding, and a code when the block is subjected to inter-screen predictive encoding by motion compensation. Code amount estimation step 2 for estimating the code amount of the encoded image data, reliability calculation step for calculating the reliability of inter-frame prediction encoding by motion compensation, and the code estimated in the code amount estimation steps 1 and 2 An encoding mode determining step for determining an encoding mode of the block based on the quantity and the reliability calculated in the reliability calculating step. Picture coding method for. The input image data representing the input image is divided into a plurality of blocks, and encoded image data is generated by selecting one of the encoding modes of intra-frame encoding or inter-frame prediction encoding by motion compensation for each block. A code amount estimation unit 1 for estimating a code amount of encoded image data when the block is subjected to intra-screen encoding, and a code when the block is subjected to inter-screen predictive encoding by motion compensation Code amount estimation means 2 for estimating the code amount of the encoded image data, reliability calculation means for calculating the reliability of inter-frame prediction encoding by motion compensation, and code estimated by the code amount estimation means 1 and 2 An image encoding apparatus comprising: an encoding mode determining unit that determines an encoding mode of the block based on a quantity and the reliability calculated by the reliability calculating unit. 3. The image encoding apparatus according to claim 2, further comprising recording means for recording the generated encoded image data. The image encoding apparatus according to claim 2, further comprising a recording medium for recording the generated encoded image data. 5. The image encoding apparatus according to claim 2, further comprising a decoding unit that decodes the generated encoded image data. 6. The image encoding apparatus according to claim 5, further comprising a decoding unit that reads and decodes the encoded image data from a recording medium on which the generated encoded image data is recorded. A recording medium on which encoded image data encoded by the image encoding device according to claim 2 is recorded. A program for executing the image encoding method according to claim 1. A recording medium on which the program according to claim 8 is recorded.

Images