JP6253564B2 - Image encoding device, image decoding device, image encoding method, image decoding method, image encoding program, and image decoding program - Google Patents

Description

  The present invention relates to an image encoding / decoding technique, and in particular, an image encoding device, an image decoding device, an image encoding method, an image decoding method, an image decoding method, and the like that significantly reduce the amount of calculation compared to conventional intra prediction encoding / decoding. The present invention relates to an encoding program and an image decoding program.

  H. is the international standard for video coding. In H.264, intra prediction encoding is performed in order to improve the compression rate in encoding using pixel correlation between blocks (see Non-Patent Document 1). This intra-screen prediction is performed in units of blocks in which several pixels are grouped, and three types of block sizes of 4 × 4, 8 × 8, and 16 × 16 can be used for the luminance signal. In each block size, a plurality of prediction modes can be selected.

  This H. In H.264, a method based on extrapolation prediction is used for intra prediction, but there is a problem that prediction efficiency is poor. In order to solve this problem, block distortion is suppressed by a deblocking filter for the entire screen, which increases the amount of calculation.

  As a technique for improving the coding efficiency in intra prediction, a technique described in Non-Patent Document 2 is known. This technique is a method of searching for a block with a small error from a coded region and coding using a prediction error for the block to be coded in intra prediction.

  FIG. 22 is a flowchart showing an example of intra prediction encoding processing according to the prior art. In the intra prediction encoding of Non-Patent Document 2, first, an image to be encoded is divided into N blocks 1 to N having the same size (step S301). Next, intra prediction encoding is performed for the first block 1 (step S302). Subsequently, in coding after block 2, inter-frame prediction coding is performed using a block with a small prediction error from the coded region as a reference image, and motion vector information and a prediction error are coded on the reference image. (Step S303). The processing in step S303 is repeated until the last block N.

ITU-T Rec. H.264, “Advanced video coding for generic audiovisual services”, March 2005. J. Yang, B. Yin, and N. Zhang, “A block-matching based intra frame prediction for H.264 / AVC”, in Proceedings of IEEE International Conference on Multimedia and Expo (ICME '06), pp. 705- 708, Toronto, Canada, July 2006.

  Although the technique of Non-Patent Document 2 is a technique for improving the coding efficiency, since a prediction error can be suppressed in a region where the same pattern is repeated, the quantization error tends to be small. Therefore, it is considered that the processing amount of the deblocking filter can be reduced.

  However, in the above method, since it is necessary to send the offset vector information indicating the relative position of the reference block to each block to the decoding side, an operation for decoding the reference block information occurs also on the decoding side. As a result, there is a problem that the amount of computation remains large.

  An object of the present invention is to solve the above-described problems and reduce the amount of encoding operation and the amount of decoding operation while suppressing a decrease in encoding efficiency.

In order to solve the above problems, the present invention performs the following processing when compressing and encoding an input image.
(1) The input image is divided into blocks of n × m pixels, and each divided block is divided into sub-blocks of n ₁ × m ₁ pixels (where 1 ≦ n ₁ <n, 1 ≦ m ₁ <m). , Sub-blocks having the same relative position in the block are collected, and divided images of the same size are generated.
(2) At least one of the divided images is subjected to intra prediction encoding. Here, the intra prediction encoding is encoding by intra prediction performed in units of divided images.
(3) For a divided image other than the divided image that has been subjected to intra prediction encoding, a predetermined filter determined by the relative position of the divided image and the encoded divided image on the original image is applied to the encoded divided image. Inter-screen predictive coding is performed using the processed image as a reference image. The inter-screen prediction encoding here is encoding by inter-screen prediction performed with each divided image as a screen unit. In this inter-screen predictive coding, when there is a divided image that has been subjected to inter-screen predictive coding, an image obtained by applying a predetermined filter to the divided image that has undergone inter-screen predictive coding is used for inter-screen predictive coding. May be an image.

  Furthermore, as one aspect of the above-described invention, in inter-screen predictive coding, at least one or more of the divided images other than the divided image that has been subjected to intra-screen predictive coding is not coded. It is also possible to perform only conversion. As another aspect of the invention, when generating a divided image, at least one or more divided images are not generated, and only the generated divided image is encoded.

Further, the present invention performs the following processing in decoding the encoded data of the compression-encoded image.
(1) The image encoding device divides the input image into blocks of n × m pixels, and each divided block is n ₁ × m ₁ pixels (where 1 ≦ n ₁ <n, 1 ≦ m ₁ <m) The sub-block is divided into sub-blocks, sub-blocks having the same relative position in the block are collected, and divided data having the same size is generated and encoded, and encoded data is input.
(2) Intra-screen predictive decoding is performed on at least one of the divided images from the input encoded data.
(3) An image obtained by subjecting a decoded divided image to a predetermined filter determined by a relative position on the original image between the divided image and the decoded divided image, for a divided image other than the divided image that has been subjected to intra-screen prediction decoding. Inter-screen predictive decoding is performed as a reference image.
(4) A decoded image is configured from the divided images decoded by intra prediction decoding and inter prediction decoding.
In the inter-screen predictive decoding, when there is a divided image that has undergone inter-screen predictive decoding, an image obtained by performing a predetermined filter on the divided image that has undergone inter-screen predictive decoding is used as a reference image for inter-screen predictive decoding. There is.

  Furthermore, as one aspect of the present invention, at least one or more of the divided images other than the divided image subjected to intra-screen prediction decoding can be generated by interpolation using the pixel information of the decoded divided image.

  The operation of the present invention will be described in comparison with the prior art.

  In the technique of Non-Patent Document 2, the encoding target image uses an encoded image as a reference image, and an offset vector representing the relative position is encoded. Therefore, a calculation amount for decoding it is generated.

  On the other hand, in the present invention, a reference image is generated by sending a filter based on the relative position of a pixel or a pixel group to the encoded divided image instead of sending an offset vector, in contrast to the technique of Non-Patent Document 2. Since a reference image with high prediction efficiency is created based on the relative position information, the decoding side does not need to decode vector information, and the amount of decoding calculation can be reduced.

  That is, in the present invention, since the encoding target image does not require an offset vector between the encoded image and the reference image, the amount of calculation for decoding does not occur. Depending on the generation method of the divided image, the pixel (or pixel group, the same shall apply hereinafter) of the encoding target image becomes an adjacent pixel that is deviated from the pixel of the encoded image by a certain distance in the spatial direction and in a certain direction. If the digitized image is uniquely determined, there is no need to send offset vector information representing the relative position. Thus, a reference pixel with high prediction efficiency can be generated by applying a filter determined by the relative position to the encoded image. Therefore, the present invention can reduce the calculation amount without reducing the encoding efficiency.

  In addition, the present invention using interpolation prediction can perform prediction more appropriately than the intra-frame coding of Non-Patent Document 1 using extrapolation prediction. More specifically, in the intra-picture encoding of Non-Patent Document 1, only the leftward or upward pixel that has been encoded can be referred to for the prediction of the pixel value, whereas the present invention provides an encoded divided image. When the prediction error encoding is performed using the reference image as the reference image, it is possible to refer to the upper, lower, left, and right pixels in the pixel value prediction, so that an appropriate pixel can be predicted. Since the filter for obtaining the predicted pixel is determined by the block division method, it is not necessary to send the filter information for each block. Thereby, pixel prediction is appropriately performed with respect to the intra-screen coding of Non-Patent Document 1, and the prediction efficiency is increased. As described above, since the prediction efficiency is high, a deblocking filter is considered unnecessary, so that the amount of calculation required for the processing can be reduced.

  Further, in one aspect of the present invention, some of the divided images are decoded by using an interpolation operation with a calculation amount of half or less instead of information source decoding, inverse DCT, deblocking filter, and the like that occupy most of the decoding processing. As a result, the amount of decoding computation can be reduced.

  That is, in one aspect of the present invention, since some of the divided images are not encoded, the decoding calculation amount for the divided images is reduced at the time of decoding. In addition, although it is necessary to interpolate the pixels of the non-encoded divided image, for example, by performing interpolation such as linear interpolation, all the divided images are added to an image having a strong pixel correlation. The amount of calculation can be suppressed as compared with the case of encoding / decoding.

  In another aspect of the present invention, instead of generating all the divided images and encoding all the divided images, the generated divided images are generated without generating at least one or more divided images. Since encoding is performed only for the, it is possible to reduce the amount of computation and the required memory when generating and encoding divided images. In addition, although it is necessary to perform interpolation on the pixels of the divided image that is not generated and encoded, the linear interpolation described above can be used for this.

  According to the present invention, it is possible to reduce the amount of encoding calculation and the amount of decoding calculation while suppressing a decrease in encoding efficiency as compared with the conventional intra prediction encoding.

It is a figure which shows the structural example of an image coding apparatus. It is a figure which shows the example of a production | generation of the division image by a division image generation part. It is a figure which shows the example of a production | generation of the reference image by a reference image generation part. It is a figure which shows the example of a division | segmentation of an encoding target image. It is a flowchart of Example 1 of an image encoding process. It is a flowchart of a divided image generation process. It is a flowchart of a prediction encoding process between screens. It is a figure which shows the example of application of an interpolation filter. It is a figure which shows the example of a division | segmentation example of an encoding target image, and the example of the division | segmentation image of encoding target. It is a flowchart of Example 3 of an image encoding process. It is a flowchart of a prediction encoding process between screens. It is a flowchart of Example 4 of an image encoding process. It is a flowchart of a divided image generation process. It is a flowchart of a prediction encoding process between screens. It is a figure which shows the structural example of an image decoding apparatus. It is a flowchart of Example 1 of an image decoding process. It is a flowchart of Example 2 of an image decoding process. It is a figure which shows the example of the interpolation of the divided image by the inter-screen interpolation part. It is a figure which shows the example of the interpolation in a screen. It is a figure which shows the hardware structural example in the case of implement | achieving an image coding apparatus using a software program. FIG. 25 is a diagram illustrating a hardware configuration example when an image decoding device is realized using a software program. It is a flowchart which shows the example of the prediction encoding process in a screen by a prior art.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Image coding device]
FIG. 1 is a diagram illustrating a configuration example of an image encoding device. The image encoding device 10 includes a divided image generation unit 11, an intra-screen prediction encoding unit 12, an inter-screen prediction encoding unit 13, and an information source encoding unit 14. The inter-screen prediction encoding unit 13 includes a reference image generation unit 131, a prediction error calculation unit 132, a prediction error encoding unit 133, and an image decoding unit 134. In addition, when the reference image generation unit 131 generates the reference image using only the divided image encoded by the intra-frame prediction encoding unit 12, the image decoding unit 134 may not be provided.

The divided image generation unit 11 divides the input image into blocks of n × m pixels, and each of the divided blocks is a sub-unit of n ₁ × m ₁ pixels (where 1 ≦ n ₁ <n, 1 ≦ m ₁ <m). Dividing into blocks, sub-blocks having the same relative position in the block are collected, and divided images of the same size are generated.

FIG. 2 is a diagram illustrating an example of generation of divided images by the divided image generation unit 11. The divided image generation unit 11 uses, for example, the original image shown in FIG. 2A as an input image, and the original image is converted into a block Mj (j = 0) of n × m pixels as shown in FIG. 2B. , 1,..., J). Next, each block Mj, as shown in FIG. _{2 (C), n 1 ×} m 1 pixels _{(where, 1 ≦ n 1 <n,} 1 ≦ m 1 <m) sub-block Bjk (k = 0, and 1, ..., K).

  Next, as shown in FIG. 2D, sub-blocks Bjk having the same relative position in the block are collected from each block Mj, and divided images Pk (k = 0, 1,..., K) having the same size. Is generated. The divided image P0 is a collection of sub-blocks B00, B10,..., BJ0, the divided image P1 is a collection of sub-blocks B01, B11,. , ..., a collection of BJK.

  The intra-screen prediction encoding unit 12 performs intra-screen prediction encoding on the first divided image generated by the divided image generation unit 11. The intra-screen predictive encoding here is an encoding method that uses only pixel information of the divided image that is the current encoding target, and does not refer to other divided images. Any encoding method may be used as long as it exists. For example, H.C. A method such as intra prediction encoding in the H.264 encoding scheme can be used.

  The inter-screen prediction encoding unit 13 performs inter-screen prediction encoding on the second and subsequent divided images generated by the divided image generation unit 11. The inter-screen predictive coding here is a coding method that performs predictive coding using a coded divided image other than the divided image that is the current coding target as a reference image.

  The reference image generation unit 131 in the inter-screen prediction encoding unit 13 encodes a predetermined filter determined by the relative position on the original image of the divided image that is the current encoding target and the encoded divided image. A reference image is generated by applying to the already divided image.

  FIG. 3 is a diagram illustrating an example of reference image generation by the reference image generation unit 131. Hereinafter, an example of generating a reference image when the divided image Pi has been encoded and then the divided image Pk is subjected to inter-screen predictive encoding will be described. A sub-block belonging to the divided image Pi is represented as Bi, and a sub-block belonging to the divided image Pk is represented as Bk.

  Assuming that the positional relationship in the original image of the sub-block Bi of the divided image Pi and the sub-block Bk of the divided image Pk is as shown in FIG. 3A, as shown in FIG. The sub-block Bi located at is extracted. In this example, four sub-blocks Bi are extracted for one sub-block Bk, but the number is not limited to four. Next, as shown in FIG. 3C, the pixel value of the sub-block Bk ′ is calculated by applying an interpolation filter to the pixel values of the four extracted sub-blocks Bi. As the filter coefficient of the interpolation filter to be used, a filter coefficient determined in advance by the relative position between the sub-block Bi and the sub-block Bk on the original image is used. Various interpolation methods using an interpolation filter are conventionally known, and the reference image may be generated using any interpolation method.

  A collection of sub-blocks Bk ′ generated by interpolation in this way is used as a reference image used for inter-frame prediction encoding of the divided image Pk.

  The prediction error calculation unit 132 subtracts each pixel value of the reference image generated by the reference image generation unit 131 from each pixel value of the divided image that is the current encoding target, and calculates a prediction error. The prediction error encoding unit 133 performs orthogonal transformation or quantization processing on the calculated prediction error, and encodes the prediction error.

  The information source encoding unit 14 entropy-encodes the encoded information of the divided images encoded by the intra prediction encoding unit 12 and the inter prediction encoding unit 13, and outputs encoded data.

  The image decoding unit 134 is used to predict the encoded divided image when the divided image encoded by the inter-screen predictive encoding unit 13 is used for generating a reference image in encoding of another divided image. The divided image is decoded by adding the prediction error of the encoding result to the reference image and sent to the reference image generation unit 131. When the prediction error is orthogonally transformed and quantized by the prediction error encoding unit 133, the image decoding unit 134 performs inverse quantization and inverse orthogonal transformation on the output of the prediction error encoding unit 133. Later, the pixel values of the reference image are added to decode the divided image.

[Example 1 of image encoding process]
Next, the flow of image encoding processing will be described according to a specific example.

  FIG. 4 shows an example of division of the encoding target image. In the example described below, the divided image generation unit 11 divides one frame of an input image to be encoded into 2 × 2 pixel blocks M0, M1,..., MJ as shown in FIG. And Furthermore, each block M0, M1,..., MJ is divided into sub-blocks B0, B1, B2, and B3 pixel by pixel. From the M0, M1,..., MJ thus divided, a collection of pixels in the upper left sub-block B0 is a divided image P0, and a collection of pixels in the upper right sub-block B1 is a divided image P1. A collection of pixels in the lower left sub-block B2 is a divided image P2, and a collection of pixels in the lower right sub-block B3 is a divided image P3.

  Here, an example in which a 2 × 2 pixel block is divided into 1 × 1 pixel sub-blocks will be described, but the block and sub-block sizes are not limited to this example, and the block size or sub-block size is not limited to this example. Even when is larger, the present invention can be applied in the same manner.

  FIG. 5 is a flowchart of Example 1 of the image encoding process. First, the divided image generating unit 11 divides the input image into blocks M0 to MJ having the same size as shown in FIG. 4 and collects pixels (sub-blocks) having the same relative position in each block to obtain divided images. P0 to PK are generated (step S10). In the example of FIG. 4, K = 3. Details will be described later with reference to FIG.

  Next, the intra prediction encoding unit 12 performs intra prediction encoding for the first divided image P0 using a conventional intra prediction encoding method or the like (step S11).

  For the divided images after the divided image P1, the inter-screen predictive encoding unit 13 uses the image obtained by applying a predetermined filter determined by the relative position on the original image to the encoded divided image as a reference image. Inter-frame prediction encoding is performed in units of and only the prediction error is encoded (step S12). Details will be described later with reference to FIG. Note that the divided images after the divided image P1 may also be encoded by intra prediction encoding.

  FIG. 6 is a flowchart of the divided image generation process (step S10 in FIG. 5). The divided image generation unit 11 divides the input image into blocks Mj (j = 0, 1,..., J) having the same size (step S101). Next, each block Mj is divided into sub-blocks Bk (k = 0, 1,..., K) having the same relative position within the block (step S102). Subsequently, for each k = 0, 1,..., K, only the sub-blocks Bk in the blocks M0 to MJ are extracted and arranged in the order of the blocks M0 to MJ to generate a divided image Pk (step S103). ). The above process is repeated until the last divided image PK is generated (step S104).

  FIG. 7 is a flowchart of the inter-screen predictive encoding process (step S12 in FIG. 5). Assume that k divided images P0 to P (k-1) have been encoded. The inter-screen prediction encoding unit 13 interpolates a pixel position of the divided image Pk on the original image into Pi (0 ≦ i ≦ k−1) among the encoded divided images P0 to P (k−1). Is applied to generate an image Pk ′ (step S121).

  Using the generated image Pk ′ as a reference image, prediction error encoding of the divided image Pk is performed (step S122). The above process is repeated until the inter-screen predictive coding of the last divided image PK is completed (step S123).

  In the encoding of the input image shown in FIG. 4, first, the divided image P0 is subjected to intra-screen prediction encoding in step S11 of FIG. 5, and then the divided image that has been encoded by the reference image generation unit 131 in step S121 of FIG. An image P1 ′ is generated by applying an interpolation filter to P0. As this interpolation filter, for example, H.D. In the H.264 motion compensation prediction, a 6-tap FIR filter used for generating a prediction signal with 1/2 pixel accuracy can be used.

  FIG. 8 is a diagram illustrating an application example of the interpolation filter. In FIG. 8, pixels indicated by ◯ are integer position pixels of the divided image P0. Pixels indicated by □, Δ, and ◇ are pixels located at 1/2 positions of P0 of the divided image. For the reference image P1 ′ of the divided image P1, a horizontal half-pixel filter as shown in the following equation is applied to the pixels A, B, C, D, E, and F of the encoded image of P0. The pixel is calculated as a half-position pixel of P0 shown in FIG.

a ₁ = 1/32 {A-5B + 20C + 20D-5E + F}
In step S122 of FIG. 7, the prediction error calculation unit 132 calculates a prediction error that is a difference between the pixel value of the divided image P1 and the pixel value of the reference image P1 ′, and the prediction error encoding unit 133 calculates the prediction error. Is encoded.

  In the next encoding of the divided image P2, the reference image generation unit 131 generates a reference image P2 ′ of the divided image P2 in step S122 of FIG. For the reference image P2 ′, a vertical half-pixel filter such as the following equation is applied to the pixels A, G, H, I, J, and K of the encoded image of P0, and P0 indicated by Δ in FIG. Is calculated as a half-position pixel.

b ₁ = 1/32 {A-5G + 20H + 20I-5J + K}
In step S122 of FIG. 7, similarly to the encoding of the divided image P1, the prediction error calculation unit 132 and the prediction error encoding unit 133 perform the inter-frame prediction of the motion vector 0 using P2 ′ as a reference image, Perform prediction error encoding.

  Further, a reference pixel P3 ′ of the divided image P3 is applied with a half-pixel filter as shown in the following equation in step S122 of FIG. 7 with respect to the result of the half-position pixel P0 calculated by P1 ′ and P2 ′. Then, as shown by ◇ shown in FIG. 8, a pixel shifted by half a pixel in the lower right direction is calculated.

c ₃ = 1/64 {a ₁ -5a ₂ + 20a ₃ + 20a ₄ -5a ₅ + a ₆ }
+1/64 {b ₁ -5b ₂ + 20b ₃ + 20b ₄ -5b ₅ + b ₆ }
In step S122 of FIG. 7, similarly to the encoding of the divided images P1 and P2, the prediction error calculation unit 132 and the prediction error encoding unit 133 perform the divided image by inter-frame prediction of the motion vector 0 using P3 ′ as a reference image. P3 prediction error encoding is performed.

[Example 2 of image encoding process]
Example 2 of image coding processing is an example in which not only an image that has been subjected to intra-frame predictive coding but also an image that has been subjected to inter-frame predictive coding is used to generate a reference image. In this case, the image decoding unit 134 decodes the divided image encoded by the prediction error encoding unit 133 and stores the decoded image in a memory included in the reference image generation unit 131.

In the first example of the image encoding process described above, the reference image P3 ′ is generated from the encoded divided image P0 in the inter-frame prediction encoding of the divided image P3 in the input image shown in FIG. Using the P2 encoded image obtained as a result of performing the inter-picture predictive encoding on the divided image P2 by the above-described method, a filter such as the following equation (refer to FIG. 8 for the positions of c ₃ and b _{1 to} b ₆ ): Is applied to generate an image P3 ′ shifted by half a pixel in the right direction.

c ₃ = 1/32 {b ₁ -5b ₂ + 20b ₃ + 20b ₄ -5b ₅ + b ₆ }
[Example 3 of image encoding process]
In the example 3 of the image encoding process, not all of the divided images P0 to PK are encoded, but one of the divided images P1 to PK to be subjected to the inter-picture predictive encoding encoded in the examples 1 and 2 described above. The partial divided image is not encoded. On the decoding side, the unencoded divided image is generated from the decoded divided image by interpolation.

  FIG. 9 shows an example of division of an encoding target image and an example of an encoding target divided image. Similarly to the example of FIG. 4, the divided image generation unit 11 divides one frame of the input image to be encoded into 2 × 2 pixel blocks M0, M1,..., MJ, and each block M0, M1,..., MJ are divided into sub-blocks B0, B1, B2, and B3 pixel by pixel.

  In this way, from each of M0, M1,..., MJ, a collection of pixels in the upper left sub-block B0 is set as a divided image P0, and a set of pixels in the upper right sub-block B1 is set as a divided image P1. A collection of pixels of the sub-block B2 is a divided image P2, and a collection of pixels of the lower right sub-block B3 is a divided image P3.

  In the example of FIG. 4, all of the divided images P0, P1, P2, and P3 are encoded. In this example, the divided image P0 is subjected to intra prediction encoding, and an interpolation filter is added to the encoded image of the divided image P0. Is applied to generate a reference image to be used for prediction of the divided image P3, and the divided image P3 is inter-screen predictively encoded. The divided images P1 and P2 are not encoded.

  FIG. 10 is a flowchart of Example 3 of the image encoding process. The divided image generation unit 11 divides the input image into blocks M0 to MJ having the same size, collects pixels (subblocks) having the same relative position in each block, and generates divided images P0 to PK (step S10). ′). Note that generation of a divided image that is not encoded here can be omitted.

  Next, the intra prediction encoding unit 12 performs intra prediction encoding for the first divided image P0 using a conventional intra prediction encoding method or the like (step S11 ′).

  Next, the inter-screen prediction encoding unit 13 does not encode one or more of the divided images P1 to PK, and encodes the other divided images (step S12 ′). In the case of the division example of FIG. 9, the inter-screen prediction encoding unit 13 does not encode the divided images P1 and P2, but only encodes the divided image P3. Details of the process of step S12 ′ will be described below.

  FIG. 11 is a flowchart of the inter-screen predictive encoding process (step S12 ′ in FIG. 10). U divided images to be encoded are selected from the divided images P1 to PK, and placed in order P1 to PU (step S121 ′). However, U is a predetermined value within the range of 1 ≦ U ≦ K−1. In the division example of FIG. 9, one divided image P3 is selected from the divided images P1 to P3, and the number is represented as P1.

  Next, the inter-screen prediction encoding unit 13 sets the pixel position of the divided image Pk on the original image to Pi (0 ≦ i ≦ k−1) among the encoded divided images P0 to P (k−1). Is applied to generate an image Pk ′ (step S122 ′). In the division example of FIG. 9, an interpolation filter of the pixel position of a new divided image P1 (position P3 in FIG. 9) on the original image is applied from the encoded divided image P0 to generate an image P1 ′. Become. This interpolation filter uses a filter for obtaining the pixel value of ◇ described in FIG. 8, that is, an expression for calculating the pixel value of the 1/2 position pixel with respect to P0.

  Using the generated image Pk ′ as a reference image, prediction error encoding of the divided image Pk is performed (step S123 ′). The above processing is repeated until the inter-frame predictive coding of the last divided image PU is completed (step S124 ′).

[Example 4 of image encoding process]
In Example 4 of the image encoding process, not all of the divided images P0 to PK are generated, but the divided images P0 to PL are generated and encoded when 0 <L <K. Among the divided images P1 to PK to be subjected to the inter-picture prediction encoding encoded in 2, some of the divided images are not encoded. On the decoding side, the unencoded divided image is generated from the decoded divided image by interpolation.

  FIG. 9 shows an example of division of an encoding target image and an example of an encoding target divided image. Similarly to the example of FIG. 4, the divided image generation unit 11 divides one frame of the input image to be encoded into 2 × 2 pixel blocks M0, M1,..., MJ, and each block M0, M1,..., MJ are divided into sub-blocks B0, B1, B2, and B3 pixel by pixel.

  In this way, from M0, M1,..., MJ, a collection of pixels in the upper left sub-block B0 is referred to as a divided image P0, and a collection of pixels in the lower right sub-block B3 is referred to as a divided image P3.

  In Example 3, it is possible to generate all of the divided images P0, P1, P2, and P3. In Example 4, however, the divided image P0 is subjected to intra-screen predictive coding, and is converted into an encoded image of the divided image P0. A reference image used for prediction of the divided image P3 is generated by applying an interpolation filter, and the divided image P3 is subjected to inter-screen prediction encoding. The divided images P1 and P2 are neither generated nor encoded.

  FIG. 12 is a flowchart of Example 4 of the image encoding process. The divided image generation unit 11 divides the input image into blocks M0 to MJ having the same size, collects pixels (sub-blocks) having the same relative position in each block, and generates divided images P0 to PL (step S10). ″). In the division example of FIG. 9, one divided image P3 among the divided images P1 to P3 is generated, and this is represented as P1.

  Next, the intra prediction encoding unit 12 performs intra prediction encoding on the first divided image P0 using a conventional intra prediction encoding method or the like (step S11 ″).

  Next, the inter-screen prediction encoding unit 13 performs encoding on the divided images P1 to PL (step S12 ″). In the case of the division example in FIG. 9, the inter-screen prediction encoding unit 13 performs the divided image P1 (FIG. In FIG. 9, only the position P3) is encoded.

  13 is a flowchart of the divided image generation process (step S10 ″ in FIG. 12). The divided image generation unit 11 divides the input image into blocks Mj (j = 0, 1,..., J) having the same size. (Step S101 ″). Next, each block Mj is divided into sub-blocks Bk (k = 0, 1,..., K) having the same relative position in the block (step S102 ″). Subsequently, each k = 0, 1 ,..., K, only up to L sub-blocks Bl in the blocks M0 to MJ are extracted, arranged in the order of the blocks M0 to MJ, and a divided image Pl is generated (step S103 ″). The above processing is repeated until the generation of the last divided image PL is completed (step S104 ″).

  Next, details of the process of step S12 ″ in FIG. 12 will be described. FIG. 14 is a flowchart of the inter-screen predictive encoding process (step S12 ″ of FIG. 12).

  The inter-screen prediction encoding unit 13 interpolates a pixel position of the divided image Pk on the original image into Pi (0 ≦ i ≦ k−1) among the encoded divided images P0 to P (k−1). Is applied to generate an image Pk ′ (step S121 ″). In the division example of FIG. 9, pixels of a new divided image P1 (position P3 in FIG. 9) on the original image from the encoded divided image P0. The position interpolation filter is applied to generate the image P1 ', which calculates the pixel value of the half position pixel with respect to P0, that is, the filter for obtaining the pixel value of ◇ described in FIG. Use the formula.

  Using the generated image Pk ′ as a reference image, prediction error encoding of the divided image Pk is performed (step S122 ″). The above processing is repeated until the inter-screen prediction encoding of the last divided image PL is completed (step S123 ″). .

[Image decoding device]
FIG. 15 is a diagram illustrating a configuration example of an image decoding device. The image decoding apparatus 20 includes an encoded data input unit 21, an information source decoding unit 22, an intra-screen prediction decoding unit 23, an inter-screen prediction decoding unit 24, and a decoded image configuration unit 25. The inter-screen prediction decoding unit 24 includes a prediction error decoding unit 241, a reference image generation unit 242, and a decoded image calculation unit 243. In addition, the decoded image configuration unit 25 may include an intra-screen interpolation unit 251 that generates the divided image from the decoded image by interpolation when a part of the divided image is not encoded on the encoding side.

  The image decoding device 20 receives the encoded data of the image compressed and encoded by the image encoding device 10 shown in FIG. 1 from the encoded data input unit 21 and decodes the image. The information source decoding unit 22 performs entropy decoding on the input encoded data.

  The intra-screen prediction decoding unit 23 decodes the encoded data encoded by the intra-screen prediction of at least one divided image by the intra-screen prediction. The inter-screen predictive decoding unit 24 performs inter-screen predictive decoding on the divided images other than the divided images that have been subjected to intra-screen predictive decoding. Therefore, the prediction error decoding unit 241 of the inter-screen prediction decoding unit 24 performs inverse quantization, inverse orthogonal transformation, and the like as necessary to decode the prediction error. The reference image generation unit 242 generates a reference image by applying a predetermined filter determined by the relative position of the divided image to be decoded and the decoded divided image on the original image to the decoded divided image. The decoded image calculation unit 243 calculates the pixel value of the decoded image by adding the pixel value of the reference image generated by the reference image generation unit 242 to the output of the prediction error decoding unit 241.

  The decoded image construction unit 25 generates a decoded image by arranging each sub-block of the divided image decoded by the intra prediction decoding unit 23 and the inter prediction decoding unit 24 at the original position on the original image. .

  When there is a divided image that has not been encoded on the encoding side, the intra-screen interpolation unit 251 generates the divided image from the decoded image by interpolation.

[Example 1 of image decoding process]
FIG. 16 is a flowchart of Example 1 of the image decoding process. Here, an example will be described in which the encoding side decodes the data encoded by dividing the input image as shown in FIG. On the encoding side, as described above, the input size is divided into blocks M0 to MJ having the same size, and pixels (sub-blocks) having the same relative position in each block are collected to generate divided images P0 to PK. The divided image P0 is encoded by intra-screen predictive encoding, and the divided images P1 to PK are encoded by inter-screen predictive encoding in units of divided images.

  In the image decoding device 20, when encoded data is input by the encoded data input unit 21, after the encoded data is entropy-decoded by the information source decoding unit 22, first, the intra-frame prediction decoding unit 23 encodes the divided image P 0. Data is decoded by intra prediction using a conventional intra prediction decoding method (step S20).

  Next, for the encoded data of the divided images after the divided image P1, the inter-screen predictive decoding unit 24 applies an image obtained by applying a predetermined filter determined by the relative position on the original image to the decoded divided image as a reference image. Then, by adding the decoded prediction error to this, inter-screen predictive decoding is performed in units of divided images (step S21). The reference image generation method in the reference image generation unit 242 is exactly the same as the reference image generation method in the image encoding device 10 described above. The decoded image construction unit 25 generates an entire decoded image from the decoded divided images P0 to PK (step S22).

[Example 2 of image decoding process]
Next, an example in which encoded data encoded by the process described in the image encoding process example 3 of the image encoding apparatus 10 is decoded will be described. In this case, the divided image P0 is encoded by intra prediction encoding, and only a part of the divided images P1 to PK is encoded by inter prediction.

  FIG. 17 is a flowchart of Example 2 of the image decoding process. First, the image decoding device 20 decodes only the information of the decoded divided image among the divided images (step S211). The divided image P0 is decoded by the in-screen prediction decoding unit 23 as in the above-described example, and among the divided images P0 to PK, the encoded divided image is decoded by the inter-screen prediction decoding unit 24.

  Next, the intra-screen interpolation unit 251 performs interpolation of a non-coded divided image from the decoded divided image to generate a non-coded divided image (step S212).

  FIG. 18 is a diagram illustrating an example of interpolation of a divided image by the in-screen interpolation unit 251. In the following, it is assumed that the divided image Pi has been decoded and Bi is a sub-block belonging to the decoded divided image Pi. Further, Bx is a sub-block of the unencoded divided image Px, that is, a sub-block that is not decoded.

  Assuming that the positional relationship in the original image of the sub-block Bi of the divided image Pi and the sub-block Bx of the divided image Px is as shown in FIG. 18A, as shown in FIG. The sub-block Bi located at is extracted. In this example, two sub blocks Bi on the left and right are extracted for one sub block Bx. Next, as shown in FIG. 18C, an interpolation filter is applied to the extracted pixel values of the two left and right sub-blocks Bi to calculate the pixel value of the sub-block Bx ′. As the filter coefficient of the interpolation filter to be used, a coefficient determined in advance by the relative position between the sub-block Bi and the sub-block Bx on the original image is used. Various interpolation methods using an interpolation filter are conventionally known, and any interpolation method may be used to calculate the pixel value of the sub-block Bx ′. The sub-blocks Bx ′ generated by interpolation in this way are collected and used as a divided image Px.

  FIG. 19 is a diagram illustrating an example of intra-screen interpolation. Assume that the image encoding device 10 generates the divided images described with reference to FIG. 9, performs intra-frame predictive coding on the divided image P <b> 0, and inter-screen predictive encodes the divided image P <b> 3. In FIG. 19, a pixel indicated by ○ is a decoded pixel by intra prediction decoding, that is, a decoded pixel of the divided image P0, and a pixel indicated by Δ is a decoded pixel by inter prediction prediction, that is, a decoded pixel of the divided image P3. is there. Pixels indicated by □ are intra-screen interpolation target pixels, that is, pixels of the divided images P1 and P2 that have not been encoded.

For example, the pixel value a of the in-screen interpolation target pixel is calculated as follows. The pixel values of the left and right pixels of the interpolation target pixel in the screen are A and B, and the pixel values of the upper and lower pixels are C and D. Let t be a predetermined threshold value.
(1) When | AB | ≦ t or | CD | ≦ t If | AB | ≦ | CD |, a = (A + B) / 2.

If | A−B |> | C |, then a = (C + D) / 2.
(2) When | A−B |> t and | CD |> t, a = (A + B + C + D) / 4.

  In this interpolation method, when the change in the pixel value in the horizontal direction or the vertical direction is small, the average of the two decoded pixel values in the direction in which the change in the pixel value is small is calculated as the pixel value of the interpolation target pixel in the screen. Otherwise, the average of four decoded pixel values that have been decoded vertically and horizontally is used as the pixel value of the interpolation target pixel in the screen. According to such interpolation of pixel values, it is possible to generate an interpolated image in which changes in pixel values are natural.

  With the image encoding method and the image decoding method described above, the encoding efficiency is reduced by about 20% as compared with the case of the intra prediction encoding using the technique of Non-Patent Document 1, but the decoding calculation amount is about 50%. It was confirmed that it could be reduced.

  FIG. 20 shows an example of a hardware configuration when the image encoding device 10 of FIG. 1 is configured by a computer and a software program. This system includes a CPU 30 that executes a program, a memory 31 such as a RAM that stores programs and data accessed by the CPU 30, and an image signal input unit 32 (disk device) that inputs an image signal to be encoded from a camera or the like. A program storage device 33 in which an image encoding program 34, which is a software program for causing the CPU 30 to execute processing for encoding an input image by this method, and a CPU 30 are provided. An encoded data output unit 35 for outputting encoded data generated by executing the image encoding program 34 loaded in the memory 31 via, for example, a network (a storage unit for storing encoded data by a disk device or the like) However, it may be connected by a bus.

  Note that the image encoding program stored in the program storage device 33 includes, for example, a conventional image encoding program that performs inter-frame prediction in units of frames, in addition to a program that encodes an input image by this method. Also good.

  FIG. 21 shows a hardware configuration example when the image decoding device 20 of FIG. 15 is configured by a computer and a software program. This system receives a CPU 40 that executes a program, a memory 41 such as a RAM that stores programs and data accessed by the CPU 40, and encoded data encoded by the image encoding apparatus 10 of FIG. A program in which an encoded data storage unit 42 (which may be an input unit via a network or the like) to be stored and an image decoding program 44 that is a software program for causing the CPU 40 to execute processing for decoding encoded data by this method is stored. A storage device 43 and a decoded image output unit 45 that outputs a decoded image obtained by decoding the encoded data by the CPU 40 executing the image decoding program 44 loaded in the memory 41 to a reproduction device or the like. , Connected by a bus.

  Note that the image decoding program stored in the program storage device 43 may include, for example, a conventional image decoding program that performs inter-frame prediction in units of frames, in addition to a program that decodes encoded data by this method. .

  The embodiments of the present invention have been described above with reference to the drawings. However, the above embodiments are merely examples of the present invention, and it is obvious that the present invention is not limited to the above embodiments. is there. Accordingly, additions, omissions, substitutions, and other modifications of the components may be made without departing from the spirit and technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Image encoding apparatus 11 Division | segmentation image generation part 12 Intra-screen prediction encoding part 13 Inter-screen prediction encoding part 131 Reference image generation part 132 Prediction error calculation part 133 Prediction error encoding part 134 Image decoding part 14 Information source encoding part DESCRIPTION OF SYMBOLS 20 Image decoding apparatus 21 Encoded data input part 22 Information source decoding part 23 Intra-screen prediction decoding part 24 Inter-screen prediction decoding part 25 Decoded image structure part 251 In-screen interpolation part

Claims (6)

In an image encoding device that compresses and encodes an input image,
When the input image is divided into blocks of n × m pixels, and each divided block is divided into sub-blocks of n ₁ × m ₁ pixels (where 1 ≦ n ₁ <n, 1 ≦ m ₁ <m). A divided image generation unit for setting divided images of the same size, each of which consists of a set of pixels of sub-blocks having the same relative position in the block;
An intra-screen predictive encoding unit for intra-screen predictive encoding at least one of the divided images;
For a divided image other than the divided image that has been subjected to the intra prediction encoding, a predetermined filter determined by a relative position on the input image between the divided image and the encoded divided image is applied to the encoded divided image. An inter-screen predictive coding unit that performs inter-screen predictive coding using the image as a reference image,
The inter-screen predictive encoding unit includes L ₁ for a divided image other than the divided image subjected to the intra-screen predictive encoding, with the first divided image as the first and the last divided image as the K ₁ , _{1 (1 <L 1 ≦ K} 1) th and subsequent divided images without coding, the image coding apparatus characterized by performing inter-picture prediction coding for the other split image. In an image decoding apparatus for decoding encoded data of a compression-encoded image,
The encoded data of the compression-encoded image is obtained by dividing the input image into blocks of n × m pixels by the image encoding device, and dividing each block into n ₁ × m ₁ pixels (where 1 ≦ n ₁ When dividing into sub-blocks of <n, 1 ≦ m ₁ <m), a divided image having the same size, each of which is a set of sub-block pixels having the same relative position in the block, is set and divided. be one encoded for each image, the 0th the first divided image, the last divided image as the _second K, including a _{second L, L 2 (1 <L} 2 ≦ K 2) th and subsequent partial Does not include the code corresponding to the split image,
An input unit for inputting compressed and encoded data;
An intra-screen predictive decoding unit that predicts and decodes at least one of the divided images from the input encoded data;
Refer to an image obtained by applying a predetermined filter determined by the relative position on the original image between the divided image and the decoded divided image to the divided divided image other than the divided image subjected to the intra prediction decoding. As an image, an inter-screen predictive decoding unit that performs inter-screen predictive decoding of at least one or more of the divided images ;
A decoded image constructing unit that configures a decoded image from the divided images decoded by the intra prediction decoding unit and the inter prediction decoding unit;
Using decoded divided image divided images of the intra prediction decoding said not decoded by inter prediction decoding unit and the unit _{L 2 (1 <L 2 ≦} K 2) th and subsequent generated by interpolation, generated by interpolation An image decoding apparatus comprising: an intra-screen interpolation unit that converts the divided image into a decoded divided image. An image encoding method in an image encoding apparatus that includes a divided image generation unit, an intra prediction encoding unit, and an inter prediction encoding unit, and compresses and encodes an input image,
The divided image generation unit divides the input image into blocks of n × m pixels, and each divided block has n ₁ × m ₁ pixels (where 1 ≦ n ₁ <n, 1 ≦ m ₁ <m). A process of setting divided images of the same size each consisting of a set of pixels of sub-blocks having the same relative position in the block when divided into sub-blocks;
The intra prediction encoding unit performing intra prediction encoding of at least one of the divided images;
The inter-screen prediction encoding unit applies a predetermined filter determined by a relative position on an input image between the divided image and the encoded divided image, for a divided image other than the divided image that has been subjected to the intra-screen predictive encoding. A process of performing inter-picture predictive coding using an image applied to an encoded divided image as a reference image,
In the process of the inter-picture prediction coding unit performs inter-picture prediction coding, the divided images except divided images the intra prediction encoding, first the first divided image, _first the last divided image K as includes _{first L, L 1 (1 <L} 1 ≦ K 1) th and subsequent divided images without coding, and performs inter-picture prediction coding for the other divided images Image coding method. Image decoding in an image decoding apparatus that includes an encoded data input unit, an intra-screen prediction decoding unit, an inter-screen prediction decoding unit, a decoded image configuration unit, and an intra-screen interpolation unit, and which decodes encoded data of a compression-encoded image A method,
The encoded data of the compression-encoded image is obtained by dividing an input image into blocks of n × m pixels in an image encoding device, and dividing each divided block into n ₁ × m ₁ pixels (where 1 ≦ n ₁ < When divided into sub-blocks of n, 1 ≦ m ₁ <m), divided images having the same size, each consisting of a set of sub-block pixels having the same relative position in the block, are set. be one obtained by encoding each 0th the first divided image, the last divided image as the _second K, including a _{second L, L 2 (1 <L} 2 ≦ K 2) th and subsequent divided It does not contain the code corresponding to the image,
A process in which the encoded data input unit inputs encoded data of a compression-encoded image;
The intra-screen predictive decoding unit performing intra-screen predictive decoding of at least one of the divided images from the input encoded data;
The inter-frame prediction decoding unit has decoded a predetermined filter determined by a relative position on the original image between the divided image and the decoded divided image, for a divided image other than the divided image subjected to the intra-screen predictive decoding. A process of performing inter-screen predictive decoding of at least one of the divided images using an image applied to the divided image as a reference image;
The decoded image construction unit constructing a decoded image from the divided images decoded by the intra prediction decoding and the inter prediction decoding;
The screen interpolation unit is interpolated using decoded divided image divided images of the intra prediction decoding said not decoded by inter prediction decoding unit and the unit _{L 2 (1 <L 2 ≦} K 2) th and subsequent And a process of converting the divided image generated by interpolation into a decoded divided image. An image encoding program for causing a computer to execute the image encoding method according to claim 3. An image decoding program for causing a computer to execute the image decoding method according to claim 4.

Images