JP2014033465A - Image encoder, image decoder, image decoding method and image decoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image decoder which reduces an information amount and improves efficiency.SOLUTION: An image decoder decodes a signal obtained by performing orthogonal transformation, quantization and encoding on an input image signal composed of a plurality of unit blocks having a plurality of pixels. When all combinations of chrominance signals, which include all non-application of selection of a reference signal and inter-signal prediction to respective predicted signals, are determined on propriety of application of selection of the reference signal from signals separated into chrominance signals and inter-signal prediction to the respective predicted signals, the applied combination of the signals is selected so that encoding cost calculated from a weight sum of a generated code amount and a distortion amount becomes minimum. The combination is encoded as inter-signal prediction information. When a code string obtained by performing orthogonal transformation, quantization and encoding on an inter-signal prediction residue signal is decoded, an inter-signal prediction signal obtained from the encoded inter-signal prediction information and an image signal decoded by using an intra-signal prediction signal are obtained.

Description

  The present invention relates to an image encoding technique in the case of compressing and encoding an image, and in particular, predicts a coding target from correlated information in the image and encodes a prediction error to improve encoding efficiency. The present invention relates to an image encoding device, an image decoding device, an image decoding method, and an image decoding program.

As a method for improving the coding efficiency in the conventional image coding, there are a method for reducing temporal redundancy and a method for reducing spatial redundancy.
As a method for reducing temporal redundancy, there are a frame difference method and a motion compensation method. In the frame difference method, two consecutive images are simply subtracted and the difference is encoded.
In the motion compensation method, an approximate image of an encoding target frame is generated by applying a motion vector to a reference frame, and a difference from the encoding target frame is encoded. This motion compensation method is superior to the frame difference method in terms of encoding efficiency because encoding is performed after reducing differences between images.
On the other hand, the motion vector estimation method used in the motion compensation method has a problem that it can not be used for still images, although it can reduce time redundancy.

  As a spatial redundancy reduction method for improving the coding efficiency, for example, there is a method of quantizing orthogonal transform coefficients. In this method, orthogonal transformation maps a pixel signal to a frequency domain and concentrates energy in a low frequency. According to this method, it is possible to increase the coding efficiency by removing the high frequency component by quantization using the fact that the human visual characteristic is not sensitive to the high frequency.

  In addition, when encoding is performed, as shown in Patent Document 1, a decoded signal is generated by performing local decoding on an encoded block existing around a region to be encoded. The prediction signal is generated by performing in-plane prediction (intra prediction) using this decoded signal, and the residual obtained by the difference between the signal component in the block to be encoded and the generated prediction signal In general, a signal is subjected to predetermined orthogonal transformation and quantization to reduce spatial redundancy and improve coding efficiency.

Japanese Patent No. 3734492

  The spatial redundancy reduction method using the combination of orthogonal transformation and quantization and the spatial prediction method described above reduces spatial redundancy in a frame, but different color signals in the same frame are processed independently, There was a problem that redundancy could not be reduced.

  The present invention has been proposed in view of the above circumstances, and a reference signal serving as a reference is adaptively selected from a plurality of color signals obtained by separating images, and another signal (covered signal) is selected from the reference signal. An object of the present invention is to provide an image encoding device, an image decoding device, an image decoding method, and an image decoding program that reduce the amount of information of a signal to be predicted and improve encoding efficiency by predicting (predicted signal).

  In order to achieve the above object, the present invention provides an intra-signal prediction residual signal obtained by performing a difference process between each pixel of a unit block composed of a plurality of pixels and each pixel predicted in the signal. When performing orthogonal transform / quantization / encoding for each unit block, the intra-signal prediction residual signal in the unit block is separated into a plurality of signals composed of one reference signal and a predicted signal. By performing inter-signal prediction, it is possible to reduce the redundancy of color signals.

Claim 1 is an image coding apparatus that performs orthogonal transform, quantization, and coding on an input image signal composed of a plurality of unit blocks each having a plurality of pixels, and performs coding for each unit block.
Based on the reconstructed image signal corresponding to the encoded region obtained by decoding the orthogonally transformed and quantized image signal, the signal in the signal corresponding to the encoding target region of the input image signal. Intra-signal prediction means for calculating intra-signal prediction information for prediction;
Intra-signal compensation means for obtaining an intra-signal prediction signal from the intra-signal prediction information from the intra-signal prediction means and the reconstructed image signal;
The intra-signal prediction residual signal obtained by the difference process between the input image signal that is the encoding target region and the intra-signal prediction signal is separated into the reference signal and the predicted signal and corresponds to each pixel of the reference signal. Inter-signal prediction means for calculating inter-signal prediction information for inter-signal prediction for each pixel of the predicted signal;
A signal for obtaining an inter-signal prediction signal in a coding target region from an intra-decoded prediction residual signal obtained by decoding an orthogonally transformed and quantized image signal and inter-signal prediction information from the inter-signal prediction means Compensation means,
By performing orthogonal transform / quantization / coding on the inter-signal prediction residual signal obtained by performing the difference process between the intra-signal prediction residual signal and the inter-signal prediction signal, each pixel of the predicted signal is obtained. While encoding
The reference signal and the predicted signal in the inter-signal prediction means are separated signals obtained by separating the intra-signal prediction residual signal into color signals,
The inter-signal prediction means includes
When determining the combination of the selection of the reference signal from the separated signal and the applicability of the inter-signal prediction to the predicted signal, the variance or amplitude of the neighboring pixels, the average, and the intra-signal prediction information are used as the applied signal combination. By calculating
It is characterized in that the combination is used as inter-signal prediction information without encoding.

A second aspect of the present invention provides an image decoding apparatus that decodes a code string obtained by performing orthogonal transform, quantization, and encoding on the inter-signal prediction residual signal by the image encoding apparatus according to claim 1,
When decoding using the inter-signal prediction signal and the intra-signal prediction signal,
For the combination of selection of the reference signal and applicability of inter-signal prediction to the signal to be predicted, an image signal to be decoded is obtained using the inter-code prediction information that is not encoded.

An image decoding apparatus that decodes a signal obtained by orthogonally transforming, quantizing, and encoding an input image signal composed of a plurality of unit blocks each having a plurality of pixels.
Based on the reconstructed image signal corresponding to the encoded region obtained by decoding the orthogonally transformed and quantized image signal, the signal in the signal corresponding to the encoding target region of the input image signal. Intra-signal prediction means for calculating intra-signal prediction information for prediction;
Intra-signal compensation means for obtaining an intra-signal prediction signal from the intra-signal prediction information from the intra-signal prediction means and the reconstructed image signal;
The intra-signal prediction residual signal obtained by the difference process between the input image signal that is the encoding target region and the intra-signal prediction signal is separated into the reference signal and the predicted signal and corresponds to each pixel of the reference signal. Inter-signal prediction means for calculating inter-signal prediction information for inter-signal prediction for each pixel of the predicted signal;
A signal for obtaining an inter-signal prediction signal in a coding target region from an intra-decoded prediction residual signal obtained by decoding an orthogonally transformed and quantized image signal and inter-signal prediction information from the inter-signal prediction means Compensation means,
By performing orthogonal transform / quantization / coding on the inter-signal prediction residual signal obtained by performing the difference process between the intra-signal prediction residual signal and the inter-signal prediction signal, each pixel of the predicted signal is obtained. While encoding
The reference signal and the predicted signal in the inter-signal prediction means are separated signals obtained by separating the intra-signal prediction residual signal into color signals,
The inter-signal prediction means selects a reference signal from signals separated into color signals and determines whether or not to apply inter-signal prediction to each predicted signal. When determining all combinations of color signals including non-application, select the combination of signals to be applied so as to minimize the encoding cost calculated from the weighted sum of the generated code amount and the distortion amount. Encode the combination as inter-signal prediction information,
When performing decoding on a code string obtained by performing orthogonal transform, quantization, and encoding on the inter-signal prediction residual signal,
An inter-signal prediction signal obtained from the encoded inter-signal prediction information and an image signal decoded using the intra-signal prediction signal are obtained.

  According to a fourth aspect of the present invention, in the image decoding device according to the third aspect, the inter-signal predicting means predicts and predicts each unit block when predicting a signal between the reference signal and the signal to be predicted. In this case, the presence or absence of prediction can be set for each pixel signal in the unit block.

  According to a fifth aspect of the present invention, in the image decoding apparatus according to the third aspect, when the inter-signal prediction means determines a combination of selection of a reference signal and applicability of inter-signal prediction to a predicted signal from the separated signals. The combination is calculated using the variance or amplitude, average, and intra-signal prediction information of neighboring pixels.

  According to a sixth aspect of the present invention, in the image decoding apparatus according to the third aspect, when the inter-signal prediction means determines a combination of selection of a reference signal and applicability of inter-signal prediction to a predicted signal from the separated signals. A dictionary is created in advance by statistical machine learning from the reference signal that minimizes the coding cost calculated from the sum of the weights of the generated code amount and the distortion amount, the optimal combination, and the input signal. The combination with respect to the signal input to is estimated.

  According to a seventh aspect of the present invention, in the image decoding apparatus according to the third aspect, the inter-signal prediction means selects a signal having a small change not to be a reference signal.

  According to an eighth aspect of the present invention, in the image decoding apparatus according to the third aspect, the inter-signal prediction means selects a signal having a large change not to be a predicted signal.

  9. The image decoding apparatus according to claim 3, wherein when the reference signal is selected and the inter-signal prediction is applied to the signal to be predicted, the inter-signal prediction is performed with a prediction coefficient for each region. It is characterized by estimating.

  According to a tenth aspect of the present invention, in the image decoding apparatus according to the ninth aspect, the prediction coefficient in the inter-signal prediction unit is configured by a multiplier and a correction value.

  An image decoding apparatus according to claim 9 is characterized in that, in the image decoding apparatus according to claim 9, the prediction coefficient in the inter-signal prediction means is estimated so as to minimize an error between a prediction signal based on a reference signal and a predicted signal.

An image decoding method for decoding a signal obtained by orthogonally transforming, quantizing, and encoding an input image signal composed of a plurality of unit blocks having a plurality of pixels,
Based on the reconstructed image signal corresponding to the encoded region obtained by decoding the orthogonally transformed and quantized image signal, the signal in the signal corresponding to the encoding target region of the input image signal. A procedure for calculating in-signal prediction information for prediction;
Obtaining an intra-signal prediction signal from the intra-signal prediction information from the intra-signal prediction means and the reconstructed image signal;
The intra-signal prediction residual signal obtained by the difference process between the input image signal that is the encoding target region and the intra-signal prediction signal is separated into the reference signal and the predicted signal and corresponds to each pixel of the reference signal. A procedure for calculating inter-signal prediction information for inter-signal prediction for each pixel of the predicted signal;
A procedure for obtaining an inter-signal prediction signal in an encoding target region from an intra-decoded prediction residual signal obtained by decoding an orthogonally transformed and quantized image signal and inter-signal prediction information from the inter-signal prediction means When,
By performing orthogonal transform / quantization / coding on the inter-signal prediction residual signal obtained by performing the difference process between the intra-signal prediction residual signal and the inter-signal prediction signal, each pixel of the predicted signal is obtained. And a procedure for performing encoding,
The reference signal and the signal to be predicted in the inter-signal prediction information are separated signals obtained by separating the intra-signal prediction residual signal into color signals,
When determining the combination of the selection of the reference signal from the separated signal and the applicability of the inter-signal prediction to the predicted signal, the variance or amplitude of the neighboring pixels, the average, and the intra-signal prediction information are used as the applied signal combination. By calculating
While making the combination inter-signal prediction information without encoding,
When decoding using the inter-signal prediction signal and the intra-signal prediction signal,
For the combination of selection of the reference signal and applicability of inter-signal prediction to the signal to be predicted, an image signal to be decoded is obtained using the inter-code prediction information that is not encoded.

An image decoding method for performing decoding of a signal obtained by orthogonally transforming, quantizing, and encoding an input image signal composed of a plurality of unit blocks having a plurality of pixels,
Based on the reconstructed image signal corresponding to the encoded region obtained by decoding the orthogonally transformed and quantized image signal, the signal in the signal corresponding to the encoding target region of the input image signal. A procedure for calculating in-signal prediction information for prediction;
Obtaining an intra-signal prediction signal from the intra-signal prediction information from the intra-signal prediction means and the reconstructed image signal;
The intra-signal prediction residual signal obtained by the difference process between the input image signal that is the encoding target region and the intra-signal prediction signal is separated into the reference signal and the predicted signal and corresponds to each pixel of the reference signal. A procedure for calculating inter-signal prediction information for inter-signal prediction for each pixel of the predicted signal;
A procedure for obtaining an inter-signal prediction signal in an encoding target region from an intra-decoded prediction residual signal obtained by decoding an orthogonally transformed and quantized image signal and inter-signal prediction information from the inter-signal prediction means When,
By performing orthogonal transform / quantization / coding on the inter-signal prediction residual signal obtained by performing the difference process between the intra-signal prediction residual signal and the inter-signal prediction signal, each pixel of the predicted signal is obtained. And a procedure for performing encoding,
The reference signal and the signal to be predicted in the inter-signal prediction information are separated signals obtained by separating the intra-signal prediction residual signal into color signals,
Color signal including selection of reference signal and non-application of inter-signal prediction to each signal to be predicted regarding whether or not to apply reference signal to each predicted signal from the signal separated into color signals When determining all the combinations, the signal combination to be applied is selected so as to minimize the coding cost calculated from the weighted sum of the generated code amount and the distortion amount, and the combination is used as inter-signal prediction information. Perform encoding,
When performing decoding on a code string obtained by performing orthogonal transform, quantization, and encoding on the inter-signal prediction residual signal,
An inter-signal prediction signal obtained from the encoded inter-signal prediction information and an image signal decoded using the intra-signal prediction signal are obtained.

  A fourteenth aspect of the present invention is an image decoding program that causes a computer to execute each procedure according to the twelfth or thirteenth aspect.

According to the image encoding device of claim 1, in the image encoding device, a reference signal serving as a reference is adaptively selected from a plurality of signals obtained by separating the image signal, and other signals are used as a reference. By predicting from a signal, it is possible to reduce spatial redundancy, reduce the amount of information of a signal to be predicted, and improve encoding efficiency.
Further, by performing inter-signal prediction using a separated signal obtained by separating the reference signal and the predicted signal into color signals, it is possible to reduce the redundancy of the color signals.
Further, the information reduction effect can be achieved by using the inter-signal prediction information without encoding the selection of the reference signal and whether or not the inter-signal prediction can be applied to the predicted signal.

  According to the image decoding apparatus of the second aspect, since decoding is performed using inter-signal prediction information that is not encoded, an information reduction effect can be achieved.

According to the image decoding device of claim 3, in the image encoding device, a reference signal serving as a reference is adaptively selected from a plurality of signals obtained by separating the image signal, and other signals are used as the reference signal. Therefore, it is possible to reduce spatial redundancy, reduce the amount of information of a signal to be predicted, and improve encoding efficiency.
Further, by performing inter-signal prediction using a separated signal obtained by separating the reference signal and the predicted signal into color signals, it is possible to reduce the redundancy of the color signals.
For inter-signal prediction information that determines a combination of selection of a reference signal from separated signals and applicability of inter-signal prediction to each predicted signal, encoding in which a combination of applied signals is calculated from a code amount and a distortion amount By selecting so as to minimize the cost, an appropriate signal prediction can be performed.

  According to the image decoding device of claim 4, signal prediction between signals can be predicted for each unit block, and the presence or absence of prediction can be set for each pixel signal in the unit block. Predictions can be made.

  According to the image decoding device of claim 5, when determining a combination of selection of the reference signal from the separated signals and whether or not to apply the inter-signal prediction to the predicted signal, the combination of the signals to be applied is distributed to neighboring pixels. Alternatively, it can be calculated using amplitude, average, and intra-signal prediction information.

  According to the image decoding apparatus of the sixth aspect, it is possible to perform proper signal prediction by estimating the reference signal and the combination with respect to the input signal based on the dictionary.

  According to the image decoding device of the seventh aspect, it is possible to improve the image quality by selecting a signal having a small change not to be a reference signal.

  According to the image decoding apparatus of the eighth aspect, it is possible to reduce the code amount by selecting a signal having a large change not to be a predicted signal.

  According to the image decoding device of claim 9, when the reference signal is selected and the inter-signal prediction is applied to the predicted signal, the inter-signal prediction suitable for each region is performed by estimating the prediction coefficient for each region. It can be performed.

  According to the image decoding apparatus of the tenth aspect, when the reference signal is selected and the inter-signal prediction is applied to the predicted signal, the inter-signal prediction can be performed using the prediction coefficient configured by the multiplier and the correction value.

  According to the image decoding apparatus of the eleventh aspect, the inter-signal prediction with good accuracy is performed by estimating the error between the prediction signal based on the reference signal and the predicted signal with respect to the prediction coefficient in the inter-signal prediction unit. be able to.

It is a block diagram which shows an example of embodiment of the image coding apparatus of this invention. It is explanatory drawing of the input image input into an image coding apparatus. It is a flowchart figure which shows the calculation procedure of the prediction information between signals in the signal prediction means and the signal compensation means of an image coding apparatus, and the signal prediction signal between signals.

An example of an embodiment of an image encoding device according to the present invention will be described with reference to FIGS.
The image coding apparatus according to the present invention performs difference processing between each pixel of a unit block composed of a plurality of existing pixels and each pixel predicted in the signal from the coded pixel. The obtained intra-prediction residual signal is subjected to orthogonal transform / quantization / encoding to perform encoding for each unit block. It is characterized in that it is separated into a plurality of signals composed of one reference signal and a signal to be predicted, and a function for predicting signals between the plurality of signals is added.

  That is, as shown in FIG. 1, the image coding apparatus of the present invention includes a transforming means 1 for transforming into a frequency domain by orthogonal transform, a quantizing means 2 for quantizing orthogonal transform coefficients, and a quantized orthogonal transform. Encoding means 3 for variable-length encoding the coefficients, inverse quantization means 4 for inversely quantizing the quantized orthogonal transform coefficients, inverse transform means 5 for inversely transforming the inversely quantized orthogonal transform coefficients, For an image coding apparatus including intra-signal prediction means 6 for determining intra-signal prediction information for reducing spatial redundancy, and intra-signal compensation means 7 for reconstructing an intra-signal prediction signal by predicting spatial redundancy. The inter-signal prediction means 8 for predicting signals between a plurality of signals and the inter-signal compensation means 9 for obtaining inter-signal prediction signals are provided.

  The first subtractor 11 reduces spatial redundancy, and for the pixels in the encoding target region, the intra-signal prediction signal predicted from the input image signal and the encoded pixel sent from the intra-signal compensation means 7. And calculate the difference. The intra-signal prediction residual signal obtained by the difference is sent to the inter-signal prediction means 8 and the second differentiator 12, respectively.

  The second subtracter 12 reduces inter-signal redundancy, and the intra-signal prediction residual signal sent from the first subtractor 11 and the inter-signal compensation means 9 are sent from the pixel in the encoding target region. The difference with the inter-signal prediction signal predicted from the encoded pixel to be calculated is calculated. The inter-signal prediction residual signal obtained by the difference is sent to the conversion means 1.

  The first adder 13 compensates for inter-signal redundancy. The reproduced (decoded) inter-signal prediction residual signal sent from the inverse transform means 5 and the inter-signal prediction sent from the inter-signal compensation means 9 are used. The intra-signal prediction residual signal is reconstructed (decoded) by calculating the sum with the signal. The intra-signal prediction residual signal obtained by the addition is sent to the inter-signal prediction means 8, the inter-signal compensation means 9, and the second adder 14, respectively.

  The second adder 14 compensates for inter-signal redundancy, and is the sum of the intra-signal prediction residual signal sent from the first adder 13 and the intra-signal prediction signal sent from the intra-signal compensation means 7. To reconstruct the image signal (reconstructed image signal) corresponding to the encoded pixel. The reconstructed pixel signal obtained by the addition is sent to the intra-signal prediction means 6 and the intra-signal compensation means 7, respectively.

An input image and an inter-signal prediction error signal sent from the second differentiator 12 are input to the transform unit 1, the signal is transformed into the frequency domain by orthogonal transform, and the transform coefficient obtained by the orthogonal transform is sent to the quantizer 2. Output. As the orthogonal transform, DCT to DCT approximate transform or DWT can be used.
Each picture (frame) of the input image is a unit block composed of a predetermined number of pixels (for example, 32 × 32 pixels, 16 × 16 pixels, 8 × 8 pixels, 4 × 4 pixels, or a combination thereof). And is encoded for each unit block. As shown in FIG. 2, each picture (frame) of the input image is composed of, for example, an R signal frame 20, a G signal frame 30, and a B signal frame 40 that are separated into RGB signals in a color space. Each unit block is configured to have a G signal block and a B signal block spatially corresponding to the R signal block 21.

When image coding is performed by the conventional method, only processing for reducing spatial redundancy is performed for each frame of the RGB signal based on the intra-signal prediction signal. For example, when the image of the R signal block 21 has a contour, a pattern, or the like (shaded portion), a coding target is predicted from surrounding information with correlation in the image, and a prediction error is encoded. ing.
In this example, an input image obtained by dividing each picture into unit blocks is input to the conversion unit 1, and the conversion unit 1 receives signals from each pixel of each RGB signal block or the second subtractor 12. A process for reducing inter-signal redundancy is performed by converting a difference value obtained with each spatially corresponding pixel based on the input inter-signal prediction residual signal.

The quantization unit 2 quantizes the transform coefficient sent from the transform unit 1. The quantized value obtained by the quantization is output to the encoding unit 3 and the inverse quantization unit 4.
The quantization parameter used for the quantization process can be set as a combination of constant values. Alternatively, the output bit rate can be kept constant by controlling according to the information amount of the transform coefficient.

  The encoding means 3 encodes the quantized transform coefficient sent from the quantization means 2 and outputs it as code information. For the encoding, a variable length code or an arithmetic code that removes redundancy between codes can be used.

  The inverse quantization means 4 inversely quantizes the quantized transform coefficient sent from the quantization means 2 by performing the reverse procedure of the quantization process. The transform coefficient including the quantization error obtained by the inverse quantization is sent to the inverse transform means 5.

  The inverse transform unit 5 performs an inverse orthogonal transform on the transform coefficient including the quantization error sent from the inverse quantization unit 4 by performing the inverse procedure of the orthogonal transform. The inter-signal prediction residual signal including the quantization error obtained by the inverse transformation is sent to the first adder 13.

Next, the calculation of the inter-signal prediction signal input to the second subtractor 12 in order to obtain the inter-signal prediction residual signal input to the conversion means 1 which is a characteristic configuration of the image encoding device of the present invention. explain.
The inter-signal prediction signal is calculated by calculating the inter-signal prediction means 8 and the inter-signal prediction means 6 and the intra-signal compensation means 7 to which the intra-signal prediction residual signal is input via the first adder 13. This is done by providing compensation means 9. Hereinafter, functions of the intra-signal prediction unit 6, the intra-signal compensation unit 7, the inter-signal prediction unit 8, and the inter-signal compensation unit 9 will be described.

The intra-signal prediction means 6 determines intra-signal prediction information for reducing spatial redundancy, and a reconstructed pixel signal corresponding to an encoded pixel including a quantization error sent from the second adder 14. Based on the above, in-signal prediction information for approximating the input signal is determined. The determined intra-signal prediction information is sent to the intra-signal compensation means 7 and the encoding means 3. In FIG. 1, the line to the quantizing means 2 is omitted, but a part of the intra-signal prediction information is quantized and encoded.
Various methods have been conventionally used for intra-signal prediction. For example, as an example, the standardized H.264 standard. When using H.264 intra prediction, an intra prediction mode (rate distortion optimization method) that individually encodes each intra prediction mode and minimizes the coding cost calculated from the code amount and the distortion amount is selected. This is assumed to be intra-signal prediction information. For example, in FIG. 2, when encoding an R signal block (unit block) 21, prediction is performed from an image (particularly, an image directly above or to the left) 22 in a hatched portion in the same encoded R signal frame 21. In-signal prediction information is obtained.

  The intra-signal compensation means 7 predicts spatial redundancy and reconstructs the intra-signal prediction signal. The intra-signal prediction information sent from the intra-signal prediction means 6 and the reconfiguration sent from the second adder 14 An intra-signal prediction signal of the region is generated from the pixel signal. The intra-signal prediction signal is sent to the first subtractor 11 and the second adder 14.

The inter-signal prediction means 8 determines inter-signal prediction information for reducing inter-signal redundancy.
The inter-signal prediction unit 8 predicts signals between a plurality of signals by separating the intra-signal prediction residual signal in the unit block into a plurality of signals composed of one reference signal and a predicted signal.
The inter-signal prediction means 8 approximates the intra-signal prediction residual signal sent from the first subtractor 11 based on the intra-signal prediction residual signal containing the quantization error sent from the first adder 13. The coefficient to do is calculated.
The calculated prediction coefficient is sent to the inter-signal compensation unit 9 and the encoding unit 3 as inter-signal prediction information. In FIG. 1, the line to the quantizing means 2 is omitted, but a part of the inter-signal prediction information is quantized and encoded.

Hereinafter, the calculation procedure of the prediction coefficient by the inter-signal prediction unit 8 and the generation procedure of the inter-signal prediction signal by the inter-signal compensation unit 9 will be described with reference to FIG.
First, in the inter-signal prediction means 8, the input signal is separated into a plurality of signals for calculation of the prediction coefficient (step 51). The type and number of signals are not limited, but as described above, as an example, the unit block is separated into RGB signals in the color space. In addition, when separating into color space signals, it is also possible to use color space signals such as YUV signals and YCbCr signals.

  Next, the separated signals are classified into a reference signal as a reference of the inter-signal prediction means 8 and a predicted signal to be predicted, and whether or not the inter-signal prediction information can be applied to the predicted signal ( Step 52), inter-signal prediction information is calculated from the reference signal (step 53). Both the reference signal and the signal to be predicted may be single or plural. A predetermined reference signal may be used in a fixed manner, or the reference signal may be changed within a certain range such as a region.

For example, when signal separation is performed on an input image using RGB signals, the smoothness of the signal to be predicted can be accurately reproduced by using the G signal as a reference signal for images with many green leaves such as leaves. Therefore, it is possible to reduce the difference between the signal to be predicted and the prediction signal to improve the encoding efficiency.
That is, image quality can be improved by selecting a signal having a large change as an image signal as a reference signal. On the contrary, since a signal with a large change as an image signal occupies a large amount of code, the code amount can be reduced by selecting a signal with a large change as a predicted signal.

In obtaining the signal to be predicted, whether to apply inter-signal prediction from the reference signal is selected. This selection is determined by a rate distortion optimization method considering inter-signal prediction information and the like.
The calculation of the inter-signal prediction information is preferably predictable for each unit block, and in the case of prediction, it is preferable that the presence / absence of prediction can be set for each pixel signal in the unit block.

When the reference signal is changed depending on the region or the like, it is possible to encode (express) or not (implicit) information indicating selection of the reference signal and whether to apply signal prediction. .
When clearly indicating, all signals are individually encoded, a signal that minimizes the encoding cost calculated from the code amount and the distortion amount is selected, and information representing the signal is included in the inter-signal prediction information. Similarly, when the signal to be predicted is limited, information representing the signal is included in the inter-signal prediction information. For example, the encoding cost is calculated as a weight sum (R + λD), where R is the code amount and D is the distortion amount.

In other words, for the RGB color signals, there are the following 10 combinations as to whether or not the reference signal can be selected and inter-signal prediction can be applied.
(1) R is used as a reference signal (reference), G signal prediction is applied (applied), B is applied (2) R criterion, G is applied, B signal prediction is not applied (not applied)
(3) R standard, G non-application, B application (4) G standard, R application, B application (5) G standard, R application, B non-application (6) G standard, R non-application, B application (7) B standard, G application, R application (8) B standard, G application, R non application (9) B standard, G non application, R application (10) non application By selecting the signal having the smallest value, it is possible to determine whether or not the selection of the reference signal for the RGB signal and the application of the signal prediction to each predicted signal can be performed.

  In this case (when specified), for example, 2 bits indicating whether the reference signal is RGB or not, and 2 bits indicating whether the inter-signal prediction is applied to the predicted signal with respect to the inter-signal prediction information. And can be encoded by variable length encoding.

On the other hand, in the case of implied, if a signal rich in change is used as a reference signal, the accuracy of prediction is improved. By selecting the reference signal, information representing the signal is not included in the inter-signal prediction information.
Alternatively, if a signal rich in change is used as the predicted signal, the encoding efficiency is improved. Therefore, the signal having the maximum variance, amplitude, and average between neighboring pixels of the reference signal and the predicted signal is selected and decoded. In addition, by selecting a signal to be predicted by the same means, information representing the signal is not included in the inter-signal prediction information.
Alternatively, if a mode in which intra-signal prediction does not impair the high correlation of the original image is selected, redundancy between signals remains in the intra-signal prediction error signal. You can also decide what you want.

  The inter-signal prediction means statistically machine-learns an optimal combination of the input signal and the input signal in advance when determining a combination of the reference signal and the signal to be predicted from the separated signals, and based on the generated dictionary. You may comprise so that the reference signal and combination with respect to the input signal may be estimated. Any classifier such as a known Decision Tree, Random Forest, or Neural Network can be used for machine learning. As the feature amount used for the discriminator, neighboring pixel information of the encoding target region, intra-signal prediction information, and the input image signal itself are used. In encoding moving images, not only a dictionary is created in advance, but also the above-mentioned encoding cost (a value calculated from the sum of weights of generated code amount and distortion amount) for each fixed frame or cut point. It is also possible to apply full-time learning by searching for an optimal combination that minimizes.

Next, when a reference signal is determined and inter-signal prediction is applied, a prediction formula and a prediction coefficient for approximating a signal to be predicted from the reference signal will be described.
As an example, when the YUV signal is separated into a color space signal, the Y signal is a reference signal, and the U signal and the V signal are prediction signals, the prediction means uses a linear combination multiplier a and a correction value b as prediction coefficients, Can be estimated in small area units. A prediction formula for the U signal and the V signal of the coordinate vector x belonging to a certain small region R is given by Equation 1.

  Here, Y (x), U (x), and V (x) each represent a pixel value of the YUV signal. X in () is a vector. However, the reference signal includes a quantization error to prevent error propagation.

The prediction coefficient is estimated so as to minimize the square of the prediction error.
Specifically, an example of a calculation method for the prediction coefficients a _u and b _u for the U signal will be described. The square E ² of the prediction error is expressed by Equation 2.

At this time, the partial differentiation of the square error E _u ^{2 by} the coefficients a and b is expressed by Equation 3.

Here, n represents the number of pixels belonging to the small region R.
Since Equation 3 needs to be “0” in order to minimize the square error E ² , the prediction coefficients a _u and b _u can be calculated by solving Equation 4.

When Equation 4 is solved, a multiplier a _u and a correction value b _u that minimize the square error E ² are obtained by _Equation 5.

Since the correction value b _u can be derived using a multiplier a _u, when encoding quantized multiplier a _u is possible to improve the prediction accuracy by calculating the correction value in multiplier including a quantization error Is possible.
In the above calculation, the prediction coefficients a _u and b _u for the U signal are obtained, but the prediction coefficients a _v and b _v for the V signal are the same by simply replacing U (vector x) in Formula 5 with V (vector x). It is possible to use this method.

The inter-signal compensation means 9 predicts inter-signal redundancy and reconstructs the inter-signal prediction signal. The inter-signal prediction information from the inter-signal prediction means 8 and the intra-signal prediction from the first adder 13 are used. An inter-signal prediction signal is obtained from the residual signal (step 54). That is, the inter-signal compensation means 9 generates an inter-signal prediction signal that approximates the predicted signal from the inter-signal prediction information sent from the inter-signal prediction means 8 and the inter-signal prediction residual signal sent from the first adder 13. To do. The generated inter-signal prediction signal is output to the first adder 13 and the second differentiator 12.
In the above example, when the inter-signal prediction means 8 uses linear combination and the inter-signal prediction information is composed of the multiplier a and the correction value b, the prediction signal is generated by the above-described formula 1.

According to the image encoding apparatus having the above-described configuration, when the conversion unit 1 performs conversion on the pixels in the encoding target area of each unit block, signals separated in the color space (RGB signal, YUV signal, YCbCr signal, etc.) ), The difference value of each spatially corresponding pixel is converted based on the inter-signal prediction error signal input from the second differentiator 12, and is encoded by the quantization means 2 and the encoding means 3.
The inter-signal prediction error signal input to the conversion means 1 is calculated by performing intra-signal prediction and inter-signal prediction from the regenerated pixel signal of the already encoded image region and the regenerated intra-signal prediction error signal.

  That is, when encoding is performed for each pixel signal in the encoding target region of the unit block, the encoded pixel signal around the encoding target pixel is passed through the inverse quantization unit 4 and the inverse transform unit 5, The inter-signal prediction residual signal is reproduced, the intra-signal prediction residual signal is reproduced via the first adder 13 and the second adder 14, and is output to the intra-signal prediction means 6 and the intra-signal compensation means 7. The At the same time, an intra-signal prediction residual signal is reproduced via the first adder 13 and output to the inter-signal prediction means 8 and the inter-signal compensation means 9.

An input image is input to the intra-signal prediction unit 6, an intra-signal prediction signal is generated by the intra-signal compensation unit 7 based on the intra-signal prediction information, and is output to the first subtractor 11.
In the first differentiator 11, an intra-signal prediction residual signal is generated based on the difference between the input image and the intra-signal prediction signal, and is output to the inter-signal prediction unit 8 and the second differentiator 12.

In the inter-signal prediction means 8, inter-signal prediction information is calculated from the intra-signal prediction residual signal from the first subtractor 11 and the intra-signal prediction residual signal from the first adder 13, and the inter-signal prediction signal is calculated. Is generated and output to the second subtractor 12.
In the second subtractor 12, an inter-signal prediction residual signal is generated by the difference between the intra-signal prediction residual signal from the first subtractor 11 and the inter-signal prediction signal, and this inter-signal prediction residual signal is converted into a conversion means. 1 is used to perform intra prediction from a pixel signal of an already encoded image region, and to predict other signals from a reference signal in a color space to obtain a predicted signal, thereby providing spatial redundancy. The amount of information of the signal to be predicted can be reduced, and the coding efficiency can be improved.

According to the above-described image encoding device, the input signal is separated into a plurality of signals, and other signals are predicted from the reference signal to employ signal prediction that reduces the amount of information generated in the predicted signal. High encoding efficiency is possible. Further, it can be combined with a conventional prediction method that reduces temporal redundancy and the like, and it is possible to further improve the coding efficiency.
In addition, a code string obtained by performing orthogonal transform, quantization, and coding on the inter-signal prediction residual signal described above is output from the encoding means 3, but in the image decoding device (not shown), the code string By using the inter-signal residual signal obtained by decoding, inverse quantization, and inverse transform based on the signal, the inter-signal prediction signal, and the intra-signal prediction signal, a process reverse to that of the image encoding device is performed. Can be decrypted.

  DESCRIPTION OF SYMBOLS 1 ... Conversion means, 2 ... Quantization means, 3 ... Coding means, 4 ... Inverse quantization means, 5 ... Inverse conversion means, 6 ... Intra-signal prediction means, 7 ... Intra-signal compensation means, 8 ... Inter-signal prediction means 9 ... Inter-signal compensation means, 11 ... First differencer, 12 ... Second differencer, 13 ... First adder, 14 ... Second adder, 20 ... R signal frame, 21 ... R signal Block (unit block).

Claims (14)

In an image coding apparatus that performs orthogonal transform / quantization / coding on an input image signal composed of a plurality of unit blocks having a plurality of pixels to perform coding for each unit block,
Based on the reconstructed image signal corresponding to the encoded region obtained by decoding the orthogonally transformed and quantized image signal, the signal in the signal corresponding to the encoding target region of the input image signal. Intra-signal prediction means for calculating intra-signal prediction information for prediction;
Intra-signal compensation means for obtaining an intra-signal prediction signal from the intra-signal prediction information from the intra-signal prediction means and the reconstructed image signal;
The intra-signal prediction residual signal obtained by the difference process between the input image signal that is the encoding target region and the intra-signal prediction signal is separated into the reference signal and the predicted signal and corresponds to each pixel of the reference signal. Inter-signal prediction means for calculating inter-signal prediction information for inter-signal prediction for each pixel of the predicted signal;
A signal for obtaining an inter-signal prediction signal in a coding target region from an intra-decoded prediction residual signal obtained by decoding an orthogonally transformed and quantized image signal and inter-signal prediction information from the inter-signal prediction means Compensation means,
By performing orthogonal transform / quantization / coding on the inter-signal prediction residual signal obtained by performing the difference process between the intra-signal prediction residual signal and the inter-signal prediction signal, each pixel of the predicted signal is obtained. While encoding
The reference signal and the predicted signal in the inter-signal prediction means are separated signals obtained by separating the intra-signal prediction residual signal into color signals,
The inter-signal prediction means includes
When determining the combination of the selection of the reference signal from the separated signal and the applicability of the inter-signal prediction to the predicted signal, the variance or amplitude of the neighboring pixels, the average, and the intra-signal prediction information are used as the applied signal combination. By calculating
An image encoding apparatus characterized in that inter-signal prediction information is used without encoding the combination. An image decoding apparatus that decodes a code string obtained by performing orthogonal transform, quantization, and encoding on the inter-signal prediction residual signal by the image encoding apparatus according to claim 1,
When decoding using the inter-signal prediction signal and the intra-signal prediction signal,
An image obtained by obtaining an image signal to be decoded using the above-described inter-coding prediction information that is not encoded with respect to a combination of selection of the reference signal and applicability of inter-signal prediction to the predicted signal. Decoding device. In an image decoding device that decodes a signal obtained by orthogonal transform / quantization / encoding an input image signal composed of a plurality of unit blocks having a plurality of pixels,
Based on the reconstructed image signal corresponding to the encoded region obtained by decoding the orthogonally transformed and quantized image signal, the signal in the signal corresponding to the encoding target region of the input image signal. Intra-signal prediction means for calculating intra-signal prediction information for prediction;
Intra-signal compensation means for obtaining an intra-signal prediction signal from the intra-signal prediction information from the intra-signal prediction means and the reconstructed image signal;
The intra-signal prediction residual signal obtained by the difference process between the input image signal that is the encoding target region and the intra-signal prediction signal is separated into the reference signal and the predicted signal and corresponds to each pixel of the reference signal. Inter-signal prediction means for calculating inter-signal prediction information for inter-signal prediction for each pixel of the predicted signal;
A signal for obtaining an inter-signal prediction signal in a coding target region from an intra-decoded prediction residual signal obtained by decoding an orthogonally transformed and quantized image signal and inter-signal prediction information from the inter-signal prediction means Compensation means,
By performing orthogonal transform / quantization / coding on the inter-signal prediction residual signal obtained by performing the difference process between the intra-signal prediction residual signal and the inter-signal prediction signal, each pixel of the predicted signal is obtained. While encoding
The reference signal and the predicted signal in the inter-signal prediction means are separated signals obtained by separating the intra-signal prediction residual signal into color signals,
The inter-signal prediction means selects a reference signal from signals separated into color signals and determines whether or not to apply inter-signal prediction to each predicted signal. When determining all combinations of color signals including non-application, select the combination of signals to be applied so as to minimize the encoding cost calculated from the weighted sum of the generated code amount and the distortion amount. Encode the combination as inter-signal prediction information,
When performing decoding on a code string obtained by performing orthogonal transform, quantization, and encoding on the inter-signal prediction residual signal,
An image decoding apparatus, wherein an inter-signal prediction signal obtained from the encoded inter-signal prediction information and an image signal decoded using the intra-signal prediction signal are obtained.   The inter-signal prediction means enables prediction for each unit block when predicting a signal between a reference signal and a signal to be predicted, and when predicting, for each pixel signal in the unit block, whether or not to predict The image decoding device according to claim 3, which can be set.   The inter-signal prediction means, when determining a combination of selection of a reference signal from separated signals and whether or not to apply inter-signal prediction to a signal to be predicted, determines the combination of applied signals as the variance or amplitude, average of neighboring pixels. The image decoding apparatus according to claim 3, wherein calculation is performed using intra-signal prediction information.   The inter-signal prediction means is calculated from the weighted sum of the generated code amount and the distortion amount when determining a combination of selection of the reference signal from the separated signals and applicability of the inter-signal prediction to the predicted signal. 4. A dictionary is created in advance by statistical machine learning from a reference signal that minimizes encoding cost, an optimal combination, and an input signal, and the combination for an input signal is estimated based on the dictionary. The image decoding apparatus described in 1.   The image decoding apparatus according to claim 3, wherein the inter-signal prediction unit selects a signal having a small change as a reference signal.   The image decoding apparatus according to claim 3, wherein the inter-signal prediction unit selects a signal having a large change not to be a predicted signal.   The image decoding device according to claim 3, wherein the inter-signal prediction means performs the inter-signal prediction by estimating a prediction coefficient for each region when a reference signal is selected and the inter-signal prediction is applied to the predicted signal. .   The image decoding apparatus according to claim 9, wherein the prediction coefficient in the inter-signal prediction unit is configured by a multiplier and a correction value.   The image decoding apparatus according to claim 9, wherein the prediction coefficient in the inter-signal prediction means estimates so as to minimize an error between a prediction signal based on a reference signal and a predicted signal. In an image decoding method for decoding an input image signal composed of a plurality of unit blocks having a plurality of pixels and performing orthogonal transformation, quantization, and encoding,
Based on the reconstructed image signal corresponding to the encoded region obtained by decoding the orthogonally transformed and quantized image signal, the signal in the signal corresponding to the encoding target region of the input image signal. A procedure for calculating in-signal prediction information for prediction;
Obtaining an intra-signal prediction signal from the intra-signal prediction information from the intra-signal prediction means and the reconstructed image signal;
The intra-signal prediction residual signal obtained by the difference process between the input image signal that is the encoding target region and the intra-signal prediction signal is separated into the reference signal and the predicted signal and corresponds to each pixel of the reference signal. A procedure for calculating inter-signal prediction information for inter-signal prediction for each pixel of the predicted signal;
A procedure for obtaining an inter-signal prediction signal in an encoding target region from an intra-decoded prediction residual signal obtained by decoding an orthogonally transformed and quantized image signal and inter-signal prediction information from the inter-signal prediction means When,
By performing orthogonal transform / quantization / coding on the inter-signal prediction residual signal obtained by performing the difference process between the intra-signal prediction residual signal and the inter-signal prediction signal, each pixel of the predicted signal is obtained. And a procedure for performing encoding,
The reference signal and the signal to be predicted in the inter-signal prediction information are separated signals obtained by separating the intra-signal prediction residual signal into color signals,
When determining the combination of the selection of the reference signal from the separated signal and the applicability of the inter-signal prediction to the predicted signal, the variance or amplitude of the neighboring pixels, the average, and the intra-signal prediction information are used as the applied signal combination. By calculating
While making the combination inter-signal prediction information without encoding,
When decoding using the inter-signal prediction signal and the intra-signal prediction signal,
An image obtained by obtaining an image signal to be decoded using the above-described inter-coding prediction information that is not encoded with respect to a combination of selection of the reference signal and applicability of inter-signal prediction to the predicted signal. Decryption method. In an image decoding method for decoding an input image signal composed of a plurality of unit blocks having a plurality of pixels and performing orthogonal transformation, quantization, and encoding,
Based on the reconstructed image signal corresponding to the encoded region obtained by decoding the orthogonally transformed and quantized image signal, the signal in the signal corresponding to the encoding target region of the input image signal. A procedure for calculating in-signal prediction information for prediction;
Obtaining an intra-signal prediction signal from the intra-signal prediction information from the intra-signal prediction means and the reconstructed image signal;
The intra-signal prediction residual signal obtained by the difference process between the input image signal that is the encoding target region and the intra-signal prediction signal is separated into the reference signal and the predicted signal and corresponds to each pixel of the reference signal. A procedure for calculating inter-signal prediction information for inter-signal prediction for each pixel of the predicted signal;
A procedure for obtaining an inter-signal prediction signal in an encoding target region from an intra-decoded prediction residual signal obtained by decoding an orthogonally transformed and quantized image signal and inter-signal prediction information from the inter-signal prediction means When,
By performing orthogonal transform / quantization / coding on the inter-signal prediction residual signal obtained by performing the difference process between the intra-signal prediction residual signal and the inter-signal prediction signal, each pixel of the predicted signal is obtained. And a procedure for performing encoding,
The reference signal and the signal to be predicted in the inter-signal prediction information are separated signals obtained by separating the intra-signal prediction residual signal into color signals,
Color signal including selection of reference signal and non-application of inter-signal prediction to each signal to be predicted regarding whether or not to apply reference signal to each predicted signal from the signal separated into color signals When determining all the combinations, the signal combination to be applied is selected so as to minimize the coding cost calculated from the weighted sum of the generated code amount and the distortion amount, and the combination is used as inter-signal prediction information. Perform encoding,
When performing decoding on a code string obtained by performing orthogonal transform, quantization, and encoding on the inter-signal prediction residual signal,
An image decoding method comprising: obtaining an inter-signal prediction signal obtained from the encoded inter-signal prediction information; and an image signal decoded using the intra-signal prediction signal.   An image decoding program that causes a computer to execute each procedure according to claim 12 or 13.

Images