KR102025413B1 - Video coding/decoding method and apparatus for multi-layers - Google Patents

Abstract

According to an embodiment of the present invention, a method of decoding an image supporting a plurality of layers may include generating an interlayer reference picture set for at least one reference layer to which a current picture may refer, and including an initial layer including the interlayer reference picture set. Generating a reference picture list and performing prediction on the current picture based on the initial reference picture list. Accordingly, a method for generating a reference picture list including pictures of another layer with respect to a layer to be currently encoded and decoded, and an apparatus using the same are provided.

Description

A video encoding and decoding method supporting a plurality of layers and a device using the same {VIDEO CODING / DECODING METHOD AND APPARATUS FOR MULTI-LAYERS}

TECHNICAL FIELD The present invention relates to video compression technology, and more particularly, to a method and apparatus for performing video coding that supports multiple layers.

Recently, the demand for high resolution and high quality images such as high definition (HD) and ultra high definition (UHD) images is increasing in various applications. As the video data becomes higher resolution and higher quality, the amount of data increases relative to the existing video data. Therefore, when the video data is transmitted or stored using a medium such as a conventional wired / wireless broadband line, The storage cost will increase. High efficiency image compression techniques can be used to solve these problems caused by high resolution and high quality image data.

An inter-screen prediction technique for predicting pixel values included in the current picture from a picture before or after the current picture using an image compression technique, an intra prediction technique for predicting pixel values included in a current picture using pixel information in the current picture, Various techniques exist, such as an entropy encoding technique for allocating a short code to a high frequency of appearance and a long code to a low frequency of appearance, and the image data can be effectively compressed and transmitted or stored.

One embodiment of the present invention is to provide a method for generating a reference picture list including pictures of another layer with respect to a layer to be currently encoded and decoded, and an apparatus using the same.

Another object of the present invention is to provide a method for adaptively including a picture of another layer in a reference picture list according to characteristics of a picture set and an apparatus using the same.

Meanwhile, another embodiment of the present invention is to provide a method for dividing a reference picture of another layer into a reference picture list and an apparatus using the same.

Another embodiment of the present invention is to provide a method for adaptively including a reference picture of another layer in a reference picture list according to the order of a viewpoint and an apparatus using the same.

Another object of the present invention is to provide a method for adaptively including a picture of another layer in a reference picture list according to an index value of a temporal sublayer of a current picture, and an apparatus using the same.

Another embodiment of the present invention is to provide a method for adaptively including a picture of another layer in a reference picture list according to a difference in index values of temporal sublayers between layers, and an apparatus using the same.

According to an embodiment of the present invention, a method of decoding an image supporting a plurality of layers may include generating an interlayer reference picture set for at least one reference layer that a current picture may refer to, and setting the interlayer reference picture set. Generating an initial reference picture list including a; and performing prediction on the current picture based on the initial reference picture list.

The generating of the inter-layer reference picture set may include generating at least one of a spatial reference layer picture set, a quality reference layer picture set, a multi-view reference layer picture set, and a depth map reference layer picture set that the current picture may refer to. can do.

The reference picture constituting the interlayer reference picture set may be a picture upsampled so that the picture of the reference layer corresponds to the current picture.

 A reference picture constituting the interlayer reference picture set may be represented as a long term reference picture.

The initial reference picture list includes an initial reference picture list 0, and the initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures having a smaller POC than a picture of count (POC) of the current picture, the interlayer reference The picture set may be configured in the order of a short-term reference picture set consisting of a short-term reference picture having a larger POC than the POC of the current picture, and a long-term reference picture set consisting of a long-term reference picture.

On the other hand, the initial reference picture list includes an initial reference picture list 1, the initial reference picture list 1 is a short-term reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture, the inter-layer reference picture set, A short-term reference picture set composed of short-term reference pictures having a smaller POC than the POC of the current picture, and a long-term reference picture set composed of long-term reference pictures may be configured.

The inter-layer reference picture set includes a first sub reference layer picture set and a second sub reference layer picture set, wherein the first sub reference layer picture set and the second sub reference layer picture set are separated to form the initial reference picture. Can be sorted in a list.

The inter-layer reference picture set includes a multi-view reference layer picture, the initial reference picture list includes an initial reference picture list 0, and the initial reference picture list 0 has a shorter reference picture with a smaller POC than the POC of the current picture. The first sub-reference layer picture set consisting of a short-term reference picture set consisting of a picture having a view order that is smaller than a start point of the current picture among the multi-view reference layer pictures, and a short-term reference picture having a POC larger than a POC of the current picture. The second set of short-term reference pictures, the long-term reference picture composed of long-term reference pictures, and the second sub-reference layer picture set composed of pictures having a viewpoint order greater than that of the current picture among the multi-view reference layer pictures. have.

The interlayer reference picture set includes a multi-view reference layer picture, the initial reference picture list includes an initial reference picture list 1, and the initial reference picture list 1 is a short-term reference picture having a larger POC than the POC of the current picture. The short-term reference picture set consisting of the second sub-reference layer picture set consisting of a picture having a view order larger than that of the current picture among the multi-view reference layer pictures, and a short-term reference picture having a POC smaller than the POC of the current picture. The first set of short-term reference pictures, the long-term reference picture composed of long-term reference pictures, and the first sub-reference layer picture set composed of pictures having a view order that is smaller than a view of the current picture among the multi-view reference layer pictures. have.

The generating of the interlayer reference picture set may be generated based on an index (tempralId) of a temporal sublayer of the current picture.

As the index (tempralId) of the temporal sublayer of the current picture is smaller, the interlayer reference picture set may be arranged in a lower order of the initial reference picture set.

As the difference between the index tempralId of the temporal sublayer of the current picture and the value of the temporal sublayer index of the reference layer is smaller, the interlayer reference picture set may be arranged in a lower order of the initial reference picture set.

The generating of the interlayer reference picture set may be generated based on the maximum number of time base sublayers that allow interlayer prediction.

When the index value of the temporal sublayer of the current picture is larger than the number of the maximum temporal sublayers, the interlayer reference picture set may not be added to the initial reference picture list.

When the temporal sublayer index value of the reference layer is larger than the number of the maximum temporal sublayers, the interlayer reference picture set may not be added to the initial reference picture list.

The method may further include generating a final reference picture list by modifying the initial reference picture list.

The performing of the prediction on the current picture may use a reference picture included in the interlayer reference picture set as a reference picture of the current picture.

According to another aspect of the present invention, there is provided an apparatus for decoding a video, the apparatus comprising: an entropy decoder configured to decode information for prediction and decoding of an image received through a bitstream; Generates an initial reference picture list including an inter-layer reference picture set and at least one inter-layer reference picture set for at least one reference layer that can be referenced by the current picture to be decoded, and based on the initial reference picture list. It may include a prediction unit that performs prediction for the current picture.

According to an embodiment of the present invention, a method for generating a reference picture list including pictures of another layer with respect to a layer to be currently encoded and decoded, and an apparatus using the same are provided.

Further, according to an embodiment of the present invention, a method for adaptively including a picture of another layer in a reference picture list according to characteristics of a picture set and an apparatus using the same are provided.

Meanwhile, according to another embodiment of the present invention, a method of dividing a reference picture of another layer into a reference picture list and an apparatus using the same are provided.

According to another embodiment of the present invention, a method for adaptively including a reference picture of another layer in a reference picture list and an apparatus using the same according to the order of viewpoints are provided.

In addition, according to another embodiment of the present invention, a method for adaptively including a picture of another layer in a reference picture list according to an index value of a temporal sublayer of a current picture and an apparatus using the same are provided.

According to another embodiment of the present invention, a method for adaptively including a picture of another layer in a reference picture list according to a difference in index values of temporal sublayers between layers and an apparatus using the same are provided.

1 is a block diagram schematically illustrating an encoding apparatus according to an embodiment of the present invention.
2 is a block diagram schematically illustrating a decoding apparatus according to an embodiment of the present invention.
3 is a conceptual diagram illustrating an example of a reference picture set including a short-term reference picture.
4 is a conceptual diagram illustrating a method of deriving a POC of a long term reference picture.
5 is a diagram illustrating an example of configuring a reference picture list.
6 is a conceptual diagram illustrating an example of configuring an inter-layer reference picture list according to an embodiment of the present invention.
7 is a diagram illustrating a configuration of a reference picture list O according to an embodiment of the present invention.
8 is a diagram illustrating a configuration of a reference picture list O according to another example of the present invention.
9 is a diagram illustrating a configuration of reference picture list 1 according to an embodiment of the present invention.
FIG. 10 is a diagram illustrating a configuration of reference picture list 1 according to another example of the present invention.
11 illustrates a reference picture list constructed by using a multi-view reference picture according to an embodiment of the present invention.
12 illustrates a reference picture list constructed using a multi-view reference picture according to another example of the present invention.
13 illustrates a reference picture list constructed by using an index of a temporal sublayer according to an embodiment of the present invention.
14 is a control flowchart illustrating a method of encoding an image according to the present invention.
15 is a control flowchart for explaining a method of decoding an image according to the present invention.

Each of the components disclosed in the embodiments and the drawings of the present invention are disclosed in an independent configuration to represent different characteristic functions of the image encoding apparatus. This does not mean that each component is necessarily made up of separate hardware or one software component. In other words, each component is included in each component for convenience of description, and at least two of the components may be combined into one component, or one component may be divided into a plurality of components to perform a function. The integrated and separated embodiments of the components are also included in the scope of the present invention, without departing from the spirit of the invention.

In addition, some of the components disclosed in the present invention may not be essential components for performing essential functions in the present invention but may be optional components for improving performance. The present invention can be implemented including only the components essential for implementing the essentials of the present invention except for the components used for improving performance, and the structure including only the essential components except for the optional components used for improving performance. Also included within the scope of the present invention.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the invention to the specific embodiments. The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the spirit of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. Hereinafter, the same reference numerals are used for the same components in the drawings, and redundant description of the same components is omitted.

Encoding and decoding of a video supporting a plurality of layers in a bitstream is called scalable coding. Since there is a strong correlation between the plurality of layers, the prediction may be performed by using this correlation to remove redundant elements of data and to improve the encoding performance of the image. Performing prediction of the current layer, which is a target of prediction using information of another layer, is referred to as inter-layer prediction in the following.

At least one of a resolution, a frame rate, and a color format may be different from each other, and upsampling or downsampling of a layer may be performed to adjust the resolution when inter-layer prediction is performed.

1 is a block diagram schematically illustrating an encoding apparatus according to an embodiment of the present invention.

The encoding apparatus 100 according to the present invention includes an encoder 100a for an upper layer and an encoder 100b for a lower layer.

The upper rare layer may be expressed as a current layer or an enhancement layer, and the lower layer may be expressed as a reference layer or a base layer. The upper layer and the lower layer may have at least one of a resolution, a frame rate, and a color format. When a resolution change is necessary to perform inter-layer prediction, upsampling or downsampling of a layer may be performed.

The encoder 100a of the upper layer includes a splitter 110, a predictor 100, an intra prediction unit 121, an inter prediction unit 122, an inter layer prediction unit 123, and a transformer 130. , Quantization unit 140, reordering unit 150, entropy encoding unit 160, inverse quantization unit 170, inverse transform unit 180, filter unit 190 and memory 195, and MUX 197. can do.

The encoder 100b of the lower layer includes a splitter 111, a predictor 125, an intra prediction unit 126, an inter prediction unit 127, a transform unit 131, a quantizer 141, and a rearrangement. The unit 151 may include an entropy encoding unit 161, an inverse quantization unit 171, an inverse transform unit 181, a filter unit 191, and a memory 196.

The encoder may be implemented by the image encoding method described in the following embodiments of the present invention, but operations in some components may not be performed to reduce the complexity of the encoding apparatus or for fast real time encoding. For example, in performing the intra prediction in the prediction unit, some limited number of methods are used without selecting the optimal intra intra coding method using all intra prediction modes in order to perform encoding in real time. A method of selecting one intra prediction mode among them as a final intra prediction mode using the intra prediction mode of the image may be used. As another example, it is possible to limit the use of the type of prediction block used in performing intra prediction or inter prediction.

The unit of a block processed by the encoding apparatus may be a coding unit that performs encoding, a prediction unit that performs prediction, or a transformation unit that performs transformation. A coding unit may be represented by a term such as a coding unit (CU), a prediction unit is a prediction unit (PU), and a transformation unit is a transform unit (TU).

The splitters 110 and 111 divide a layer image into a combination of a plurality of coding blocks, prediction blocks, and transform blocks, and one of the coding blocks, prediction blocks, and transform blocks according to a predetermined criterion (for example, a cost function). You can split the layer by selecting the combination of. For example, to split a coding unit in a layer image, a recursive tree structure such as a quad tree structure may be used. Hereinafter, in the embodiment of the present invention, the meaning of the coding block may be used not only as a block for encoding but also as a block for decoding.

The prediction block may be a unit for performing prediction such as intra prediction or inter prediction. The block for performing intra prediction may be a block having a square shape such as 2N × 2N or N × N. Predictive block partitioning is performed by using 2NxN, Nx2N, or asymmetric Asymmetric Motion Partitioning (AMP), which splits a square or square type prediction block into the same form. There is a way. Depending on the shape of the prediction block, the transform unit 115 may change a method of performing the transform.

The prediction units 120 and 125 of the encoders 100a and 100b may include the intra prediction units 121 and 126 performing intra prediction and the inter prediction unit performing inter prediction. (1122, 126). The predictor 120 of the higher layer encoder 100a further includes an inter-layer predictor 123 that performs prediction on the upper layer by using information of the lower layer.

The prediction units 120 and 125 may determine whether to use inter prediction or intra prediction on the prediction block. The processing unit in which the prediction is performed and the processing block in which the prediction method is determined may be different. For example, in performing intra prediction, a prediction mode is determined based on a prediction block, and a process of performing prediction may be performed based on a transform block. The residual value (residual block) between the generated prediction block and the original block may be input to the transformers 130 and 131. In addition, prediction mode information and motion vector information used for prediction may be encoded by the entropy encoder 130 together with the residual value and transmitted to the decoding apparatus.

When using the Pulse Coded Modulation (PCM) coding mode, the original block may be encoded as it is and transmitted to the decoder without performing prediction through the prediction units 120 and 125.

The intra prediction units 121 and 126 may generate an intra prediction block based on reference pixels present around the current block (the block to be predicted). In the intra prediction method, the intra prediction mode may include a directional prediction mode using reference pixel information according to a prediction direction and a non-directional mode using no directional information when performing prediction. The mode for predicting luma information and the mode for predicting color difference information may be different. In order to predict the color difference information, intra prediction mode information or predicted luma signal information may be used. If the reference pixel is not available, the prediction block may be generated by replacing the unavailable reference pixel with another pixel.

The prediction block may include a plurality of transform blocks. If the prediction block has the same size as the transform block when the intra prediction is performed, pixels present on the left side of the prediction block, pixels present on the upper left side, and top Intra-prediction of the prediction block may be performed based on the pixels present in the. However, when the prediction block is different from the size of the transform block when the intra prediction is included, and a plurality of transform blocks are included in the prediction block, the intra prediction is performed using a reference pixel determined based on the transform block. can do.

The intra prediction method may generate a prediction block after applying a mode dependent intra smoothing (MDIS) filter to a reference pixel according to the intra prediction mode. The type of MDIS filter applied to the reference pixel may be different. The MDIS filter is an additional filter applied to the predicted block in the picture by performing the intra prediction, and may be used to reduce the step present in the predicted block in the picture generated after performing the prediction with the reference pixel. In performing MDIS filtering, filtering on a reference pixel and some columns included in the predicted block in the screen may perform different filtering according to the direction of the intra prediction mode.

The inter prediction units 122 and 127 may perform prediction by referring to information of a block included in at least one of a previous picture or a subsequent picture of the current picture. The inter prediction units 122 and 127 may include a reference picture interpolator, a motion predictor, and a motion compensator.

The reference picture interpolation unit may receive reference picture information from the memories 195 and 196 and generate pixel information of an integer pixel or less in the reference picture. In the case of a luma pixel, a DCT-based 8-tap interpolation filter having different filter coefficients may be used to generate pixel information of integer pixels or less in units of 1/4 pixels. In the case of a chrominance signal, a DCT-based interpolation filter having different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/8 pixels.

The inter prediction units 122 and 127 may perform motion prediction based on the reference picture interpolated by the reference picture interpolator. As a method for calculating a motion vector, various methods such as a full search-based block matching algorithm (FBMA), a three step search (TSS), and a new three-step search algorithm (NTS) may be used. The motion vector may have a motion vector value of 1/2 or 1/4 pixel units based on the interpolated pixels. The inter prediction units 122 and 127 may perform prediction on the current block by applying one inter prediction method among various inter prediction methods.

For example, various methods, such as a skip method, a merge method, and an MVP (Motion Vector Prediction) method, may be used as the inter prediction method.

In inter prediction, motion information, that is, information such as an index of a reference picture, a motion vector, and a residual signal, is entropy coded and transmitted to a decoder. When the skip mode is applied, residuals may not be generated, transformed, quantized, or transmitted.

The interlayer prediction unit 123 performs interlayer prediction for predicting an upper layer by using information of a lower layer. The inter-layer prediction unit 123 uses inter-layer texture prediction and inter-layer inter prediction by using textures of lower layers, intra prediction mode information, motion information, and syntax information. ), Inter-layer syntax prediction, and the like.

Inter-layer texture prediction means using the texture of the reference block in the lower layer as a prediction sample of the current block of the upper layer. In this case, the texture of the reference block may be scaled by upsampling.

Inter-layer texture prediction includes the intra BL and upsampled lower layers, which upsample the reconstructed values of the reference blocks in the lower layers and encode the residuals with the current blocks using the upsampled reference blocks as predictions for the current block. There may be a reference index method that stores the in memory and uses the stored lower layer as a reference index.

Intra-prediction of the upper layer may be performed using intra-prediction mode information of the lower layer. In this case, the intra-prediction mode of the lower layer may be referred to as a BL intra mode.

Inter-layer motion prediction is also called inter-layer inter prediction, and according to inter-layer motion prediction, prediction of a current block of an upper layer may be performed using motion information of a lower layer. The motion information may include a motion vector and a reference picture index.

The inter-layer prediction unit 123 may also perform inter-layer syntax prediction for predicting or generating a texture of the current block by using syntax information of the lower layer. In this case, the syntax information of the lower layer used for the prediction of the current block may be information about an intra prediction mode, motion information, and the like.

As another example of inter-layer prediction, according to the difference prediction between layers, the prediction of the current block is performed by using the difference image generated as a difference value between the reconstructed image of the upper layer and the upsampled reconstructed image of the lower layer. Can be.

As an example of the inter-layer prediction, the inter-layer texture prediction, the inter-layer motion prediction, the inter-layer syntax prediction, and the inter-layer differential prediction have been described, but the inter-layer prediction applicable to the present invention is not limited thereto.

A residual block including residual information, which is a difference between the predicted block generated by the predictors 120 and 125 and the reconstructed block of the predicted block, is generated, and the residual block is input to the transformers 130 and 131.

The transform units 130 and 131 may transform the residual block using a transform method such as a discrete cosine transform (DCT) or a discrete sine transform (DST). Whether DCT or DST is applied to transform the residual block may be determined based on intra prediction mode information of the prediction block used to generate the residual block and size information of the prediction block. That is, the transformers 130 and 131 may apply the transformation method differently according to the size of the prediction block and the prediction method.

The quantizers 140 and 141 may quantize the values transformed by the transformers 130 and 131 into the frequency domain. The quantization coefficient may change depending on the block or the importance of the image. The values calculated by the quantizers 140 and 141 may be provided to the dequantizers 170 and 17 and the reordering units 150 and 151.

The reordering units 150 and 151 may reorder coefficient values with respect to the quantized residual value. The reordering units 150 and 151 may change the two-dimensional block shape coefficients into a one-dimensional vector form through a coefficient scanning method. For example, the realignment units 150 and 151 may scan DC coefficients to coefficients in the high frequency region by using a Zig-Zag scan method and change them into one-dimensional vectors. Depending on the size of the transform block and the intra prediction mode, a vertical scan method for scanning two-dimensional block shape coefficients in a column direction, not a zig-zag scan method, and a horizontal scan method for scanning two-dimensional block shape coefficients in a row direction Can be used. That is, according to the size of the transform block and the intra prediction mode, it is possible to determine which scan method among zigzag-scan, vertical scan and horizontal scan is used.

The entropy encoders 160 and 161 may perform entropy encoding based on the values calculated by the reordering units 150 and 151. Entropy encoding may use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

The entropy encoders 160 and 161 transmit residual value coefficient information, block type information, prediction mode information, partition unit information, prediction block information, and the like of the coding block from the reordering units 150 and 151 and the prediction units 120 and 125. Entropy encoding may be performed based on a predetermined encoding method by receiving various information such as unit information, motion vector information, reference frame information, interpolation information of a block, and filtering information. In addition, the entropy encoder 160 or 161 may entropy-encode coefficient values of coding units input from the reordering unit 150 or 151.

The entropy encoders 160 and 161 may encode the intra prediction mode information of the current block by performing binarization on the intra prediction mode information. The entropy encoder 160 or 161 may include a codeword mapping unit for performing such a binarization operation, and may perform different binarization according to the size of a prediction block for performing intra prediction. In the codeword mapping unit, the codeword mapping table may be adaptively generated or stored in advance through a binarization operation. As another embodiment, the entropy encoders 160 and 161 may express prediction mode information in the current screen using a codenum mapping unit for performing codenum mapping and a codeword mapping unit for performing codeword mapping. In the codenum mapping unit and the codeword mapping unit, a codenum mapping table and a codeword mapping table may be generated or stored.

The inverse quantizers 170 and 171 and the inverse transformers 180 and 181 inverse quantize the quantized values in the quantizers 140 and 141 and inversely transform the converted values in the transformers 130 and 131. The residual values generated by the inverse quantizers 170 and 171 and the inverse transformers 180 and 181 may be predicted by the motion estimator, the motion compensator, and the intra prediction unit included in the predictors 120 and 125. It may be combined with the prediction block to generate a reconstructed block.

The filters 190 and 191 may include at least one of a deblocking filter, an offset corrector, and an adaptive loop filter (ALF).

The deblocking filter may remove block distortion caused by boundaries between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply a deblocking filter to the current block based on the pixels included in several columns or rows included in the block. When the deblocking filter is applied to the block, a strong filter or a weak filter may be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be performed in parallel when vertical filtering and horizontal filtering are performed.

The offset correction unit may correct the offset with respect to the original image on a pixel-by-pixel basis for the deblocking image. In order to perform offset correction for a specific picture, the pixels included in the image are divided into a predetermined number of areas, and then, an area to be offset is determined, an offset is applied to the corresponding area, or offset considering the edge information of each pixel. You can use this method.

The adaptive loop filter (ALF) may perform filtering based on a value obtained by comparing the filtered reconstructed image with the original image. After dividing the pixels included in the image into at least one group, one filter to be applied to the group may be determined and filtering may be performed for each group.

The filter units 190 and 191 may apply only the deblocking filter, only the deblocking filter and the ALF, or may apply only the deblocking filter and the offset correction unit without applying all of the deblocking filter, the ALF, and the offset correction unit.

The memories 195 and 196 may store reconstructed blocks or pictures calculated by the filters 190 and 191, and the stored reconstructed blocks or pictures may be provided to the predictors 120 and 125 when performing inter prediction. have.

The information output from the entropy encoder 100b of the lower layer and the information output from the entropy encoder 100a of the upper layer may be multiplexed by the MUX 197 and output as a bitstream.

The MUX 197 may be included in the encoder 100b of the lower layer or may be implemented as an independent device or module separate from the encoder 100.

2 is a block diagram schematically illustrating a decoding apparatus according to an embodiment of the present invention.

As shown in FIG. 2, the decoding apparatus 200 includes a decoder 200a of an upper layer and a decoder 200b of a lower layer.

The decoder 200a of the upper layer includes an entropy decoder 210, a reordering unit 220, an inverse quantization unit 230, an inverse transform unit 245, a prediction unit 250, a filter unit 260, and a memory 240. ) May be included.

The lower layer decoding unit 200b includes an entropy decoding unit 211, a reordering unit 221, an inverse quantization unit 231, an inverse transform unit 241, a prediction unit 251, a filter unit 261, and a memory 271. ) May be included.

When a bitstream including a plurality of layers is transmitted from the encoding apparatus, the DEMUX 280 may demultiplex information for each layer and transmit the information to the decoders 200a and 200b for each layer. The input bitstream may be decoded in a procedure opposite to that of the encoding apparatus.

The entropy decoders 210 and 211 may perform entropy decoding in a procedure opposite to that of the entropy encoder in the encoding apparatus. Information for generating a prediction block among the information decoded by the entropy decoders 210 and 211 is provided to the predictors 250 and 251, and the residual value of the entropy decoding performed by the entropy decoder is the reordering unit 220 or 221. Can be entered.

Like the entropy encoders 160 and 161, the entropy decoders 210 and 211 may perform inverse transform using at least one of CABAC and CAVLC.

The entropy decoders 210 and 211 may decode information related to intra prediction and inter prediction performed by the encoding apparatus. The entropy decoding unit may include a codeword mapping unit and include a codeword mapping table for generating a received codeword as an intra prediction mode number. The codeword mapping table may be stored in advance or generated adaptively. When using the codenum mapping table, a codenum mapping unit for performing codenum mapping may be additionally provided.

The reordering units 220 and 221 may reorder the bitstreams entropy decoded by the entropy decoding units 210 and 211 based on a method of rearranging the bitstreams by the encoder. Coefficients expressed in the form of a one-dimensional vector can be rearranged by restoring the coefficients in a two-dimensional block form. The reordering unit may be realigned by receiving information related to coefficient scanning performed by the encoder and performing reverse scanning based on the scanning order performed by the encoder.

The inverse quantization units 230 and 231 may perform inverse quantization based on quantization parameters provided by the encoding apparatus and coefficient values of the rearranged block.

The inverse transformers 240 and 241 may perform inverse DCT and inverse DST on the DCT and DST performed by the transformers 130 and 131 with respect to the quantization result performed by the encoding apparatus. The inverse transform may be performed based on a transmission unit determined by the encoding apparatus. The DCT and DST may be selectively performed by the transform unit of the encoding apparatus according to a plurality of pieces of information, such as a prediction method, a size and a prediction direction of the current block, and the inverse transform unit 225 of the decoding apparatus may be performed by the transform unit of the encoding apparatus. Inverse transformation may be performed based on the transformation information. When the transform is performed, the transform may be performed based on the coding block rather than the transform block.

The prediction units 250 and 251 may generate the prediction blocks based on the prediction block generation related information provided by the entropy decoding units 210 and 211 and previously decoded blocks or picture information provided by the memories 270 and 271. .

The predictors 250 and 251 may include a prediction unit determiner, an inter prediction unit, and an intra prediction unit.

The prediction unit discriminator receives various information such as prediction unit information input from the entropy decoder, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and distinguishes the prediction block from the current coding block. It is possible to determine whether to perform this inter prediction or intra prediction.

The inter prediction unit uses information required for inter prediction of the current prediction block provided by the encoding apparatus to the current prediction block based on information included in at least one of a previous picture or a subsequent picture of the current picture including the current prediction block. Inter prediction can be performed. Whether the motion prediction method of the prediction block included in the coding block is skip mode, merge mode, or AMVP mode to perform inter prediction. Can be determined.

The intra prediction unit may generate a prediction block based on pixel information in the current picture. When the prediction block is a prediction block that performs intra prediction, intra prediction may be performed based on intra prediction mode information of the prediction block provided by the encoding apparatus. The intra prediction unit is an MDIS filter that performs filtering on the reference pixels of the current block, a reference pixel interpolator which generates reference pixels in pixel units of an integer value or less by interpolating the reference pixels, and filters when the prediction mode of the current block is DC mode. It may include a DC filter for generating a prediction block through.

The predictor 250 of the upper layer decoder 200a may further include an inter-layer predictor that performs inter-layer prediction for predicting an upper layer by using information of the lower layer.

The inter-layer prediction unit uses inter-layer texture prediction, inter-layer inter prediction, and inter-layer prediction by using textures of lower layers, intra prediction mode information, motion information, and syntax information. Inter-layer syntax prediction may be performed.

According to the inter-layer texture prediction, prediction may be performed using the texture of the reference block in the lower layer as a prediction value of the current block of the upper layer. The texture of the reference block can be scaled by upsampling.

Inter-layer texture prediction includes the intra BL and upsampled base layers that upsample the reconstructed values of the reference blocks in the lower layers and encode residuals with the current blocks using the upsampled reference blocks as predictions for the current block. Is stored in memory and uses the stored base layer as a reference index.

Intra-prediction of the upper layer may be performed using intra-prediction mode information of the lower layer, and in this case, the intra-prediction mode of the lower layer may be expressed as a BL intra mode.

According to inter-layer motion prediction, prediction of a current block of an upper layer may be performed using motion information of a lower layer.

The inter-layer prediction unit may also perform inter-layer syntax prediction for predicting or generating a texture of the current block using syntax information of the lower layer. In this case, the syntax information of the lower layer used for the prediction of the current block may be information about an intra prediction mode, motion information, and the like.

In addition, the inter-layer prediction unit may perform differential prediction between layers predicting the current block by using the difference image generated as a difference value between the reconstructed image of the upper layer and the resampled image of the lower layer.

As an example of the inter-layer prediction, the inter-layer texture prediction, the inter-layer motion prediction, the inter-layer syntax prediction, and the inter-layer differential prediction have been described, but the inter-layer prediction applicable to the present invention is not limited thereto.

The reconstructed block or picture may be provided to the filter units 260 and 261. The filter units 260 and 261 may include a deblocking filter, an offset corrector, and an ALF.

Information about whether a deblocking filter is applied to the corresponding block or picture and when the deblocking filter is applied to the block or picture may be provided from the encoding apparatus as to whether a strong filter or a weak filter is applied. The deblocking filter of the decoding apparatus may receive the deblocking filter related information provided by the encoding apparatus and perform the deblocking filtering on the corresponding block in the decoding apparatus.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction and offset value information applied to the image during encoding.

The adaptive loop filter (ALF) may perform filtering based on a value obtained by comparing the restored image with the original image after performing the filtering. The ALF may be applied to the coding unit based on the ALF application information, the ALF coefficient information, and the like provided from the encoding apparatus. Such ALF information may be provided included in a specific parameter set.

The memories 270 and 271 may store the reconstructed picture or block to be used as the reference picture or the reference block, and output the reconstructed picture.

The encoding apparatus and the decoding apparatus may encode three or more layers instead of two layers, and in this case, the encoder for the upper layer and the decoder for the upper layer are provided in plural numbers corresponding to the number of upper layers. Can be.

Hereinafter, although an embodiment of the present invention uses a coding unit as a coding block for convenience of description, it may be a block for performing decoding as well as encoding. Hereinafter, a method of generating a reference picture list described with reference to the drawings according to an embodiment of the present invention may be implemented according to the functions of the respective modules described above with reference to FIGS. 1 and 2, and such an encoding device and a decoding device are rights of the present invention. It is included in a range.

As described above, in coding of an image supporting a multi-layer structure, a method of encoding / decoding an enhancement layer using motion information and texture information of the base layer, a coding unit (CU) of the base layer, Alternatively, there is a method of encoding / decoding using an up-sampled base layer as a reference picture without changing a decoding method in units of prediction units (PUs). The latter method may be expressed in a reference picture index (RefIdx) mode.

When the reference picture index (RefIdx) mode is used, a picture already decoded in the enhancement layer may be used as a reference picture, or an image of a base layer or a lower layer may be upsampled to be used as a reference picture.

Hereinafter, when an upsampled base layer or lower layer image is used as a reference picture, the upsampled base layer or lower layer image is referred to as an inter-layer reference picture.

Even in an image that supports a multi-layer structure for multiple views, inter prediction may be performed using a layer for a view other than the current layer as a reference picture, and a layer of another view referred to for encoding and decoding of the layer may also be inter It may be expressed as a layer reference picture.

That is, when constructing a reference picture set of an enhancement layer for coding an image having a multi-layer structure in which scalability is supported instead of a single layer, an interlayer reference picture should also be considered.

In general, inter prediction may use at least one of a previous picture or a next picture of a current picture as a reference picture, and perform prediction on the current block based on the reference picture. An image used for prediction of the current block is called a reference picture or a reference frame.

The reference picture is specified by a reference picture index refIdx, and a predetermined region in the reference picture is specified as a reference block through a motion vector.

The inter prediction may select a reference picture and a reference block corresponding to the current block in the reference picture to generate a prediction block for the current block.

In the inter prediction, the encoding apparatus and the decoding apparatus may derive the motion information of the current block and then perform the inter prediction and / or motion compensation based on the derived motion information. In this case, the encoding apparatus and the decoding apparatus may move a coll block corresponding to the current block in a neighboring block and / or a collocated picture that has already been restored. By using the information, the encoding / decoding efficiency can be improved.

Here, the reconstructed neighboring block is a block in the current picture that is already encoded and / or decoded and reconstructed, and may include a block adjacent to the current block and / or a block located at an outer corner of the current block. In addition, the encoding apparatus and the decoding apparatus may determine a predetermined relative position based on a block existing at a position spatially corresponding to the current block in the call picture, and the predetermined relative position (spatially corresponding to the current block) The call block may be derived based on the location of the inside and / or outside of the block existing at the location. Here, as an example, the call picture may correspond to one picture among the reference pictures included in the reference picture list.

In inter prediction, a prediction block may be generated such that a residual signal with the current block is minimized and a motion vector size is also minimized.

Meanwhile, the motion information derivation scheme may vary depending on the prediction mode of the current block. Prediction modes applied for inter prediction may include Advanced Motion Vector Predictor (AMVP), merge, and the like.

For example, when an advanced motion vector predictor (AMVP) is applied, the encoding apparatus and the decoding apparatus may generate a motion vector candidate list using the reconstructed motion vector of the neighboring block and / or the motion vector of the call block. That is, the motion vector of the reconstructed neighboring block and / or the motion vector of the call block may be used as a motion vector candidate. The encoding apparatus may transmit a predicted motion vector index indicating the optimal motion vector selected from the motion vector candidates included in the list to the decoding apparatus. In this case, the decoding apparatus may select the predicted motion vector of the current block from the motion vector candidates included in the motion vector candidate list using the motion vector index.

The encoding apparatus may obtain a motion vector difference (MVD) between the motion vector of the current block and the predictive motion vector, and may encode the same and transmit the encoded motion vector to the decoding apparatus. In this case, the decoding apparatus may decode the received motion vector difference, and may derive the motion vector of the current block through the sum of the decoded motion vector difference and the predicted motion vector.

The encoding apparatus may also transmit a reference picture index or the like indicating the reference picture to the decoding apparatus.

The decoding apparatus may predict the motion vector of the current block using the motion information of the neighboring block, and may derive the motion vector for the current block using the residual received from the encoding apparatus. The decoding apparatus may generate a prediction block for the current block based on the derived motion vector and the reference picture index information received from the encoding apparatus.

As another example, when merge is applied, the encoding apparatus and the decoding apparatus may generate the merge candidate list using the motion information of the reconstructed neighboring block and / or the motion information of the call block. That is, when there is motion information of the reconstructed neighboring block and / or the call block, the encoding apparatus and the decoding apparatus may use this as a merge candidate for the current block.

The encoding apparatus may select a merge candidate capable of providing an optimal encoding efficiency among the merge candidates included in the merge candidate list as motion information for the current block. In this case, a merge index indicating the selected merge candidate may be included in the bitstream and transmitted to the decoding apparatus. The decoding apparatus may select one of the merge candidates included in the merge candidate list by using the transmitted merge index, and determine the selected merge candidate as motion information of the current block. Therefore, when the merge mode is applied, the motion information of the restored neighboring block and / or the call block may be used as the motion information of the current block. The decoding apparatus may reconstruct the current block by adding the prediction block and the residual transmitted from the encoding apparatus.

In the above-described AMVP and merge mode, the motion information of the reconstructed neighboring block and / or the motion information of the call block may be used to derive the motion information of the current block.

In the case of a skip mode, which is one of other modes used for inter prediction, information of neighboring blocks may be used as is in the current block. Accordingly, in the skip mode, the encoding apparatus does not transmit syntax information such as residual to the decoding apparatus other than the information indicating which block motion information to use as the motion information of the current block.

The encoding apparatus and the decoding apparatus may generate a prediction block of the current block by performing motion compensation on the current block based on the derived motion information. Here, the prediction block may mean a motion compensated block generated as a result of performing motion compensation on the current block. Also, the plurality of motion compensated blocks may constitute one motion compensated image.

The decoding apparatus may check and derive motion information necessary for inter prediction of the current block, for example, a skip flag, a merge flag, and the like, received from the encoding apparatus, corresponding to the motion vector, the reference picture index, and the like.

The processing unit in which the prediction is performed and the processing unit in which the prediction method and the details are determined may be different. For example, a prediction mode may be determined in units of prediction blocks, and prediction may be performed in units of transform blocks, or a prediction mode may be determined in units of prediction blocks, and intra prediction may be performed in units of transform blocks.

Pictures encoded / decoded prior to the current picture may be stored in a memory (eg, a Decoded Picture Buffer (DPB)) and used for prediction of the current block (current picture). The list of pictures available for inter prediction of the current block is maintained as a reference picture list.

P slices are slices that are decoded through intra prediction or inter prediction using at most one motion vector and one reference picture. A B slice is a slice decoded through intra prediction or inter prediction using up to two motion vectors and two reference pictures. In this case, the reference picture includes a short term reference picture and a long term reference picture. The picture may be specified as a picture order count (POC) indicating a display order, and the short-term reference pictures may be pictures that do not have a large POC difference from the current picture, and the long-term reference pictures may be pictures having a large POC difference from the current picture. Can be.

Reference picture list 0 (referred to as 'L0' for convenience of description) is a reference picture list used for inter prediction of a P slice or a B slice. Reference picture list 1 (referred to as 'L1' for convenience of description) is used for inter prediction of a B slice. Therefore, L0 is used for inter prediction on blocks of P slices that perform unidirectional prediction, and L0 and L1 are used for inter prediction on blocks of B slices which perform bidirectional prediction.

The decoding apparatus constructs a reference picture list when decoding P slices and B slices through inter prediction. The reference picture used for inter prediction is specified through a reference picture list. The reference picture index is an index indicating a reference picture on the reference picture list.

The reference picture list may be configured based on a reference picture set transmitted from the encoding apparatus. The reference picture set may include a POC of a picture used as a reference picture and a flag (used_by_curr_pic_s0_flag) indicating whether the corresponding picture is directly referenced. Reference pictures constituting the reference picture list may be stored in a memory (eg, DPB). Pictures stored in the memory (pictures encoded / decoded before the current picture) are managed by the encoding apparatus and the decoding apparatus.

3 is a conceptual diagram illustrating an example of a reference picture set including a short-term reference picture.

As shown, a reference picture set (RPS) is a POC of a picture for a short-term reference picture that should be stored in the DPB at this point in time for each picture, and a flag indicating whether the current picture directly refers to a specific picture. It can consist of information.

For example, when inter prediction of a picture with a POC of 21, all three pictures (pictures with POCs of 20, 19, and 16) can be referenced as short-term reference pictures, and all three pictures have a POC of 1 since the used_by_curr_pic_s0_flag value is 1. Are all used directly for prediction of a picture with 21.

Meanwhile, when inter prediction of a picture having a POC of 26, all three pictures (POCs of 25, 24, and 20) may be referenced as short-term reference pictures, but the used_by_curr_pic_s0_flag value of a picture having a POC of 25 is 24 and 20 of a POC. Unlike used_by_curr_pic_s0_flag value of 0. In this case, a picture having a POC of 25 is not directly used for inter prediction of a picture having a POC of 26.

Pictures not shown in the reference picture set in the current picture may be removed from the DPB after an unused for reference indicating that they are not used as the reference picture.

4 is a conceptual diagram illustrating a method of deriving a POC of a long term reference picture.

As shown in the figure, since the difference between the current picture and the POC is large, the long-term reference picture may be represented using the least significant bit (LSB) and the most significant bit (MSB) of the POC.

MaxPocLsb means the maximum value that can be represented by the LSB. If MaxPocLsb is 32, the long term reference picture (LTRF) having a POC of 84 may be expressed as 32 * 2 + 20, where the LSB is 20 and the MSB is 2.

In the case of the current picture having the POC of 338, if the maximum value that can be represented by the LSB is 32, 32 * 10 + 11 can be represented, so that 10 is the MSB value and 11 is the LSB value.

delta_poc_msb_cycle_lt is a value for determining DeltaPocMsbCycleLt which is the MSB of the POC in the long-term reference picture set of the current picture. DeltaPocMsbCycleLt may correspond to the MSB difference value of the POC of the current picture and the MSB of the POC of the reference picture.

 In the case of a long-term reference picture, the POC of the reference picture can be derived by using the LSB value of the POC of the reference picture and the MSB difference of the POC of the current picture and the MSB of the POC of the reference picture.

For example, a long-term reference picture (LTRP [0]) having a POC of 331 and a POC indexed by 0 of 308 having an LSB value of 20, and an MSB difference value between the MSB of the POC of the current picture and the POC of the reference picture is Using 1 can be expressed as 331-1 * 32-11 + 20.

Similarly, a long-term reference picture (LTRP [1]) with a POC indexed by 1 of 170 is 331-5 * 32- using an LSB value of 10, the difference between the MSB of the POC of the current picture and the MSB of the POC of the reference picture. It can be expressed as 11 + 10.

As shown in FIGS. 3 and 4, an initial reference picture list that can be referenced by the current picture may be configured with a list of pictures existing in the short-term reference picture buffer and the long-term reference picture buffer.

5 is a diagram illustrating an example of configuring a reference picture list.

Reference pictures are the first short-term reference picture set RefPicSetStCurr0 consisting of reference pictures Ref 1 and Ref 2 that are smaller than the POC of the current picture Curr based on the current picture, and reference pictures that are larger than the POC of the current picture. It may be classified into a second short-term reference picture set RefPicSetStCurr1 composed of Ref 3 and Ref 4) and a long-term reference picture set RefPicSetLtCurr composed of long-term reference pictures Ref LT1 and Ref LT2.

At this time, the first short-term reference picture set (RefPicSetStCurr0) is composed of pictures having a used_by_curr_pic_s0_flag value of 1 (delta_poc_s0 with used_by_curr_pic_s0_flag = 1), and the second short-term reference picture set (RefPicSetStCurr1_pic_curl_fla) is also used as a value of__curr_flag. (Delta_poc_s1 with used_by_curr_pic_s1_flag = 1).

The initial reference picture list may be composed of a set of reference picture sets having different properties.

As illustrated in FIG. 5, the reference picture list 0, that is, L0 is configured in the order of the first short-term reference picture set RefPicSetStCurr0, the second short-term reference picture set RefPicSetStCurr1, and the long-term reference picture set RefPicSetLtCurr.

On the other hand, the reference picture list 1, that is, L1 is configured in the order of the second short-term reference picture set RefPicSetStCurr1, the first short-term reference picture set RefPicSetStCurr0, and the long-term reference picture set RefPicSetLtCurr.

The number of reference pictures that may be included in the reference picture list may be determined based on information transmitted from the encoding apparatus. For example, after configuring a reference picture list, the encoding apparatus determines the number of reference pictures to use, and transmits information about the number of reference pictures to use (for example, num_ref_idx_lX_default_active_minus1, X = 0 or 1) to a sequence parameter set (SPS). ) Can be transmitted to the decoding device as a syntax element. The decoding apparatus may use the number of reference pictures specified by the received num_ref_idx_lX_default_active_minus1 plus 1 as a default value in the current sequence.

In addition, when the encoding apparatus intends to specify the number of reference pictures for each picture or slice, the encoding apparatus may include additional information indicating the number of reference pictures (for example, num_ref_idx_l1_active_minus1, X = 0, or 1) for a picture parameter set (PPS). ) Or through a slice header. The decoding apparatus may apply a value specified by adding 1 to the received num_ref_idx_l1_active_minus1 as the number of reference pictures for the current picture or the current slice.

When performing inter prediction, motion compensation may be performed by using the specified reference picture in the reference picture list configured as described above.

In a multi-layer structure providing spatial scalability or multi-view scalability, a reference picture of an enhancement layer may be composed of reference pictures on the same layer and inter-layer reference picture.

In this case, signaling for the inter-layer reference picture may be performed through information for identifying a layer and information for identifying a reference picture. For example, a nuh_layer_id value present in the same access unit as the current picture of the i th layer and transmitted in the NAL unit header (i is greater than j) is a nuh_layer_id value. If it is equal to the RefPiclayerId value for, the picture may be determined to be used as a reference picture for the current picture. The interlayer reference picture may be displayed as a long term reference picture.

RefPicLayerId is a value that can be signaled as a syntax element inter_layer_pred_layer_idc included in the slice header, and refers to a layer that the current layer refers to inter-layer prediction.

6 is a conceptual diagram illustrating an example of configuring an inter-layer reference picture list according to an embodiment of the present invention.

As shown in FIG. 6, the initial reference picture list in the multi-layer structure image is a short-term reference picture set (RefPicSetStCurrBefore [i], hereinafter referred to as a first reference picture set) consisting of short-term reference pictures having a smaller POC than the POC of the current picture, and the current picture. A short-term reference picture set (RefPicSetStCurr After [i], hereinafter referred to as the second reference picture set) consisting of short-term reference pictures with a larger POC than the POC, followed by the long-term reference picture set (RefPicSetLtCurr [i], hereinafter referred to as the third reference picture set), and It may consist of an inter-layer reference picture set (RefPicSetILCurr [i], hereinafter referred to as a fourth reference picture set).

The reference picture included in the fourth reference picture set may be a picture of a layer supporting spatial scalability, and a picture or depth scalability or quality scalability included in a layer supporting multiview scalability. It may be a picture of a supporting layer. In this case, spatial scalability and image quality scalability may be coded with the same codec structure.

If the bitstream supports more than one scalability, the reference pictures for each scalability may be collectively composed of one reference picture set. When one reference picture set is configured for a plurality of scalabilities, information about an order of scalability types aligned with the reference picture set may be transmitted from the encoding apparatus to the decoding apparatus.

Meanwhile, a reference picture set may be separately configured for each of a plurality of scalabilities. That is, a fourth reference picture set consisting of inter-layer reference pictures, a fifth reference picture set, a sixth reference picture set, etc. are generated for the current picture in addition to the first reference picture set, the second reference picture set, and the third reference picture set. Can be. Such an inter-layer reference picture set may be adaptively added to the reference picture list according to the frequency of occurrence. For example, a high frequency reference picture set may be assigned to a low index of the reference picture list.

The initial reference picture list of FIG. 6 is L0 in which the first reference picture set is configured in the lowest order of the list. In this case, the fourth reference picture set may be added in the last order of the initial reference picture list L0.

The initial reference picture list 0 includes pictures having flag information (used_by_curr_pic_flag) of 1 indicating whether the reference picture is included as a reference picture among the reference pictures included in each reference picture set. As shown, the initial reference picture list 0 may be composed of only the first and second pictures among the four pictures included in each reference picture set.

The initial reference picture list may be modified for each picture or slice, and the reference pictures may be modified according to the change. The initial reference picture list may be changed by syntax elements included in the slice hair (eg, ref_pic_list_modification_flag_l0, list_entry_l0, ref_pic_list_modification_flag_l1, list_entry_l1).

ref_pic_list_modification_flag_l0 and ref_pic_list_modification_flag_l1 explicitly indicate whether reference picture list 0 and reference picture list 1 are transmitted from the encoding device to the decoding device. If the flag value is 1, the reference picture list is explicitly specified using the transmitted reference picture information. If the flag value is 0, the reference picture list is implicitly derived to the initial reference picture set.

When the flag value is 1, list_entry_l0 indicates specific entry information constituting the reference picture list for L0, that is, the index of the reference picture, and list_entry_l1 indicates specific entry information constituting the reference picture list for L1.

With this change, the final reference picture list 0 is the first and second reference pictures (0, 1) of the first reference picture set, the first and second reference pictures (0, 1) and the first of the second reference picture set. It may consist of the first reference picture (0) of the 4 reference picture set.

According to another example of the present invention, when configuring the reference picture list, the fourth reference picture set for the inter-layer reference picture may be arranged in an order other than the last in the list of FIG. 6. In the multi-layer structure, since there is a strong correlation between the enhancement layer and the base layer due to the characteristics of the image, the inter-layer reference picture may be frequently referred to. The encoding performance of the reference picture list may be improved by adding the high frequency interlayer reference picture to a position other than the last of the reference picture list.

In consideration of the above points, the inter-layer reference picture may be added at various positions of the reference picture list. 7 to 10 are conceptual diagrams illustrating an example of configuring an interlayer reference picture list according to other examples of the present invention.

FIG. 7 illustrates a structure of a reference picture list O according to an embodiment of the present invention. As illustrated, a fourth reference picture set, which is an interlayer reference picture set, may be arranged in a second order of the reference picture list.

The reference picture list O consists of a first reference picture set, a fourth reference picture set, a second reference picture set, and a third reference picture set.

Reference picture list 0 is composed of pictures having flag information (used_by_curr_pic_flag) of 1 indicating whether the reference picture is included as a reference picture among reference pictures included in each reference picture set. As shown, the initial reference picture list 0 may be composed of only the first and second pictures among the four pictures included in each reference picture set.

The initial reference picture list configured as described above may be modified for each picture or slice, and the entry of the reference pictures may be modified according to the change.

With this change, the final reference picture list 0 is the first and second reference pictures (0, 1) of the first reference picture set, the first and second reference pictures (0, 1) of the fourth reference picture set, It may consist of the first reference picture (0) of the second reference picture set.

8 is a diagram illustrating a configuration of a reference picture list O according to another example of the present invention.

As shown in <a> of FIG. 8, the reference picture list O includes a first reference picture set composed of short-term reference pictures having a smaller POC than the POC of the current picture, and a second reference picture including a short-term reference picture having a larger POC than the POC of the current picture. The set may be configured in the order of a fourth reference picture set composed of inter-layer reference pictures and a third reference picture set composed of long-term reference pictures.

Alternatively, the reference picture list O may include a first reference picture set including a short-term reference picture, in which a fourth reference picture set consisting of inter-layer reference pictures is aligned first, and having a POC smaller than that of the current picture, as shown in FIG. The second reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture and the third reference picture set consisting of a long-term reference picture may be ordered.

In the case of <b>, when the frequency of referring to the inter-layer reference picture is high, there is an advantage in that encoding and decoding efficiency of the reference picture list can be improved.

Alternatively, according to another embodiment, similar to generating a reference picture list of a single layer, a first reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, and a short-term reference picture having a POC larger than the POC of the current picture. After configuring the second reference picture set consisting of the second reference picture set and the third reference picture set consisting of the long-term reference picture, it is possible to specify a position to add the interlayer reference picture in the slice or sequence level, that is, the slice header or the sequence parameter header.

A reference picture list can be constructed by adding an inter-layer reference picture from the specified position. Such signaling may be encoded in a slice header, a sequence header, or a video parameter set and transmitted to the decoding apparatus.

FIG. 9 illustrates a configuration of reference picture list 1 according to an embodiment of the present invention. As shown in FIG. 9, a fourth reference picture set, which is an interlayer reference picture set, may be arranged in the last order of the reference picture list.

Reference picture list 1 includes a second reference picture set, a first reference picture set, a third reference picture set, and a fourth reference picture set.

Reference picture list 1 includes pictures having flag information (used_by_curr_pic_flag) of 1 indicating whether the reference picture is included as a reference picture among reference pictures included in each reference picture set. As shown, the initial reference picture list 1 may be composed of only the first and second pictures among the four pictures included in each reference picture set.

The initial reference picture list configured as described above may be modified for each picture or slice, and the entry of the reference pictures may be modified according to the change.

With this change, the final reference picture list 1 is used to determine the first reference picture (0) of the second reference picture set, the first and second reference pictures (0, 1) of the first reference picture set, and the fourth reference picture set. It may consist of the first and second reference pictures (0, 1).

10 is a diagram illustrating a configuration of reference picture list 1 according to another example of the present invention.

As shown in FIG. 10A, reference picture list 1 includes a second reference picture set including short-term reference pictures having a larger POC than the POC of the current picture, and a first reference picture including a short-term reference picture having a POC smaller than the POC of the current picture. The set may be configured in the order of a fourth reference picture set composed of inter-layer reference pictures and a third reference picture set composed of long-term reference pictures.

Alternatively, in reference picture list 1, as shown in <b> of FIG. 10, a second reference picture set composed of short-term reference pictures having a larger POC than the POC of the current picture is allocated to the first order, and a fourth reference picture composed of inter-layer reference pictures is allocated. Sets can be assigned in second order. Subsequently, reference picture list 1 may be configured in order of a first reference picture set composed of short-term reference pictures having a smaller POC than a POC of the current picture, and a third reference picture set composed of long-term reference pictures.

Alternatively, similar to generating a reference picture list of a single layer, a second reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture, and a first reference picture including a short-term reference picture having a POC smaller than the POC of the current picture. After configuring the third reference picture set consisting of the set and the long-term reference picture, the position at which the inter-layer reference picture is added may be specified in the slice or sequence level, that is, the slice header or the sequence parameter header.

A reference picture list can be constructed by adding an inter-layer reference picture from the specified position. Such signaling may be encoded in a slice header, a sequence header, or a video parameter set and transmitted to the decoding apparatus.

When the scalability type is a multiview in an image that supports multiple layers, a layer for a different view may be used as a reference picture of the current layer. When a plurality of viewpoints, that is, a plurality of layers are used as reference pictures, a fourth reference picture set composed of inter-layer reference pictures may be divided and included in the reference picture list. The reference picture list may be configured in various ways according to the viewpoint order of the current layer and the layer of the reference picture or the degree of proximity of the viewpoint.

For example, the reference pictures may be separated and sorted when configuring the reference picture list according to the difference between the current view and the reference view, and the lower view and the higher view than the current view may be separated and sorted based on the current view. It may be.

11 illustrates a reference picture list constructed by using a multi-view reference picture according to an embodiment of the present invention.

As shown, when the bestrime supports five view layers, or when a reference relationship is established between the five view layers, view 2 may be assumed to be the current layer. A view 0 and a view 1 having a view order lower than the current view based on the current view may constitute one sub reference picture set (first sub reference picture set) or may be included in the first sub reference picture set, and may be included in the current view. Viewpoints 3 and 4 having higher view order may constitute one sub-reference picture set (second sub-reference picture set) or may be included in the second sub-reference picture set.

That is, the first sub-reference layer picture set may be composed of pictures having a viewpoint order that is smaller than the viewpoint of the current picture among the multi-view reference layer pictures, and the second sub-reference layer picture set may be configured of the current picture among the multi-view reference layer pictures. It may be composed of pictures having a view order larger than a view.

As shown in <a>, the reference picture list 0 includes a first reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, a first sub-reference picture set, and a short-term reference picture having a larger POC than the POC of the current picture. The third reference picture set may include a second reference picture set, a second sub reference picture set, and a long term reference picture.

Reference picture list 1 includes a second reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture, a second sub-reference picture set, a first reference picture set consisting of a short-term reference picture having a POC smaller than the POC of the current picture, The first reference picture set may be configured in order of the third reference picture set including the long-term reference picture.

Alternatively, reference picture list 0 may be a short-term reference picture having a POC larger than the POC of the current picture, the first reference picture set, the first sub-reference picture set, and the POC of the current picture, each having a shorter POC than the POC of the current picture, as shown in <b>. The second reference picture set may be configured, the third reference picture set may be configured by a long-term reference picture, and the second sub-reference picture set may be configured in this order. That is, the second sub-reference picture set may be added to the end of the reference picture list.

Reference picture list 1 similarly includes a second reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture, a second sub-reference picture set, and a first reference picture consisting of a short-term reference picture having a POC smaller than the POC of the current picture. The reference picture list may be configured in the order of the set and the third reference picture set including the long-term reference picture, and the first sub-reference picture set may be added in the last order.

Although the multi-layer reference picture set of FIG. 11 includes a multi-view reference picture, although the type of scalability is spatial scalability, even when pictures of a plurality of layers are used as the reference picture, the reference picture shown in FIG. 11 is used. The construction of the list can be applied. That is, the reference pictures may be divided and added to the reference picture list according to a difference in the order of identifying the current picture and the spatial layer.

12 illustrates a reference picture list constructed using a multi-view reference picture according to another example of the present invention.

The lower the difference between the current layer and the viewpoint, the more likely that the picture of the layer is used as the inter-layer reference picture. In other words, the greater the difference between the current layer and the viewpoint, the less likely the current picture is to refer to. According to the present embodiment, the order in which the reference picture is added to the reference picture list may be adaptively changed according to the frequency of occurrence of the reference picture, that is, the proximity order in consideration of the reference degree between viewpoints.

That is, a low index of a reference picture list may be allocated to a reference picture of a layer having a small difference between the current layer and a viewpoint, and a high index of a reference picture list may be allocated to a reference picture of a layer having a large difference between the current layer and a viewpoint. have.

For example, as illustrated in FIG. 12, reference picture 0 among reference picture 0, reference picture 1, reference picture 2, and reference picture 3, which may be included in a multi-view reference picture set, is arranged in front of reference picture list 0. The reference picture 1, the reference picture 2, and the reference picture 3 may be aligned at the end of the reference picture list 0 which is the next order of the third reference picture set.

In this case, the reference picture 0 may be a reference picture of a layer having a small difference between the current layer and a viewpoint, and the reference picture 1, the reference picture 2, and the reference picture 3 may be pictures having a large difference between the current layer and the viewpoint.

The difference value between the current layer and the start point of the reference layer that may be aligned in the front in the reference picture list may be a default value, and may be variably set in the encoding apparatus and transmitted to the decoding apparatus.

According to another embodiment of the present invention, the order of the inter-layer reference picture set may be set according to the index (tempralId) of the temporal sub-layer of the current picture.

13 illustrates a reference picture list constructed by using an index of a temporal sublayer according to an embodiment of the present invention.

In the multi-layer structure, the frequency of using the inter-layer reference picture may vary according to the temporal sublayer of the current picture belonging to the enhancement layer. As the tempralId which is an index value of the temporal sublayer is larger, it may be more effective to perform inter prediction with reference to pictures in the same layer than to reference pictures of other layers.

In other words, if the tempralId is small, it is more effective to code the interlayer picture than the same layer, and if the tempralId is larger, it is better to code the picture of the same layer as the reference picture than to use the interlayer as the reference picture. It can be efficient.

In this embodiment, in consideration of the fact that the reference occurrence frequency of the interlayer picture varies according to the temporal sublayer, the position at which the interlayer reference picture is added to the initial reference picture list is adaptively changed according to the index tempralId of the temporal sublayer. Can be.

FIG. 13 is a diagram illustrating the configuration of a reference picture list 0 when the tempralId value is 0 to 2. FIG.

When the tempralId value is 0, since the interlayer prediction is highly likely to be performed on the enhancement layer, the fourth reference picture set composed of the interlayer reference pictures may be added to the beginning of the initial reference picture list.

An initial reference picture list consisting of pictures having a flag information (used_by_curr_pic_flag) of 1 indicating whether the reference picture included in each reference picture set is used as a reference picture is four pictures included in the fourth reference picture set (0 , 1, 2, 3), the first reference picture set, the second reference picture set, and the first and second pictures (0, 1) included in the third reference picture set.

As shown in the drawing, after the reference picture list initial value is set, the reference picture list may be modified and then the final reference picture list may be set. Information for modifying the reference picture list may be signaled in the slice header.

The modified last reference picture list 0 is the first and second pictures (0,1) of the first reference picture set, the first and second pictures (0,1) of the fourth reference picture set, and the second reference picture of the second reference picture set. It consists of the first picture (0).

When the tempralId value is 1, the fourth reference picture set including the interlayer reference picture may be added after the first reference picture set in the initial reference picture list 0.

The initial reference picture list, which consists of pictures with flag information (used_by_curr_pic_flag) equal to 1 indicating whether it is used as a reference picture, is included in the first reference picture set, the fourth reference picture set, the second reference picture set, and the third reference picture set. It consists of the first and second pictures (0, 1).

The modified last reference picture list 0 is the first and second pictures (0,1) of the first reference picture set, the first and second pictures (0,1) of the fourth reference picture set, and the second reference picture of the second reference picture set. It consists of the first picture order.

When the tempralId value is 2, it is more likely that inter prediction between the same layers is performed than inter layer prediction of the enhancement layer, so that the fourth reference picture set composed of the interlayer reference pictures may have a third reference picture set in the initial reference picture list. Then it can be added last.

An initial reference picture list 0 consisting of pictures with flag information (used_by_curr_pic_flag) of 1 indicating whether it is used as a reference picture is assigned to the first reference picture set, the second reference picture set, and the third reference picture set and the fourth reference picture set. It consists of the first and second pictures (0, 1) included.

The modified last reference picture list includes the first and second pictures (0,1) of the first reference picture set, the first and second pictures (0,1) of the second reference picture set, and the first of the fourth reference picture sets. It consists of the first picture (0).

Similar to the above, the initial reference list L1 may also change the position where the fourth reference picture set is added according to the temporalId value. For example, when the temporalId value is 0, the initial reference list L1 may be configured in the order of the fourth reference picture set, the second reference picture set, the first reference picture set, and the third reference picture set, and the temporalId value is one. The second reference picture set, the fourth reference picture set, the first reference picture set, and the third reference picture set may be configured in this order. When the temporalId value is large as 2, the fourth reference picture set may be added in the last order of the initial reference list L1.

In summary, as shown in FIG. 13, the position at which the interlayer reference picture is added to the initial reference picture list may be adaptively changed according to the temporalId value of the temporal sublayer.

Meanwhile, according to another embodiment of the present invention, the order of configuring the reference picture list may be changed according to the difference in the layer index temporalId value of the temporal sublayer between layers in the multi-layer structure.

In the multi-layer structure, if the temporalId values of the enhancement layer and the base layer are different from each other or the difference between the temporalId values is large, there is less possibility that the enhancement layer refers to the base layer, that is, there is an interlayer reference relationship.

On the other hand, when the temporalId value of the enhancement layer and the baseor temporalId value are the same or the difference between the temporalId values is small, the possibility that the enhancement layer refers to the base layer, that is, the existence of an inter-layer reference relationship increases.

Since the frequency of occurrence of the interlayer reference varies according to the difference between the temporalId values of the enhancement layer and the base layer, the position at which the interlayer reference picture is added to the reference picture list can be adaptively changed according to the difference of the temporalId values between the layers. .

For example, if the temporalId value of a picture belonging to an enhancement layer and the temporalId value of a picture belonging to a base layer or a reference layer are the same, the interlayer reference picture set is added to the beginning of the reference picture list, and the picture belonging to the two layers is added. If the temporalId value is different, the inter-layer reference picture set may be added at the end after the long-term reference picture set.

Alternatively, when adding an interlayer reference picture to the reference picture list, if the difference between the temporalId value of the picture belonging to the enhancement layer and the temporalId value of the picture belonging to the base layer or the reference layer is smaller than a specific value, the first part of the reference picture list is displayed. In addition, when a difference between temporalId values of pictures belonging to two layers is greater than a specific value, an inter-layer reference picture set may be added at the end after the long-term reference picture set.

Alternatively, when an interlayer reference picture is added to the front of the reference picture list, it may be added as a second reference picture set other than the first in the reference picture list, or when the interlayer reference picture is added to the back of the reference picture list. It may be added before the last non-long-term reference picture set of the list.

For example, if the current picture of the enhancement layer and the temporalId value of the base layer picture used as the interlayer reference picture are both equal to 2, the interlayer reference picture may be added to the beginning of the reference picture list.

On the other hand, when the temporalId value of the current picture of the enhancement layer is 4 and the temporalId value of the base layer picture used as the interlayer reference picture is 2, the interlayer reference picture is added to the reference picture list in the order of the long-term reference picture set. Can be.

According to another embodiment of the present invention, a reference picture list may be adaptively constructed according to the number of maximum time base sublayers that allow interlayer prediction.

The bitstream supporting the multi-layer may include information about a maximum temporal sub-layer that allows inter-layer prediction. Such information may be transmitted in syntax elements of max_tid_il_ref_pics_plus1 included in the video parameter set. A value obtained by subtracting 1 from max_tid_il_ref_pics_plus1 represents the maximum number of time base sublayers that allow interlayer prediction.

In this case, when constructing the reference picture list, if the temporalId value of the current layer is greater than the value specified by subtracting 1 from max_tid_il_ref_pics_plus1, it is better to use the picture on the same layer as the reference picture than the upsampled base layer picture. It is advantageous in terms of encoding and decoding efficiency. Therefore, when the temporalId value of the current layer is larger than the value specified by max_tid_il_ref_pics_plus1-1, the interlayer reference may not be used. In this case, the interlayer reference picture set may not be added to the reference picture list, or a separate signal may not be signaled even if the interlayer reference picture set is added.

On the other hand, when the temporalId value of the current layer is less than or equal to the value specified by max_tid_il_ref_pics_plus1, the reference picture list may be configured by combining the short-term reference picture set, the long-term reference picture set, and the inter-layer reference picture set for the current layer. In this case, the reference picture set may be configured according to the embodiments described with reference to FIGS. 6 to 13 and the embodiments that can be modified in the embodiments.

Meanwhile, among lower layers of the current layer, a lower layer whose temporalId value is greater than the value obtained by subtracting 1 from max_tid_il_ref_pics_plus1 is not referenced when predicting the current layer.

Since inter prediction does not refer to a slice or picture whose termporalId value is larger than the current slice or picture, the slice or picture of a base layer having a termporalId value larger than the termporalId value of the current slice or picture is not referred to. Can be.

As such, when the inter-layer reference picture set is not used according to the temporalId of the temporal sublayer, the reference picture list may be constructed using only the short-term reference picture set and the long-term reference picture set for the current layer.

14 is a control flowchart illustrating a method of encoding an image according to the present invention. A method of encoding an image according to the present invention will be described with reference to FIG. 14 as follows.

First, the encoding apparatus generates an inter-layer reference picture set for at least one reference layer that can be referenced by the current picture (S1410).

The inter-layer reference picture set may include at least one of a spatial reference layer picture set, a quality reference layer picture set, a multi-view reference layer picture set, and a depth map reference layer picture set.

The inter-layer reference picture set may consist of reconstructed pictures of the reference layer, and if the inter-layer reference picture set is a spatial reference layer picture set, the reference picture is a picture that upsampled the picture of the reference layer to correspond to the current picture. Can be.

The inter-layer reference picture set may be divided into a plurality of subsets according to characteristics of pictures constituting the reference picture set.

For example, the inter-layer reference picture set may include a multi-view reference layer picture set, and the multi-view reference layer picture set may include a picture having a viewpoint order that is smaller than the view point of the current picture among the multi-view reference layer pictures. It may be formed of a plurality of subsets consisting of pictures having a viewpoint order larger than the viewpoint of the current picture among the point reference layer pictures.

When the interlayer reference picture set is generated, the encoding apparatus generates an initial reference picture list including the interlayer reference picture set (S1420).

The initial reference picture list may include an initial reference picture list 0 and an initial reference picture list 1, and the initial reference picture list 0 and the initial reference picture list 1 each include a short-term reference picture composed of short-term reference pictures having a POC smaller than the POC of the current picture. A set, a short-term reference picture set composed of short-term reference pictures having a larger POC than the POC of the current picture, a long-term reference picture set composed of long-term reference pictures, and an inter-layer reference picture set may be included. In this case, the order of the reference picture set may be variably set.

For example, initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures consisting of short-term reference pictures with a smaller POC than the POC of the current picture, an interlayer reference picture set, and a short-term reference picture consisting of short-term reference pictures with a larger POC than the POC of the current picture. Set, and a long-term reference picture set consisting of a long-term reference picture.

The initial reference picture list 1 is a short-term reference picture set consisting of short-term reference pictures with a larger POC than the POC of the current picture, an interlayer reference picture set, a short-term reference picture set consisting of a short-term reference picture with a POC smaller than the POC of the current picture, and a long-term reference. It may be configured in the order of a long-term reference picture set composed of pictures.

Or when the inter-layer reference picture set includes a multi-view reference layer picture set, the initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, and the current one of the multi-view reference layer pictures. A picture with a view order that is smaller than the picture's view point, a short-term reference picture set consisting of short-term reference pictures with a larger POC than the POC of the current picture, and a long-term reference picture set consisting of a long-term reference picture. It may be configured in the order of pictures having a large view order.

In this case, the initial reference picture list 1 is a short-term reference picture set composed of short-term reference pictures having a larger POC than the POC of the current picture, a picture having a viewpoint order larger than that of the current picture among the multi-view reference layer pictures, and a POC of the current picture. A short-term reference picture set consisting of short-term reference pictures having a small POC, a long-term reference picture set consisting of a long-term reference picture, and a picture having a viewpoint order smaller than that of the current picture among the multi-view reference layer pictures may be configured.

Alternatively, the interlayer reference picture set may be generated based on an index (tempralId) of a temporal sublayer of the current picture or based on the maximum number of timebase sublayers that allow interlayer prediction.

The encoding apparatus performs prediction on the current picture based on the final reference picture list when the final reference picture list is generated through the modification of the reference picture list, or on the basis of the initial reference picture list if the initial reference picture list is not modified. The encoding of the current picture is performed (S1430).

In the prediction of the current picture, a reference picture included in the interlayer reference picture set is represented as a long-term reference picture, and the encoding apparatus uses the reference picture included in the interlayer reference picture set as a reference picture of the current picture.

15 is a control flowchart for explaining a method of decoding an image according to the present invention.

A method of decoding an image according to the present invention will be described with reference to FIG. 15 as follows.

First, the decoding apparatus generates an inter-layer reference picture set for at least one reference layer that can be referenced by the current picture (S1510). The inter-layer reference picture set may be generated using information for identifying a layer and information for identifying a reference picture. For example, a nuh_layer_id value present in the same access unit as the current picture of the i th layer and transmitted in the NAL unit header (i is greater than j) is a nuh_layer_id value. If it is equal to the RefPiclayerId value for, the picture may be determined to be used as a reference picture for the current picture.

The inter-layer reference picture set may include at least one of a spatial reference layer picture set, a quality reference layer picture set, a multi-view reference layer picture set, and a depth map reference layer picture set.

The inter-layer reference picture set may include reconstructed pictures of the reference layer, and if the inter-layer reference picture set is a spatial reference layer picture set, the reference picture may be a picture that is upsampled so that the picture of the reference layer corresponds to the current picture. .

The inter-layer reference picture set may be divided into a plurality of subsets according to characteristics of pictures constituting the reference picture set.

For example, the inter-layer reference picture set may include a multi-view reference layer picture set, and the multi-view reference layer picture set may include a picture having a viewpoint order that is smaller than the view point of the current picture among the multi-view reference layer pictures. It may be formed of a plurality of subsets consisting of pictures having a viewpoint order larger than the viewpoint of the current picture among the point reference layer pictures.

When the interlayer reference picture set is generated, the decoding apparatus generates an initial reference picture list including the interlayer reference picture set (S1520).

The initial reference picture list may include an initial reference picture list 0 and an initial reference picture list 1, and the initial reference picture list 0 and the initial reference picture list 1 each include a short-term reference picture composed of short-term reference pictures having a POC smaller than the POC of the current picture. A set, a short-term reference picture set composed of short-term reference pictures having a larger POC than the POC of the current picture, a long-term reference picture set composed of long-term reference pictures, and an inter-layer reference picture set may be included. The initial reference picture list is composed of pictures having flag information (used_by_curr_pic_flag) of 1 indicating whether the reference picture is used as a reference picture among the reference pictures included in the reference picture set.

In this case, the order of the reference picture sets arranged in the initial reference picture list may be variably set.

For example, initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures consisting of short-term reference pictures with a smaller POC than the POC of the current picture, an interlayer reference picture set, and a short-term reference picture consisting of short-term reference pictures with a larger POC than the POC of the current picture. Set, and a long-term reference picture set consisting of a long-term reference picture.

Initial reference picture list 1 consists of a short-term reference picture set consisting of short-term reference pictures with a larger POC than the POC of the current picture, an interlayer reference picture set, a short-term reference picture set consisting of a short-term reference picture with a POC smaller than the POC of the current picture, and a long-term reference picture. It may be configured in the order of a long-term reference picture set consisting of.

Or when the inter-layer reference picture set includes a multi-view reference layer picture set, the initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, and the current one of the multi-view reference layer pictures. A picture with a view order that is smaller than the picture's view point, a short-term reference picture set consisting of short-term reference pictures with a larger POC than the POC of the current picture, and a long-term reference picture set consisting of a long-term reference picture. It may be configured in the order of pictures having a large view order.

In this case, the initial reference picture list 1 is a short-term reference picture set composed of short-term reference pictures having a larger POC than the POC of the current picture, a picture having a viewpoint order larger than that of the current picture among the multi-view reference layer pictures, and a POC of the current picture. A short-term reference picture set consisting of short-term reference pictures having a small POC, a long-term reference picture set consisting of a long-term reference picture, and a picture having a viewpoint order smaller than that of the current picture among the multi-view reference layer pictures may be configured.

Alternatively, the interlayer reference picture set may be generated based on an index (tempralId) of a temporal sublayer of the current picture or based on the maximum number of timebase sublayers that allow interlayer prediction.

The decoding apparatus performs prediction on the current picture based on the last reference picture list when the final reference picture list is generated through the modification of the reference picture list, or on the basis of the initial reference picture list if the initial reference picture list is not modified. Decoding of the current picture is performed (S1530).

In prediction of the current picture, a reference picture included in the interlayer reference picture set is represented as a long-term reference picture, and the decoding apparatus uses the reference picture included in the interlayer reference picture set as a reference picture of the current picture.

In the exemplary system described above, the methods are described based on a flowchart as a series of steps or blocks, but the invention is not limited to the order of steps, and certain steps may occur in a different order or concurrently with other steps than those described above. Can be. In addition, since the above-described embodiments may include examples of various aspects, a combination of each embodiment should also be understood as an embodiment of the present invention. Accordingly, the invention is intended to embrace all other replacements, modifications and variations that fall within the scope of the following claims.

* 100: encoding device 200: decoding device
100a: encoder of the upper layer 100b: encoder of the lower layer
200a: decoder of upper layer 200b: decoder of lower layer

Claims (5)

In the decoding method of an image supporting a plurality of layers,
Generating an inter-layer reference picture set for at least one reference layer that the current picture may refer to;
Generating an initial reference picture list including the inter-layer reference picture set;
Modifying the initial reference picture list to generate a final reference picture list; And
Performing prediction on the current picture based on the last reference picture list,
The inter-layer reference picture set includes a first sub reference layer picture set and a second sub reference layer picture set.
The first sub reference layer picture set and the second sub reference layer picture set are separated and arranged in the initial reference picture list.
The initial reference picture list may include a short-term reference picture set including a short-term reference picture having a smaller POC than the POC of the current picture, the first sub-reference layer picture set, and a short-term reference picture having a POC larger than the POC of the current picture. And a reference picture set, a long-term reference picture set consisting of a long-term reference picture, and the second sub-reference layer picture set. The method of claim 1,
Generating the inter-layer reference picture set
And generating at least one of a spatial reference layer picture set, a quality reference layer picture set, a multi-view reference layer picture set, and a depth map reference layer picture set that the current picture may refer to. The method of claim 1,
And a reference picture constituting the interlayer reference picture set is a picture upsampled so that the picture of the reference layer corresponds to the current picture. delete The method of claim 1,
The first sub-reference layer picture set is composed of pictures having a viewpoint order smaller than the viewpoint of the current picture, and the second sub-reference layer picture set is composed of pictures having a viewpoint order larger than the viewpoint of the current picture. Method of decoding.

Images