KR102179125B1 - Video coding/decoding method and apparatus for multi-layers - Google Patents

Abstract

In the method of decoding an image supporting a plurality of layers according to the present invention, the step of receiving position information on a position at which an inter-layer reference picture set for at least one reference layer that can be referenced by a current picture is added to a reference picture list. And, generating an initial reference picture list including the inter-layer reference picture set based on the location information, and performing prediction on the current picture based on the initial reference picture list. . Accordingly, a method of signaling position information on a position at which an inter-layer reference picture is added to a reference picture list and an apparatus using the same are provided.

Description

A video encoding and decoding method supporting multiple layers, and an apparatus using the same {VIDEO CODING/DECODING METHOD AND APPARATUS FOR MULTI-LAYERS}

The present invention relates to a video compression technique, and more particularly, to a method and apparatus for performing video coding supporting multiple layers.

Recently, demand for high-resolution and high-quality images such as high definition (HD) images and ultra high definition (UHD) images is increasing in various application fields. The higher the resolution and quality of the video data, the higher the amount of data is compared to the existing video data. Therefore, if the video data is transmitted using a medium such as a wired/wireless broadband line or stored using an existing storage medium, the transmission cost and The storage cost increases. High-efficiency image compression techniques can be used to solve these problems that occur as image data becomes high-resolution and high-quality.

Inter-screen prediction technology that predicts pixel values included in the current picture from a picture before or after the current picture using image compression technology, intra prediction technology that predicts pixel values included in the current picture using pixel information in the current picture, Various technologies exist, such as an entropy encoding technology that allocates a short code to a value with a high frequency of appearance and a long code to a value with a low frequency of appearance, and it is possible to effectively compress and transmit or store image data using this image compression technology.

An object of the present invention is to provide a method of generating a reference picture list including pictures of different layers for a layer to be currently encoded and decoded and an apparatus using the same.

Another object of the present invention is to provide a method of adaptively including pictures of different layers in a reference picture list according to characteristics of a picture set, and an apparatus using the same.

Meanwhile, another embodiment of the present invention is to provide a method of dividing reference pictures of different layers and including them in a reference picture list, and an apparatus using the same.

Another embodiment of the present invention is to provide a method of adaptively including reference pictures of different layers in a reference picture list according to the order of views, and an apparatus using the same.

Another embodiment of the present invention is to provide a method of adaptively including a picture of another layer in a reference picture list according to a temporal sub-layer index value of a current picture and an apparatus using the same.

Another embodiment of the present invention is to provide a method of adaptively including pictures of different layers in a reference picture list according to a difference in temporal sub-layer index values between layers, and an apparatus using the same.

Another embodiment of the present invention is to provide a method of signaling position information on a position at which an inter-layer reference picture is added to a reference picture list, and an apparatus using the same.

In the decoding method of an image supporting a plurality of layers according to an embodiment of the present invention, position information on a position at which an inter-layer reference picture set for at least one reference layer that can be referenced by a current picture is added to a reference picture list. Receiving, generating an initial reference picture list including the inter-layer reference picture set based on the location information, and performing prediction on the current picture based on the initial reference picture list. Can include.

The method further comprises generating a short-term reference picture set and a long-term reference picture set composed of pictures existing in the same rare as the current picture, and generating the initial reference picture list may include the An inter-layer reference picture set may be added to a position of the initial reference picture list indicated by the position information.

The location information may be received by being included in a slice header or a sequence header.

The receiving of the location information includes: receiving an identification flag indicating whether an identification index indicating a location at which an inter-layer reference picture that can be referenced by the current picture is added to the initial reference picture list is transmitted; and the When the identification flag is 1, it may include the step of receiving the identification index.

The identification flag may be included in the video parameter set of each layer and received.

The identification index may be received by being included in a slice header or a sequence header.

The initial reference picture list includes an initial reference picture list 0, and the initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures having a smaller POC than a picture of count (POC) of the current picture, and the inter-layer reference It may be configured in the order of a picture set, a short-term reference picture set consisting of a short-term reference picture having a POC larger than that of the current picture, and a long-term reference picture set consisting of a long-term reference picture.

The initial reference picture list includes an initial reference picture list 1, and the initial reference picture list 1 is a short-term reference picture set composed of a short-term reference picture having a larger POC than a POC of the current picture, the inter-layer reference picture set, and the current A short-term reference picture set composed of short-term reference pictures having a smaller POC than that of the picture may be configured in the order of a long-term reference picture set composed of a long-term reference picture.

The inter-layer reference picture set includes a first sub-reference layer picture set and a second sub-reference layer picture set, and the first sub-reference layer picture set and the second sub-reference layer picture set are separated and the initial reference picture Can be sorted in a list.

The inter-layer reference picture set is composed of a multiview reference layer picture, the initial reference picture list includes an initial reference picture list 0, and the initial reference picture list 0 is a short-term reference picture having a smaller POC than the POC of the current picture. A short-term reference picture set consisting of, the first sub-reference layer picture set consisting of pictures having a view order smaller than the viewpoint of the current picture among the multi-view reference layer pictures, and a short-term reference picture having a POC larger than the POC of the current picture. The second sub-reference layer picture set composed of a configured short-term reference picture set, a long-term reference picture set composed of a long-term reference picture, and a picture having a view order greater than that of the current picture among the multi-view reference layer pictures may be arranged in order. have.

The inter-layer reference picture set is composed of multi-view reference layer pictures, the initial reference picture list includes an initial reference picture list 1, and the initial reference picture list 1 is a short-term reference picture having a larger POC than the POC of the current picture A short-term reference picture set consisting of, the second sub-reference layer picture set consisting of pictures having a view order greater than that of the current picture among the multi-view reference layer pictures, and a short-term reference picture having a POC smaller than the POC of the current picture. The first sub-reference layer picture set composed of a configured short-term reference picture set, a long-term reference picture set composed of a long-term reference picture, and a picture having a view order smaller than that of the current picture among the multi-view reference layer pictures may be arranged in order. have.

The inter-layer reference picture set may further include receiving flag information indicating whether the inter-layer reference picture set is divided into at least two sub-reference layer picture sets and added to the reference picture list.

If the flag information is 1, the step of receiving information on the number of sub-reference layer picture sets from which the inter-layer reference picture set is separated may be further included.

The generating of the inter-layer reference picture set may be generated based on a temporal sub-layer index (tempralId) of the current picture.

The generating of the inter-layer reference picture set may be generated based on the number of maximum temporal sub-layers allowing inter-layer prediction.

It may further include generating a final reference picture list by modifying the initial reference picture list.

In performing prediction on the current picture, a reference picture included in the inter-layer reference picture set may be used as a reference picture of the current picture.

According to another aspect of the present invention, an apparatus for decoding an image supporting a plurality of layers includes an entropy decoder for decoding information for prediction and decoding of an image received through a beast stream; Inducing position information on a position at which an inter-layer reference picture set for at least one reference layer that can be referenced by a current picture is added to a reference picture list, and including the inter-layer reference picture set based on the position information It may include a prediction unit that generates an initial reference picture list and performs prediction on the current picture based on the initial reference picture list.

According to an embodiment of the present invention, a method of generating a reference picture list including pictures of different layers for a layer to be currently encoded and decoded and an apparatus using the same are provided.

In addition, according to an embodiment of the present invention, a method of adaptively including pictures of different layers in a reference picture list according to characteristics of a picture set and an apparatus using the same are provided.

Meanwhile, according to another embodiment of the present invention, a method of dividing a reference picture of another layer and including it in a reference picture list and an apparatus using the same are provided.

According to another embodiment of the present invention, a method of adaptively including reference pictures of different layers in a reference picture list according to an order of views and an apparatus using the same are provided.

In addition, according to another embodiment of the present invention, a method of adaptively including a picture of another layer in a reference picture list according to a temporal sub-layer index value of a current picture and an apparatus using the same are provided.

According to another embodiment of the present invention, a method of adaptively including pictures of different layers in a reference picture list according to a difference in temporal sub-layer index values between layers and an apparatus using the same are provided.

According to another embodiment of the present invention, a method for signaling position information on a position at which an inter-layer reference picture is added to a reference picture list and an apparatus using the same are provided.

1 is a block diagram schematically illustrating an encoding apparatus according to an embodiment of the present invention.
2 is a schematic block diagram of a decoding apparatus according to an embodiment of the present invention.
3 is a conceptual diagram illustrating an example of a reference picture set including a short-term reference picture.
4 is a conceptual diagram illustrating a method of inducing a POC of a long-term reference picture.
5 is a diagram illustrating an example of configuring a reference picture list.
6 is a conceptual diagram illustrating an example of configuring an inter-layer reference picture list according to an example of the present invention.
7 is a diagram illustrating a configuration of a reference picture list O according to an example of the present invention.
8 is a diagram illustrating a configuration of a reference picture list O according to another example of the present invention.
9 is a diagram illustrating a configuration of a reference picture list 1 according to an example of the present invention.
10 is a diagram illustrating a configuration of a reference picture list 1 according to another example of the present invention.
11 is a diagram illustrating a reference picture list constructed using a temporal sub-layer index according to an example of the present invention.
12 is a diagram illustrating a reference picture list configured using a multiview reference picture according to an example of the present invention.
13 is a diagram for describing a method of generating a reference picture list according to an embodiment of the present invention.
14 is a diagram illustrating a method of generating a reference picture list according to another embodiment of the present invention.

Each of the components disclosed in the embodiments of the present invention and the drawings is disclosed as an independent configuration to represent different characteristic functions of the video encoding apparatus. It does not mean that each component is made up of separate hardware or a single software component. That is, each constituent part is listed and included as a constituent part for convenience of explanation, and at least two of the constituent parts are combined to form a single constituent part, or one constituent part is divided into a plurality of constituent parts to perform a function. The integrated embodiments and separate embodiments of the components are also included in the scope of the present invention unless departing from the essence of the present invention.

In addition, some of the constituent elements disclosed in the present invention are not essential constituent elements that perform essential functions in the present invention, but may be optional constituent elements merely for improving performance. The present invention can be implemented by including only the components essential to implement the essence of the present invention excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments. The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the technical idea of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance the possibility.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

Encoding and decoding of a video supporting a plurality of layers in a bitstream is called scalable coding. Since a strong correlation exists between a plurality of layers, if prediction is performed using this correlation, redundant elements of data can be removed and encoding performance of an image can be improved. Prediction of a current layer to be predicted using information of other layers is hereinafter referred to as inter-layer prediction.

The plurality of layers may have at least one of a resolution, a frame rate, and a color format different from each other, and up-sampling or down-sampling of a layer may be performed to adjust a resolution during inter-layer prediction.

1 is a block diagram schematically illustrating an encoding apparatus according to an embodiment of the present invention.

The encoding apparatus 100 according to the present invention includes an encoding unit 100a for an upper layer and an encoding unit 100b for a lower layer.

The upper layer may be expressed as a current layer or an enhancement layer, and the lower layer may be expressed as a reference layer or a base layer. The upper layer and the lower layer may have at least one of a resolution, a frame rate, and a color format different from each other. When a resolution change is required to perform inter-layer prediction, up-sampling or down-sampling of a layer may be performed.

The encoding unit 100a of the upper layer includes a splitter 110, a prediction unit 100, an intra prediction unit 121, an inter prediction unit 122, an inter-layer prediction unit 123, and a transform unit 130. , A quantization unit 140, a rearrangement unit 150, an entropy encoding unit 160, an inverse quantization unit 170, an inverse transform unit 180, a filter unit 190, and a memory 195 and a MUX 197 can do.

The lower layer encoder 100b includes a splitter 111, a prediction unit 125, an intra prediction unit 126, an inter prediction unit 127, a transform unit 131, a quantization unit 141, and rearrangement. A unit 151, an entropy encoder 161, an inverse quantization unit 171, an inverse transform unit 181, a filter unit 191, and a memory 196 may be included.

The encoding unit may be implemented by the image encoding method described in the following embodiments of the present invention, but operations in some components may not be performed to reduce the complexity of the encoding apparatus or for fast real-time encoding. For example, in performing intra prediction in the prediction unit, a limited number of parts without using a method of selecting the optimal intra prediction mode by using all intra prediction mode methods to perform encoding in real time. A method of selecting one intra-prediction mode among them as the final intra-prediction mode using the intra prediction mode of may be used. As another example, in performing intra prediction or inter prediction, it is possible to restrict the use of the shape of the prediction block used.

A unit of a block processed by the encoding apparatus may be a coding unit that performs encoding, a prediction unit that performs prediction, and a transformation unit that performs transformation. A coding unit may be expressed in terms of a coding unit (CU), a prediction unit (PU), and a transform unit (TU).

The splitters 110 and 111 divide the layer image into a combination of a plurality of coding blocks, prediction blocks, and transform blocks, and use one of the coding blocks, prediction blocks, and transform blocks based on a predetermined criterion (for example, a cost function). You can divide the layer by selecting a combination of. For example, a recursive tree structure such as a quadtree structure may be used to divide a coding unit in a layer image. Hereinafter, in an embodiment of the present invention, the meaning of a coded block may be used not only as a block to be encoded, but also to a block to be decoded.

The prediction block may be a unit that performs prediction such as intra prediction or inter prediction. A block that performs intra prediction may be a square block such as 2Nx2N and NxN. As a block that performs inter-screen prediction, the prediction block is divided using 2NxN, Nx2N, or AMP (Asymmetric Motion Partitioning), which is a form obtained by dividing a square shape such as 2Nx2N, NxN or a square prediction block into the same shape. There is a way. A method of performing the transformation in the transform unit 115 may vary according to the shape of the prediction block.

The prediction units 120 and 125 of the encoding units 100a and 100b are the intra prediction units 121 and 126 that perform intra prediction and the inter prediction units that perform inter prediction. (1122, 126) may be included. The prediction unit 120 of the upper layer encoder 100a further includes an inter-layer prediction unit 123 that performs prediction on an upper layer using information on the lower layer.

The prediction units 120 and 125 may determine whether to use inter prediction or intra prediction on the prediction block. A processing unit in which prediction is performed and a unit of a processing block in which a prediction method is determined may be different. For example, in performing intra prediction, a prediction mode may be determined based on a prediction block, and a process of performing prediction may be performed based on a transform block. A residual value (residual block) between the generated prediction block and the original block may be input to the transform units 130 and 131. In addition, prediction mode information, motion vector information, and the like used for prediction may be encoded by the entropy encoder 130 together with a residual value and transmitted to a decoding apparatus.

When using the PCM (Pulse Coded Modulation) encoding mode, it is possible to encode the original block as it is and transmit it to the decoder without performing prediction through the prediction units 120 and 125.

The intra prediction units 121 and 126 may generate an intra-predicted block based on a reference pixel existing around the current block (a block to be predicted). In the intra prediction method, the intra prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional mode in which no directional information is used when prediction is performed. A mode for predicting luma information and a mode for predicting color difference information may have different types. In order to predict color difference information, intra prediction mode information for predicting luma information or predicted luma signal information may be used. If the reference pixel is not available, a prediction block may be generated by replacing the non-available reference pixel with another pixel.

The prediction block may include a plurality of transform blocks. If the size of the prediction block and the size of the transform block are the same when performing intra prediction, a pixel on the left side of the prediction block, a pixel on the top left side, and the top In-screen prediction on the prediction block may be performed based on the pixels existing in the screen. However, when performing intra prediction, if the size of the prediction block and the size of the transform block are different and a plurality of transform blocks are included in the prediction block, intra prediction is performed using a reference pixel determined based on the transform block. can do.

In the intra prediction method, a prediction block may be generated after applying a Mode Dependent Intra Smoothing (MDIS) filter to a reference pixel according to an intra prediction mode. The types of MDIS filters applied to the reference pixels may be different. The MDIS filter is an additional filter applied to an intra-predicted block by performing intra prediction, and may be used to reduce a step that exists in an intra-predicted block generated after prediction with a reference pixel. In performing MDIS filtering, different filtering may be performed on a reference pixel and some columns included in the predicted block in the screen according to the direction of the prediction mode in the screen.

The inter prediction units 122 and 127 may perform prediction by referring to information on a block included in at least one picture of a picture before or after a current picture. The inter prediction units 122 and 127 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolation unit may receive reference picture information from the memories 195 and 196 and generate pixel information of an integer number of pixels or less from the reference picture. In the case of a luma pixel, a DCT-based interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/4 pixels. In the case of a color difference signal, a DCT-based interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/8 pixels.

The inter prediction units 122 and 127 may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. Various methods such as a full search-based block matching algorithm (FBMA), a three step search (TSS), and a new three-step search algorithm (NTS) can be used as a method for calculating a motion vector. The motion vector may have a motion vector value of 1/2 or 1/4 pixel based on the interpolated pixel. The inter prediction units 122 and 127 may perform prediction on the current block by applying one of several inter prediction methods.

Various methods, such as a skip method, a merge method, and a motion vector prediction (MVP) method, may be used as an inter prediction method.

In inter prediction, motion information, that is, information such as an index of a reference picture, a motion vector, and a residual signal, is entropy-coded and transmitted to a decoder. When the skip mode is applied, residuals may not be generated, transformed, quantized, or transmitted.

The inter-layer prediction unit 123 performs inter-layer prediction for predicting an upper layer using information on a lower layer. The inter-layer prediction unit 123 uses inter-layer texture prediction and inter-layer inter prediction using texture of a lower layer, intra prediction mode information, motion information, and syntax information. ), inter-layer syntax prediction, etc. may be performed.

Inter-layer texture prediction means using a texture of a reference block in a lower layer as a prediction sample of a current block of an upper layer, and in this case, the texture of the reference block may be scaled by upsampling.

For inter-layer texture prediction, the intra-BL method and the up-sampled lower layer upsample the reconstructed value of the reference block in the lower layer and use the upsampled reference block as a prediction value for the current block to encode the residual value from the current block. There may be a reference index method that stores in memory and uses the stored lower layer as a reference index.

The intra prediction of the upper layer may be performed by using the intra prediction mode information of the lower layer. In this case, the intra prediction mode of the lower layer may be referred to as a BL intra mode.

Inter-layer motion prediction is also referred to as inter-layer inter prediction, and according to inter-layer motion prediction, prediction of a current block of an upper layer may be performed using motion information of a lower layer. The motion information may include a motion vector and a reference picture index.

In addition, the inter-layer prediction unit 123 may predict the texture of the current block or perform inter-layer syntax prediction by using the syntax information of the lower layer. In this case, the syntax information of the lower layer used for prediction of the current block may be information about an intra prediction mode, motion information, and the like.

As another example of inter-layer prediction, according to the inter-layer difference prediction, prediction for the current block is performed using a difference image generated as a difference value between a reconstructed image of an upper layer and an image obtained by up-sampling a reconstructed image of a lower layer. I can.

As an example of inter-layer prediction, inter-layer texture prediction, inter-layer motion prediction, inter-layer syntax prediction, and inter-layer difference prediction have been described, but inter-layer prediction applicable in the present invention is not limited thereto.

A residual block including residual information that is a difference value from the prediction block generated by the prediction blocks 120 and 125 and the reconstructed block of the prediction block is generated, and the residual block is input to the transform units 130 and 131.

The transform units 130 and 131 may transform the residual block using a transform method such as Discrete Cosine Transform (DST) or Discrete Sine Transform (DST). Whether to apply DCT or DST to transform the residual block may be determined based on intra prediction mode information of the prediction block used to generate the residual block and information on the size of the prediction block. That is, the transform units 130 and 131 may apply different transform methods according to the size of the prediction block and the prediction method.

The quantization units 140 and 141 may quantize values converted into the frequency domain by the transform units 130 and 131. Quantization coefficients may vary depending on the block or the importance of the image. The values calculated by the quantization units 140 and 141 may be provided to the inverse quantization units 170 and 17 and the rearrangement units 150 and 151.

The reordering units 150 and 151 may rearrange coefficient values on the quantized residual values. The reordering units 150 and 151 may change a 2-dimensional block shape coefficient into a 1-dimensional vector shape through a coefficient scanning method. For example, the rearrangement units 150 and 151 may scan from DC coefficients to coefficients in a high frequency region using a Zig-Zag Scan method and change the form to a one-dimensional vector. Depending on the size of the transform block and the prediction mode in the screen, a vertical scan method that scans two-dimensional block shape coefficients in a column direction rather than a zig-zag scan method, and a horizontal scan method that scans two-dimensional block shape coefficients in a row direction Can be used. That is, it is possible to determine which scan method is to be used among zig-zag scan, vertical direction scan, and horizontal direction scan according to the size of the transform block and the intra prediction mode.

The entropy encoding units 160 and 161 may perform entropy encoding based on values calculated by the reordering units 150 and 151. Entropy coding may use various coding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

The entropy encoders 160 and 161 transmit residual coefficient information and block type information, prediction mode information, division unit information, prediction block information, and transmission of the coding block from the reordering units 150 and 151 and the prediction units 120 and 125. Entropy encoding may be performed based on a predetermined encoding method by receiving various information such as unit information, motion vector information, reference frame information, block interpolation information, and filtering information. In addition, the entropy encoders 160 and 161 may entropy-encode the coefficient values of the coding units input from the reordering units 150 and 151.

The entropy encoders 160 and 161 may encode intra prediction mode information of the current block by performing binarization on the intra prediction mode information. The entropy encoders 160 and 161 may include a codeword mapping unit for performing such a binarization operation, and may perform different binarization according to the size of a prediction block performing intra prediction. In the codeword mapping unit, the codeword mapping table may be adaptively generated or stored in advance through a binarization operation. In another embodiment, the entropy encoders 160 and 161 may express information on the prediction mode in the current screen by using a codenum mapping unit performing codenum mapping and a codeword mapping unit performing codeword mapping. In the codenum mapping unit and the codeword mapping unit, a codenum mapping table and a codeword mapping table may be generated or stored.

The inverse quantization units 170 and 171 and the inverse transform units 180 and 181 inverse quantize values quantized by the quantization units 140 and 141 and inverse transform the values transformed by the transform units 130 and 131. The residual values generated by the inverse quantization units 170 and 171 and the inverse transform units 180 and 181 are predicted through a motion estimation unit, a motion compensation unit, and an intra prediction unit included in the prediction units 120 and 125. The prediction block may be combined to generate a reconstructed block.

The filter units 190 and 191 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion caused by the boundary between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on the pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter may be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be processed in parallel when vertical filtering and horizontal filtering are performed.

The offset correction unit may correct an offset from the original image for the deblocking image in pixel units. In order to perform offset correction for a specific picture, the pixels included in the image are divided into a certain number of areas, and then the area to be offset is determined and the offset is applied to the area, or offset by considering the edge information of each pixel. You can use the method of applying.

ALF (Adaptive Loop Filter) may perform filtering based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into at least one group, one filter to be applied to the corresponding group may be determined, and filtering may be performed differentially for each group.

The filter units 190 and 191 may apply only the deblocking filter, only the deblocking filter and ALF, or only the deblocking filter and the offset correction unit without applying all of the deblocking filter, ALF, and offset correction unit.

The memories 195 and 196 may store the reconstructed blocks or pictures calculated through the filter units 190 and 191, and the stored reconstructed blocks or pictures may be provided to the prediction units 120 and 125 when performing inter prediction. have.

Information output from the entropy encoder 100b of the lower layer and information output from the entropy encoder 100a of the upper layer may be multiplexed by the MUX 197 and output as a bitstream.

The MUX 197 may be included in the encoder 100b of the lower layer, or may be implemented as an independent device or module separate from the encoder 100.

2 is a schematic block diagram of a decoding apparatus according to an embodiment of the present invention.

As shown in FIG. 2, the decoding apparatus 200 includes a decoder 200a of an upper layer and a decoder 200b of a lower layer.

The decoding unit 200a of the upper layer includes an entropy decoding unit 210, a rearrangement unit 220, an inverse quantization unit 230, an inverse transform unit 245, a prediction unit 250, a filter unit 260, and a memory 240. ) Can be included.

The decoding unit 200b of the lower layer includes an entropy decoding unit 211, a rearrangement unit 221, an inverse quantization unit 231, an inverse transform unit 241, a prediction unit 251, a filter unit 261, and a memory 271. ) Can be included.

When a bitstream including a plurality of layers is transmitted from the encoding apparatus, the DEMUX 280 may demultiplex the information for each layer and transmit the demultiplexed information to the decoders 200a and 200b for each layer. The input bitstream may be decoded in a procedure opposite to that of the encoding device.

The entropy decoders 210 and 211 may perform entropy decoding in a procedure opposite to that of entropy encoding performed by the entropy encoder of the encoding apparatus. Among the information decoded by the entropy decoding units 210 and 211, information for generating a prediction block is provided to the prediction units 250 and 251, and the residual value obtained by performing entropy decoding in the entropy decoding unit is reordering units 220 and 221. Can be entered as

Like the entropy encoders 160 and 161, the entropy decoders 210 and 211 may perform inverse transform using at least one of a coefficient coding method of CABAC or CAVLC.

The entropy decoders 210 and 211 may decode information related to intra prediction and inter prediction performed by the encoding apparatus. The entropy decoding unit may include a codeword mapping unit to include a codeword mapping table for generating a received codeword as an intra prediction mode number. The codeword mapping table may be stored in advance or may be adaptively generated. When using the code-num mapping table, a code-num mapping unit for performing the code-num mapping may be additionally provided.

The reordering units 220 and 221 may perform reordering based on a method of rearranging the bitstream entropy-decoded by the entropy decoders 210 and 211 in the encoder. The coefficients expressed in the form of a one-dimensional vector can be reconstructed into coefficients in the form of a two-dimensional block and rearranged. The reordering unit may perform reordering through a method of receiving information related to coefficient scanning performed by the encoder and performing reverse scanning based on the scanning order performed by the corresponding encoder.

The inverse quantization units 230 and 231 may perform inverse quantization based on a quantization parameter provided by an encoding apparatus and a coefficient value of a rearranged block.

The inverse transform units 240 and 241 may perform inverse DCT and inverse DST with respect to the DCT and DST performed by the transform units 130 and 131 on the quantization result performed by the encoding apparatus. The inverse transformation may be performed based on a transmission unit determined by the encoding device. In the transform unit of the encoding device, DCT and DST may be selectively performed according to a plurality of information such as a prediction method, the size of the current block, and a prediction direction, and in the inverse transform unit 225 of the decoding device, the transform unit of the encoding device Inverse transformation may be performed based on transformation information. When performing transformation, transformation may be performed based on a coding block rather than a transformation block.

The prediction units 250 and 251 may generate the prediction blocks based on the prediction block generation-related information provided from the entropy decoders 210 and 211 and previously decoded block or picture information provided from the memories 270 and 271. .

The prediction units 250 and 251 may include a prediction unit determining unit, an inter prediction unit, and an intra prediction unit.

The prediction unit determining unit receives various information such as prediction unit information input from the entropy decoder, prediction mode information of the intra prediction method, motion prediction related information of the inter prediction method, and so on, and classifies the prediction block from the current coding block. It is possible to determine whether inter prediction is performed or intra prediction is performed.

The inter-prediction unit uses information necessary for inter-prediction of the current prediction block provided by the encoding device to determine the current prediction block based on information included in at least one picture of a previous picture or a subsequent picture of the current picture including the current prediction block. You can perform prediction between screens. Whether the motion prediction method of the prediction block included in the coding block is a skip mode, merge mode, or AMVP mode based on the coding block to perform inter prediction Can judge.

The intra prediction unit may generate a prediction block based on pixel information in the current picture. When the prediction block is a prediction block that has performed intra prediction, intra prediction may be performed based on intra prediction mode information of the prediction block provided by the encoding device. The intra prediction unit is an MDIS filter that performs filtering on the reference pixel of the current block, a reference pixel interpolation unit that interpolates the reference pixel to generate a reference pixel of an integer value or less, and filters when the prediction mode of the current block is the DC mode. A DC filter that generates a prediction block through may be included.

The prediction unit 250 of the upper layer decoder 200a may further include an inter-layer prediction unit that performs inter-layer prediction for predicting an upper layer using information on a lower layer.

The inter-layer prediction unit uses the texture of the lower layer, intra prediction mode information, motion information, and syntax information to predict inter-layer texture prediction, inter-layer inter prediction, and inter-layer. Inter-layer syntax prediction can be performed.

According to inter-layer texture prediction, prediction using a texture of a reference block in a lower layer as a prediction value of a current block of an upper layer may be performed. The texture of the reference block can be scaled by upsampling.

For inter-layer texture prediction, the intra BL method and the upsampled base layer are used to upsample the reconstructed value of the reference block in the lower layer and use the upsampled reference block as a prediction value for the current block to encode the residual value from the current block. There is a reference index method that stores the data in memory and uses the stored base layer as a reference index.

The intra prediction of the upper layer may be performed using the intra prediction mode information of the lower layer, and in this case, the intra prediction mode of the lower layer may be expressed as a BL intra mode.

According to inter-layer motion prediction, prediction of a current block of an upper layer may be performed using motion information of a lower layer.

In addition, the inter-layer prediction unit may predict the texture of the current block or perform inter-layer syntax prediction to generate the texture of the current block by using the syntax information of the lower layer. In this case, the syntax information of the lower layer used for prediction of the current block may be information about an intra prediction mode, motion information, and the like.

In addition, the inter-layer prediction unit may perform inter-layer differential prediction for predicting a current block by using a difference image generated as a difference value between a reconstructed image of an upper layer and an image obtained by up-sampling a reconstructed image of a lower layer.

As an example of inter-layer prediction, inter-layer texture prediction, inter-layer motion prediction, inter-layer syntax prediction, and inter-layer difference prediction have been described, but inter-layer prediction applicable in the present invention is not limited thereto.

The reconstructed block or picture may be provided to the filter units 260 and 261. The filter units 260 and 261 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter has been applied to a corresponding block or picture, and when a deblocking filter is applied, information on whether a strong filter or a weak filter is applied may be provided from the encoding apparatus. In the deblocking filter of the decoding apparatus, information related to the deblocking filter provided by the encoding apparatus may be provided, and the decoding apparatus may perform deblocking filtering on a corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding and offset value information.

The adaptive loop filter (ALF) may perform filtering based on a value obtained by comparing the reconstructed image and the original image after filtering is performed. The ALF may be applied to a coding unit based on information on whether to apply ALF and information on ALF coefficients provided from the encoding device. Such ALF information may be included in a specific parameter set and provided.

The memories 270 and 271 may store the reconstructed picture or block so that it can be used as a reference picture or a reference block, and output the reconstructed picture.

The encoding device and the decoding device may perform encoding on three or more layers instead of two layers, and in this case, the encoder for the upper layer and the decoder for the upper layer are provided in multiple numbers corresponding to the number of upper layers. Can be.

Hereinafter, in an embodiment of the present invention, for convenience of description, a coding unit is used as a term of a coding block, but it may be a block that performs not only encoding but also decoding. Hereinafter, a method for generating a reference picture list described with reference to the drawings according to an embodiment of the present invention may be implemented according to the functions of each module described above in FIGS. 1 and 2, and such an encoding device and a decoding device are the rights of the present invention. Included in the scope.

As described above, in coding an image supporting a multi-layer structure, a method of encoding/decoding an enhancement layer using motion information and texture information of the base layer, and a coding unit (CU) of the base layer, Alternatively, there is a method of encoding/decoding using an up-sampled base layer as a reference picture without changing a decoding method in units of prediction units (PU). The latter method may be expressed in a reference picture index (RefIdx) mode.

When the reference picture index (RefIdx) mode is used, a picture already decoded in the enhancement layer can be used as a reference picture, or an image of a base layer or a lower layer can be upsampled to be used as a reference picture.

Hereinafter, in the present invention, when an image of a base layer or a lower layer is upsampled and used as a reference picture, the upsampled base layer or lower layer image is expressed as an inter-layer reference picture.

Even in an image that supports a multi-layer structure for multi-views, inter prediction can be performed by using a layer for a view other than the current layer as a reference picture. Layers of other views referenced for encoding and decoding of the corresponding layer are also inter-predictioned. It may be expressed as a layer reference picture.

That is, when constructing a reference picture set of an enhancement layer for coding an image having a multi-layer structure supporting scalability rather than a single layer, an inter-layer reference picture must also be considered.

In general, inter prediction may use at least one of a previous picture or a subsequent picture of the current picture as a reference picture, and prediction on a current block may be performed based on the reference picture. An image used for prediction of the current block is called a reference picture or a reference frame.

The reference picture is specified by a reference picture index (refIdx), and a predetermined region in the reference picture is specified as a reference block through a motion vector.

The area within the reference picture may be indicated using a reference picture index (refIdx) indicating the reference picture, a motion vector, and the like.

Inter prediction can generate a prediction block for the current block by selecting a reference picture and a reference block corresponding to the current block from within the reference picture.

In inter prediction, the encoding device and the decoding device may derive motion information of a current block and then perform inter prediction and/or motion compensation based on the derived motion information. In this case, the encoding apparatus and the decoding apparatus move a collocated block corresponding to the current block within the reconstructed neighboring block and/or the already reconstructed collocated picture. By using the information, it is possible to improve the encoding/decoding efficiency.

Here, the reconstructed neighboring block is a block in the current picture that has already been encoded and/or decoded and reconstructed, and may include a block adjacent to the current block and/or a block located at an outer corner of the current block. In addition, the encoding device and the decoding device may determine a predetermined relative position based on a block existing at a position spatially corresponding to the current block in the collocated picture, and the determined relative position (spatially corresponding to the current block The call block may be derived based on a location inside and/or outside of a block existing at a location where the call is made. Here, as an example, the collocated picture may correspond to one picture from among the reference pictures included in the reference picture list.

In the inter prediction, a prediction block may be generated such that a residual signal with a current block is minimized and a motion vector size is also minimized.

Meanwhile, the motion information derivation method may vary according to the prediction mode of the current block. Prediction modes applied for inter prediction may include Advanced Motion Vector Predictor (AMVP), merge, and the like.

For example, when AMVP (Advanced Motion Vector Predictor) is applied, the encoding device and the decoding device may generate a motion vector candidate list using a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block. That is, a motion vector of a reconstructed neighboring block and/or a motion vector of a collocated block may be used as a motion vector candidate. The encoding apparatus may transmit a predicted motion vector index indicating an optimal motion vector selected from among the motion vector candidates included in the list to the decoding apparatus. In this case, the decoding apparatus may select a predicted motion vector of the current block from among motion vector candidates included in the motion vector candidate list using the motion vector index.

The encoding apparatus may obtain a motion vector difference (MVD) between the motion vector of the current block and the predicted motion vector, encode the motion vector, and transmit it to the decoding apparatus. In this case, the decoding apparatus may decode the received motion vector difference, and may derive a motion vector of the current block through the sum of the decoded motion vector difference and the predicted motion vector.

The encoding apparatus may also transmit a reference picture index indicating a reference picture to the decoding apparatus.

The decoding apparatus may predict a motion vector of a current block using motion information of a neighboring block, and derive a motion vector for the current block using a residual received from the encoding apparatus. The decoding apparatus may generate a prediction block for the current block based on the derived motion vector and reference picture index information received from the encoding apparatus.

As another example, when merge is applied, the encoding apparatus and the decoding apparatus may generate a merge candidate list by using motion information of a reconstructed neighboring block and/or motion information of a collocated block. That is, the encoding apparatus and the decoding apparatus may use the reconstructed neighboring block and/or the collocated block motion information as a merge candidate for the current block when there is motion information.

The encoding apparatus may select a merge candidate capable of providing optimal encoding efficiency among merge candidates included in the merge candidate list as motion information for the current block. In this case, a merge index indicating the selected merge candidate may be included in the bitstream and transmitted to the decoding apparatus. The decoding apparatus may select one of merge candidates included in a merge candidate list by using the transmitted merge index, and determine the selected merge candidate as motion information of the current block. Accordingly, when the merge mode is applied, motion information of a reconstructed neighboring block and/or a collocated block may be used as it is as motion information of the current block. The decoding apparatus may reconstruct the current block by adding the prediction block and the residual transmitted from the encoding apparatus.

In the above-described AMVP and merge modes, motion information of a reconstructed neighboring block and/or motion information of a collocated block may be used to derive motion information of a current block.

In the case of the skip mode, which is one of the other modes used for inter prediction, information of neighboring blocks may be used as it is for the current block. Accordingly, in the case of the skip mode, the encoding apparatus does not transmit syntax information such as residual to the decoding apparatus other than information indicating which block motion information is to be used as motion information of the current block.

The encoding apparatus and the decoding apparatus may generate a prediction block of the current block by performing motion compensation on the current block based on the derived motion information. Here, the prediction block may mean a motion-compensated block generated as a result of performing motion compensation on the current block. In addition, a plurality of motion-compensated blocks may constitute one motion-compensated image.

The decoding apparatus may check the skip flag, merge flag, etc. received from the encoding apparatus for motion information necessary for inter prediction of the current block, such as motion vector and reference picture index, and derive correspondingly thereto.

A processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different. For example, a prediction mode may be determined in units of prediction blocks, and prediction may be performed in units of transform blocks, or a prediction mode may be determined in units of prediction blocks, and intra prediction may be performed in units of transform blocks.

Pictures encoded/decoded before the current picture may be stored in a memory (eg, Decoded Picture Buffer: DPB) and used for prediction of a current block (current picture). The list of pictures available for inter prediction of the current block is maintained as a reference picture list.

The P slice is a slice that is decoded through intra prediction or inter prediction using at most one motion vector and one reference picture. The B slice is a slice that is decoded through intra prediction or inter prediction using up to two motion vectors and two reference pictures. In this case, the reference picture includes a short term reference picture and a long term reference picture. A picture may be specified as a POC (Picture Order Count) indicating a display order, short-term reference pictures may be pictures with a small difference in POC from the current picture, and long-term reference pictures are pictures having a large difference in POC from the current picture. I can.

Reference picture list 0 (reference picture list 0, referred to as'L0' for convenience of description below) is a reference picture list used for inter prediction of a P slice or a B slice. Reference picture list 1 (referred to as'L1' for convenience of description below) is used for inter prediction of B slices. Accordingly, L0 is used for inter prediction of a block of P slice performing unidirectional prediction, and L0 and L1 are used for inter prediction of a block of B slice performing bidirectional prediction.

When decoding a P slice and a B slice through inter prediction, the decoding apparatus constructs a reference picture list. A reference picture used for inter prediction is designated through a reference picture list. The reference picture index is an index indicating a reference picture on the reference picture list.

The reference picture list may be configured based on a reference picture set transmitted from an encoding device. The reference picture set may include a POC of a picture used as a reference picture and a flag (used_by_curr_pic_s0_flag) indicating whether the picture is directly referenced. Reference pictures constituting the reference picture list may be stored in a memory (eg, DPB). Pictures stored in the memory (pictures encoded/decoded before the current picture) are managed by an encoding device and a decoding device.

3 is a conceptual diagram illustrating an example of a reference picture set including a short-term reference picture.

As shown, the reference picture set (RPS) is a POC of a picture for a short-term reference picture to be stored in the DPB at the current time point for each picture, and a flag indicating whether or not a specific picture is directly referenced in the current picture. It can be composed of information.

For example, when inter prediction of a picture with a POC of 21, all three pictures (pictures with a POC of 20, 19, and 16) can be referenced as short-term reference pictures, and all three pictures have a used_by_curr_pic_s0_flag value of 1, so the POC They are all directly used for prediction of pictures with 21.

Meanwhile, when inter prediction of a picture with a POC of 26, all three pictures (POCs of 25, 24, and 20) may be referenced as short-term reference pictures, but the used_by_curr_pic_s0_flag value of a picture with a POC of 25 is a picture with a POC of 24 and 20. Unlike the value of used_by_curr_pic_s0_flag, it is 0. In this case, a picture with a POC of 25 is not directly used for inter prediction of a picture with a POC of 26.

A picture not shown in the reference picture set in the current picture may be removed from the DPB after an indication that it is not used as a reference picture (unused for reference).

4 is a conceptual diagram illustrating a method of inducing a POC of a long-term reference picture.

As shown, in the case of a long-term reference picture, since the difference between the current picture and the POC is large, it can be expressed using the least significant bit (LSB) and the most significant bit (MSB) of the POC.

MaxPocLsb means the maximum value that can be expressed in LSB. When MaxPocLsb is 32, a long-term reference picture (LTRF) with a POC of 84 can be expressed as 32*2+20, where LSB is 20 and MSB is 2.

In the case of the current picture with a POC of 338, it can be expressed as 32*10+11 when the maximum value that can be expressed in LSB is 32, so 10 becomes the MSB value and 11 becomes the LSB value.

delta_poc_msb_cycle_lt is a value for determining DeltaPocMsbCycleLt, the MSB of POC, in the long-term reference picture set of the current picture. DeltaPocMsbCycleLt may correspond to a difference between the MSB of the POC of the current picture and the MSB of the POC of the reference picture.

In the case of a long-term reference picture, the POC of the reference picture may be derived using the LSB value of the POC of the reference picture and the MSB of the POC of the current picture and the POC of the reference picture.

For example, a long-term reference picture (LTRP[0]) with a POC of 331 of the current picture and a POC of 308 indexed with 0 has an LSB value of 20, which is the difference between the MSB of the POC of the current picture and the POC of the reference picture. If you use 1, it can be expressed as 331-1*32-11+20.

Similarly, a long-term reference picture (LTRP[1]) with a POC of 170 indexed by 1 uses an LSB value of 10, and the MSB difference between the MSB of the current picture's POC and the reference picture's POC of 5, 331-5*32- It can be expressed as 11+10.

As shown in FIGS. 3 and 4, an initial reference picture list that can be referenced by a current picture may be constructed with a list of pictures existing in the short-term reference picture buffer and the long-term reference picture buffer.

5 is a diagram illustrating an example of configuring a reference picture list.

The reference pictures are a first short-term reference picture set (RefPicSetStCurr0) composed of reference pictures (Ref 1, Ref 2) smaller than the POC of the current picture (Curr) based on the current picture, and reference pictures larger than the POC of the current picture ( It may be classified into a second short-term reference picture set (RefPicSetStCurr1) composed of Ref 3 and Ref 4) and a long-term reference picture set (RefPicSetLtCurr) composed of long-term reference pictures Ref LT1 and Ref LT2.

In this case, the first short-term reference picture set (RefPicSetStCurr0) is composed of pictures with a used_by_curr_pic_s0_flag value of 1 (delta_poc_s0 with used_by_curr_pic_s0_flag=1), and the second short-term reference picture set (RefPicSetStCurr1 with a value of 1 used_by_curr_pics is also used_pics_flag1). Becomes (delta_poc_s1 with used_by_curr_pic_s1_flag=1).

An initial reference picture list may be constructed from a set of reference picture sets having different properties.

Reference picture list 0, i.e., L0, is composed of a first short-term reference picture set (RefPicSetStCurr0), a second short-term reference picture set (RefPicSetStCurr1), and a long-term reference picture set (RefPicSetLtCurr) in this order, as shown in FIG.

On the other hand, reference picture list 1, that is, L1, is configured in the order of a second short-term reference picture set (RefPicSetStCurr1), a first short-term reference picture set (RefPicSetStCurr0), and a long-term reference picture set (RefPicSetLtCurr).

The number of reference pictures that may be included in the reference picture list may be determined based on information transmitted from the encoding device. For example, after constructing a reference picture list, the encoding apparatus determines the number of reference pictures to be used, and sets information on the number of reference pictures to be used (e.g., num_ref_idx_lX_default_active_minus1, X=0 or 1) as a sequence parameter set (SPS). ), which can be transmitted to the decoding device. The decoding apparatus may use the number of reference pictures specified as a value obtained by adding 1 to the received num_ref_idx_lX_default_active_minus1 as a default value in the current sequence.

In addition, in the case of designating the number of reference pictures for each picture or slice, the encoding apparatus uses separate information indicating the number of reference pictures (e.g., num_ref_idx_l1_active_minus1, X=0 or 1) as a picture parameter set (PPS). ) Or through a slice header. The decoding apparatus may apply a value specified by adding 1 to the received num_ref_idx_l1_active_minus1 as the number of reference pictures for the current picture or the current slice.

When performing inter prediction, motion compensation may be performed using a specified reference picture in the reference picture list configured as described above.

In a multi-layer structure providing spatial scalability or multi-view scalability, a reference picture of an enhancement layer may be composed of reference pictures on the same layer and an inter-layer reference picture.

In this case, signaling for the inter-layer reference picture may be performed through information identifying a layer and information identifying a reference picture. For example, the value of nuh_layer_id, which is the j-th layer identifier (i is greater than j), which exists in the same access unit as the current picture of the i-th layer and is transmitted by being included in the NAL (Network Abstraction Layer) unit header, is the current picture. If it is the same as the RefPiclayerId value for, it may be determined that the corresponding picture is used as a reference picture for the current picture. The inter-layer reference picture can be displayed as a long-term reference picture.

RefPicLayerId is a value that can be signaled as a syntax element inter_layer_pred_layer_idc included in a slice header, and refers to a layer that the current layer refers to for inter-layer prediction.

6 is a conceptual diagram illustrating an example of configuring an inter-layer reference picture list according to an example of the present invention.

6, the initial reference picture list in the multi-layered image is a short-term reference picture set (RefPicSetStCurrBefore[i], hereinafter, a first reference picture set) composed of short-term reference pictures having a smaller POC than that of the current picture. A short-term reference picture set (RefPicSetStCurr After[i], hereinafter, a second reference picture set) consisting of short-term reference pictures having a larger POC than the POC, in the order of a long-term reference picture set (RefPicSetLtCurr[i], hereinafter, a third reference picture set), and It may be composed of an inter-layer reference picture set (RefPicSetILCurr[i], hereinafter, a fourth reference picture set).

The reference picture included in the fourth reference picture set may be a picture of a layer supporting spatial scalability, a picture included in a layer supporting multiview scalability, or depth scalability or quality scalability. It may be a picture of a supported layer. In this case, spatial scalability and quality scalability may be coded with the same codec structure.

When a bitstream supports more than one scalability, a reference picture for each scalability may be integrated into one reference picture set. When one reference picture set is configured for a plurality of scalability, information on the order of scalability types arranged in the reference picture set may be transmitted from the encoding device to the decoding device.

Meanwhile, a reference picture set may be individually configured for each of a plurality of scalability. That is, in addition to the first reference picture set, the second reference picture set, and the third reference picture set, for the current picture, a fourth reference picture set composed of inter-layer reference pictures, a fifth reference picture set, and a sixth reference picture set are generated. Can be. The inter-layer reference picture set may be adaptively added to the reference picture list according to the frequency of occurrence. For example, a reference picture set with a high occurrence frequency may be allocated to a low index of the reference picture list.

The initial reference picture list of FIG. 6 is L0 in which the first reference picture set is configured in the lowest order of the list. In this case, the fourth reference picture set may be added in the last order of the initial reference picture list L0.

The initial reference picture list 0 is composed of pictures in which flag information (used_by_curr_pic_flag) indicating whether or not to be used as a reference picture among reference pictures included in each reference picture set is 1. As illustrated, the initial reference picture list 0 may be configured only with first and second pictures of four pictures included in each reference picture set.

The initial reference picture list may be modified for each picture or slice, and reference pictures may be modified according to the change. The initial reference picture list may be changed by syntax elements (eg, ref_pic_list_modification_flag_l0, list_entry_l0, ref_pic_list_modification_flag_l1, list_entry_l1) included in the slice hair.

ref_pic_list_modification_flag_l0 and ref_pic_list_modification_flag_l1 explicitly indicate whether the reference picture list 0 and reference picture list 1 are transmitted from the encoding device to the decoding device.If the flag value is 1, the reference picture list is explicitly specified using the transmitted reference picture information. If the flag value is 0, the reference picture list is implicitly derived as an initial reference picture set.

When the flag value is 1, list_entry_l0 indicates specific entry information constituting the reference picture list for L0, that is, the index of the reference picture, and list_entry_l1 indicates specific entry information constituting the reference picture list for L1.

Through this change, the final reference picture list 0 is the first and second reference pictures (0, 1) of the first reference picture set, the first and second reference pictures (0, 1) of the second reference picture set, and the second reference picture set. It may consist of the first reference picture (0) of the 4 reference picture set.

According to another example of the present invention, when configuring a reference picture list, the fourth reference picture set for the inter-layer reference picture may be arranged in an order other than the last of the list of FIG. 6. In a multi-layer structure, since there is a strong correlation between the enhancement layer and the base layer due to the characteristics of an image, a case of referring to an inter-layer reference picture can often occur. If an inter-layer reference picture with a high occurrence frequency is added to a position other than the last of the reference picture list, encoding performance of the reference picture list can be improved.

In consideration of the above, the inter-layer reference picture may be added to various positions of the reference picture list. 7 to 10 are conceptual diagrams illustrating an example of configuring an inter-layer reference picture list according to other examples of the present invention.

7 illustrates a configuration of a reference picture list O according to an example of the present invention. As illustrated, a fourth reference picture set, which is an inter-layer reference picture set, may be arranged in the second order of the reference picture list.

The reference picture list O is composed of a first reference picture set, a fourth reference picture set, a second reference picture set, and a third reference picture set in that order.

Reference picture list 0 is composed of pictures in which flag information (used_by_curr_pic_flag) indicating whether to be used as a reference picture is 1 among reference pictures included in each reference picture set. As illustrated, the initial reference picture list 0 may be configured only with first and second pictures of four pictures included in each reference picture set.

The initial reference picture list configured as described above may be modified for each picture or slice, and entries of reference pictures may be modified according to the change.

Through this change, the final reference picture list 0 is the first and second reference pictures (0, 1) of the first reference picture set, the first and second reference pictures (0, 1) of the fourth reference picture set, It may be composed of the first reference picture (0) of the second reference picture set.

8 is a diagram illustrating a configuration of a reference picture list O according to another example of the present invention.

As shown in <a> of FIG. 8, reference picture list O is a first reference picture set composed of short-term reference pictures having a smaller POC than the current picture's POC, and a second reference picture composed of short-term reference pictures having a larger POC than the current picture A set, a fourth reference picture set composed of an inter-layer reference picture, and a third reference picture set composed of a long-term reference picture may be configured in this order.

Alternatively, in the reference picture list O, as shown in <b> of FIG. 8, a fourth reference picture set consisting of inter-layer reference pictures is arranged first, and a first reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, They may be arranged in the order of a second reference picture set composed of short-term reference pictures and a third reference picture set composed of long-term reference pictures having a POC larger than that of the current picture.

In the case of <b>, when the frequency of referencing the inter-layer reference picture is high, there is an advantage of improving the encoding and decoding efficiency of the reference picture list.

Alternatively, according to another embodiment, similar to generating a reference picture list of a single layer, a first reference picture set consisting of short-term reference pictures having a smaller POC than the POC of the current picture, a short-term reference picture having a larger POC than the POC of the current picture After configuring in the order of a second reference picture set composed of and a third reference picture set composed of a long-term reference picture, a position to add an inter-layer reference picture to a slice or sequence level, that is, a slice header or a sequence parameter header may be specified.

A reference picture list can be constructed by adding an inter-layer reference picture from the specified position. Such signaling may be included in a slice header, a sequence header, or a video parameter set, encoded, and transmitted to the decoding apparatus.

9 illustrates a configuration of reference picture list 1 according to an example of the present invention. As shown, a fourth reference picture set, which is an inter-layer reference picture set, may be arranged in the last order of the reference picture list.

Reference picture list 1 is composed of a second reference picture set, a first reference picture set, a third reference picture set, and a fourth reference picture set in this order.

Reference picture list 1 is composed of pictures in which flag information (used_by_curr_pic_flag) indicating whether or not to be used as a reference picture among reference pictures included in each reference picture set is 1. As shown, the initial reference picture list 1 may be configured only with first and second pictures of four pictures included in each reference picture set.

The initial reference picture list configured as described above may be modified for each picture or slice, and entries of reference pictures may be modified according to the change.

Through this change, the final reference picture list 1 is the first reference picture (0) of the second reference picture set, the first and second reference pictures (0, 1) of the first reference picture set, and the fourth reference picture set. It may consist of the first and second reference pictures (0, 1).

10 is a diagram illustrating a configuration of a reference picture list 1 according to another example of the present invention.

As shown in <a> of FIG. 10, reference picture list 1 is a second reference picture set composed of short-term reference pictures having a larger POC than the POC of the current picture, and a first reference picture composed of short-term reference pictures having a smaller POC than the POC of the current picture. A set, a fourth reference picture set composed of an inter-layer reference picture, and a third reference picture set composed of a long-term reference picture may be configured in this order.

Alternatively, in the reference picture list 1, as shown in <b> of FIG. 10, a second reference picture set consisting of short-term reference pictures having a larger POC than that of the current picture is allocated in the first order, and a fourth reference picture consisting of inter-layer reference pictures Sets can be assigned to the second order. Next, reference picture list 1 may be configured in the order of a first reference picture set composed of short-term reference pictures having a smaller POC than that of the current picture and a third reference picture set composed of long-term reference pictures.

Alternatively, similar to generating a single-layer reference picture list, a second reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture, and a first reference picture consisting of short-term reference pictures having a smaller POC than the POC of the current picture. After the third reference picture set consisting of the set and the long-term reference picture is configured in order, a position to add the inter-layer reference picture to the slice or sequence level, that is, a slice header or a sequence parameter header may be specified.

A reference picture list can be constructed by adding an inter-layer reference picture from the specified position. Such signaling may be included in a slice header, a sequence header, or a video parameter set, encoded, and transmitted to the decoding apparatus.

According to another embodiment of the present invention, the sorting order of the inter-layer reference picture set may be set according to the temporal sub-layer index (tempralId) of the current picture.

11 is a diagram illustrating a reference picture list constructed using a temporal sub-layer index according to an example of the present invention.

In a multi-layer structure, the frequency of using an inter-layer reference picture may be different according to a temporal sub-layer of a current picture belonging to an enhancement layer. As tempralId, which is a temporal sub-layer index value, is larger, it may be more effective to perform inter prediction by referring to a picture in the same layer rather than referring to a reference picture of another layer.

In other words, if the tempralId is small, it is more effective for coding to refer to a picture of the inter-layer than for the same layer, and as the tempralId is large, it is more effective to use a picture of the same layer as a reference picture than to use the inter-layer as a reference picture. It can be efficient.

In this embodiment, taking into account that the frequency of reference occurrence of the inter-layer picture varies according to the temporal sub-layer, the position at which the inter-layer reference picture is added to the initial reference picture list can be adaptively changed according to the temporal sub-layer index tempralId. have.

11 is a diagram illustrating a configuration of a reference picture list 0 when a tempralId value is 0 to 2.

When the tempralId value is 0, since inter-layer prediction is highly likely to be performed on the enhancement layer, the fourth reference picture set composed of inter-layer reference pictures may be added to the beginning of the initial reference picture list.

The initial reference picture list consisting of pictures whose flag information (used_by_curr_pic_flag) indicating whether or not to be used as a reference picture among the reference pictures included in each reference picture set is 1 is four pictures included in the fourth reference picture set (0 ,1,2,3), the first reference picture set, the second reference picture set, and the first and second pictures (0,1) included in the third reference picture set.

As shown, after the initial value of the reference picture list is set, the reference picture list may be modified, and then a final reference picture list may be set. Information for modifying the reference picture list may be signaled in the slice header.

The modified final reference picture list 0 is the first and second pictures of the first reference picture set (0,1), the first and second pictures of the fourth reference picture set (0,1), and the second reference picture set. It is organized in order of the first picture (0).

When the tempralId value is 1, the fourth reference picture set composed of the inter-layer reference picture may be added after the first reference picture set in the initial reference picture list 0.

The initial reference picture list consisting of pictures with flag information (used_by_curr_pic_flag) indicating whether to be used as a reference picture is 1 is included in the first reference picture set, the fourth reference picture set, the second reference picture set, and the third reference picture set It consists of the first and second pictures (0,1).

The modified final reference picture list 0 is the first and second pictures of the first reference picture set (0,1), the first and second pictures of the fourth reference picture set (0,1), and the second reference picture set. It is organized in order of the first picture.

When the tempralId value is 2, the possibility of inter-prediction between the same layers is higher than that of the inter-layer prediction of the enhancement layer.Therefore, the fourth reference picture set consisting of inter-layer reference pictures is the third reference picture set in the initial reference picture list. Then it can be added last.

The initial reference picture list 0 consisting of pictures with flag information (used_by_curr_pic_flag) indicating whether to be used as a reference picture is 1 is in the first reference picture set, the second reference picture set, the third reference picture set, and the fourth reference picture set. It consists of the included first and second pictures (0,1).

The modified final reference picture list includes the first and second pictures of the first reference picture set (0,1), the first and second pictures of the second reference picture set (0,1), and the first of the fourth reference picture set. It consists of the first picture (0).

Similar to the above, the position to which the fourth reference picture set is added may be changed in the initial reference list L1 according to the temporalId value. For example, when the temporalId value is 0, the initial reference list L1 may be configured in the order of a fourth reference picture set, a second reference picture set, a first reference picture set, and a third reference picture set, and when the temporalId value is 1 May be configured in the order of a second reference picture set, a fourth reference picture set, a first reference picture set, and a third reference picture set. When the temporalId value is as large as 2, the fourth reference picture set may be added in the last order of the initial reference list L1.

In summary, as illustrated in FIG. 11, the position of adding the inter-layer reference picture to the initial reference picture list can be adaptively changed according to the temporalId value of the current picture.

Meanwhile, when the current picture refers to multiple layers, that is, when the number of inter-layer reference pictures is plural, the plurality of reference pictures may be divided to form a plurality of inter-layer reference picture sets. The plurality of inter-layer reference picture sets may be arranged in a different order with the initial reference picture set.

For example, in an image supporting multi-view scalability, when a plurality of viewpoints, that is, a plurality of layers, are used as reference pictures, a fourth reference picture set consisting of an inter-layer reference picture is divided and included in the reference picture list. I can. The reference picture list may be configured in various ways according to the view order of the layers of the current layer and the reference picture or the degree of proximity of the views.

The smaller the difference between the current layer and the viewpoint, the higher the possibility that the picture of the corresponding layer is used as an inter-layer reference picture. In other words, the likelihood that the current picture refers to a layer having a larger difference between the current layer and the viewpoint decreases. According to the present embodiment, it is possible to adaptively change the order in which reference pictures are added to the reference picture list according to the proximity order taking into account the frequency of occurrence of such reference pictures, that is, the degree of reference between viewpoints. That is, a position to add the inter-layer reference picture to the reference picture list may be adaptively determined according to the number of inter-layer reference pictures and the order of the inter-layer reference pictures.

A low index of the reference picture list may be allocated to a reference picture of a layer having a small difference between the current layer and a viewpoint, and a high index of the reference picture list may be allocated to a reference picture of a layer having a large difference between the current layer and the viewpoint.

12 is a diagram illustrating a reference picture list constructed using a multiview reference picture according to an example of the present invention.

As shown in FIG. 12, among reference picture 0, reference picture 1, reference picture 2, and reference picture 3 that may be included in the multi-view reference picture set, reference picture 0 and reference picture 1 are arranged at the front of the reference picture list 0. Reference picture 2 and reference picture 3 may be aligned after the first reference picture set composed of short-term reference pictures.

In this case, reference picture 0 and reference picture 1 may be reference pictures of a layer having a small difference between a current layer and a viewpoint, and reference picture 2 and a reference picture 3 may be pictures having a large difference between the current layer and the viewpoint.

According to another example, reference picture 0, which is a first inter-layer reference picture, is arranged in the first order of an initial reference picture list, and reference picture 1, which is a second inter-layer reference picture, is a first reference picture set composed of short-term reference pictures. Subsequently, inter-layer reference pictures thereafter may be sequentially added after a long-term reference picture set composed of a long-term reference picture.

A difference value between a view point of a reference layer and a current layer that can be arranged in the front of the reference picture list may be a value determined by default, or may be variably set in the encoding device and transmitted to the decoding device.

In addition, when adding an inter-layer reference picture set to the initial reference picture list, information on a criterion for applying a different position may be determined as a fixed value in the encoding device, or included in the VPS and signaled to the decoding device. have.

In reference picture list 1, reference pictures of a layer with a small difference between the current layer and the viewpoint may be arranged at the front of the initial reference picture list, or after a second reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture. Can be. The reference picture set of reference pictures having a large difference between the current layer and the viewpoint may be arranged at the end of the initial reference picture list or may be arranged in an order before the third reference picture set composed of long-term reference pictures.

Alternatively, a reference picture of a layer having a large difference between the current layer and the viewpoint may be arranged at the front of the initial reference picture list or after the second reference picture set consisting of short-term reference pictures having a larger POC than the POC of the current picture. The reference picture set of reference pictures having a small difference between the current layer and the viewpoint may be arranged at the end of the initial reference picture list or may be arranged in an order before the third reference picture set consisting of long-term reference pictures.

The inter-layer reference picture set of FIG. 12 includes a multi-view reference picture, but even when the type of scalability is spatial scalability and pictures of multiple layers are used as a reference picture, the reference picture shown in FIG. 12 List configuration can be applied. That is, reference pictures may be divided according to a difference between a current picture and a dependency ID (ID) identifying a spatial layer and added to the reference picture list.

As described above, when an inter-layer reference picture set is included in the reference picture list, a position to which the inter-layer reference picture set is added may be variously set, and there may be multiple methods of signaling this.

13 is a diagram for describing a method of generating a reference picture list according to an embodiment of the present invention.

First, the decoding apparatus generates a short-term reference picture set and a long-term reference picture set composed of pictures existing in the same rare as the current picture (S1310).

The short-term reference picture set may include a short-term reference picture set consisting of a short-term reference picture having a smaller POC than the POC of the current picture, and a short-term reference picture set consisting of a short-term reference picture having a larger POC than the POC of the current picture.

The long-term reference picture set is composed of a long-term reference picture having a large difference between the current picture and the POC.

Initially, the initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures with a smaller POC than the current picture's POC, a short-term reference picture set consisting of short-term reference pictures with a larger POC than the current picture's POC, and a long-term reference picture set. Can be aligned.

Initial reference picture list 1 is also primarily a short-term reference picture set consisting of short-term reference pictures with a larger POC than the current picture's POC, a short-term reference picture set consisting of short-term reference pictures with a smaller POC than the current picture, and a long-term reference picture set. Can be aligned.

The decoding apparatus derives location information on a location to which the inter-layer reference picture set is to be added (S1320). Such location information is information signaled from the encoding device to the decoding device, and may be included in a slice or sequence level, such as a slice header or a sequence header, and transmitted.

In the case of a bitstream supporting a plurality of view layers such as multi-view scalability, there are many cases in which the characteristics of the image for each layer are different and the correlation with the reconstructed image of the base layer is different. In addition, the temporalId value for each multi-layer may be different, and the frequency of occurrence of inter-layer reference may be different according to the temporalId value.

Since the most optimal position to add the inter-layer reference picture to each layer may be different according to these various conditions, the position to add the inter-layer reference picture to each multi-layer may be signaled.

When the location information is derived, the decoding apparatus adds an inter-layer reference picture set to a corresponding location in the initial reference picture list indicated by the location information (S1330).

The inter-layer reference picture set may be generated before the location information is derived, or may be generated simultaneously while the inter-layer reference picture is added to the initial reference picture list after the location information is derived.

The initial reference picture list is composed of pictures in which flag information (used_by_curr_pic_flag) indicating whether or not to be used as a reference picture among reference pictures included in the reference picture set is 1.

In this case, the order of the reference picture sets arranged in the initial reference picture list may be variably set according to position information.

For example, the initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures with a smaller POC than the current picture's POC, an inter-layer reference picture set, and a short-term reference picture consisting of a short-term reference picture with a larger POC than the POC of the current picture. It may be configured in order of a set and a long-term reference picture set consisting of a long-term reference picture.

The initial reference picture list 1 is a short-term reference picture set consisting of short-term reference pictures with a larger POC than the current picture's POC, an inter-layer reference picture set, a short-term reference picture set consisting of a short-term reference picture with a smaller POC than the current picture's POC, and a long-term reference. It may be configured in the order of a long-term reference picture set composed of pictures.

Alternatively, when the inter-layer reference picture set includes a plurality of sub-inter-layer reference picture sets, and the sub-reference layer picture set is separated and added to the initial reference picture list, the position information is a position at which each sub-reference layer picture set is added. May include information of.

For example, in the case of a layer supporting multi-view scalability, initial reference picture list 0 is a short-term reference picture set consisting of short-term reference pictures having a smaller POC than that of the current picture, and of the current picture among multi-view reference layer pictures. A sub-interlayer reference picture set consisting of pictures with a view order smaller than the viewpoint, a short-term reference picture set consisting of a short-term reference picture with a larger POC than the current picture's POC, a long-term reference picture set consisting of a long-term reference picture, a multi-view reference layer picture Among them, the sub-interlayer reference picture set may be configured in order of a picture having a view order greater than that of the current picture.

The initial reference picture list 1 is a short-term reference picture set consisting of short-term reference pictures having a greater POC than the current picture's POC, a sub-interlayer reference picture set consisting of pictures having a view order greater than the view of the current picture among multi-view reference layer pictures, A short-term reference picture set consisting of short-term reference pictures with a smaller POC than the current picture's POC, a long-term reference picture set consisting of long-term reference pictures, and a sub-inter consisting of pictures having a view order smaller than that of the current picture among multi-view reference layer pictures. It may be configured in the order of layer reference picture sets.

Alternatively, the inter-layer reference picture set may be generated based on a temporal sub-layer index (tempralId) of the current picture or may be generated based on the number of maximum temporal sub-layers allowing inter-layer prediction.

For example, as the temporal sub-layer index (tempralId) of the current picture is smaller, the inter-layer reference picture set may be arranged in a lower order of the initial reference picture set. In this case, a specific inter-layer reference picture set is added. Location can be signaled.

Alternatively, as the difference between the tempral sub-layer index tempralId of the current picture and the temporal sub-layer index of the reference layer is smaller, the inter-layer reference picture set may be arranged in the lower order of the initial reference picture set. A specific position to which a picture set is added may be signaled.

When the final reference picture list is generated through modification of the reference picture list, the decoding apparatus performs prediction on the current picture based on the final reference picture list, or based on the initial reference picture list if the initial reference picture list is not modified, and The current picture is decoded (S1340).

When predicting the current picture, a reference picture included in the inter-layer reference picture set is displayed as a long-term reference picture, and the decoding apparatus uses the reference picture included in the inter-layer reference picture set as a reference picture of the current picture.

Meanwhile, in order to signal the position to which the inter-layer reference picture is added, the encoding apparatus calculates a bit distortion factor that considers image quality versus the number of bits at each position where the inter-layer reference picture can be added, and based on the calculation result. Thus, a position where an inter-layer reference picture can be added must be determined. In this case, the amount of encoding operation to be performed by the encoding device may increase.

In order to reduce the amount of computation in the encoding step while maintaining the diversity of images for each multilayer, the encoding apparatus limits the positions to which reference pictures are to be added to a predetermined number, and sets the positions to which the inter-layer reference pictures can be added by the number. After that, the bit distortion rate can be calculated only for the set position. An optimal position to which an inter-layer reference picture can be added may be derived based on the calculated bit distortion rate, and the encoding apparatus may transmit this position to the decoding apparatus.

For example, it may be set that the inter-layer reference picture can be added at three positions as follows: the first of the initial reference picture list, after the short reference picture set, and then the long reference picture set. In this case, the encoding apparatus may encode and signal an identification index of a position to which an inter-layer reference picture is added among three positions.

The position to which the inter-layer reference picture is added and the index for identifying the position may be mapped as shown in Table 1.

Referring to Table 1, when an inter-layer reference picture is added to the beginning of the reference picture list, the index is 0, when it is added after the short-term reference picture set, the index is 1, and the long-term reference picture set is added after the index. In this case, the index can be set to 2.

A location to add an inter-layer reference picture may be different for each layer or for each slice, and in this case, the identification index may be signaled for each layer or for each slice.

In addition, an identification flag indicating whether an identification index indicating a position to which an inter-layer reference picture is added is transmitted may be included in a video parameter set of each layer and signaled.

14 is a diagram illustrating a method of generating a reference picture list according to another embodiment of the present invention.

First, the decoding apparatus receives an identification flag indicating whether an identification index indicating a position at which an inter-layer reference picture for each layer is added to the initial reference picture list is transmitted, and whether an identification index is transmitted based on the received identification flag. To grasp (S1410)

This identification flag may be included in the video parameter set of each layer and signaled.

If the identification flag is 1 (S1420), the decoding apparatus receives and derives the identification index to derive a position where the inter-layer reference picture is added to the initial reference picture list for each layer (S1430).

A location to which an inter-layer reference picture is added may be mapped to an identification index as shown in Table 1, and the decoding apparatus may derive a location to which an inter-layer reference picture is added using a lookup table shown in Table 1.

On the other hand, if the identification flag is 0 (S1420), the decoding apparatus may determine position information on a position to which the inter-layer reference picture set is added from a preset reference picture list (S1440). Thereafter, the decoding apparatus configures an initial reference picture list by adding an inter-layer reference picture set to a corresponding position in the initial reference picture list indicated by the position information preset as a default value (S1450).

The decoding apparatus may predict a current picture based on an initial reference picture list and perform decoding on the current picture (S1460).

Steps S1450 and S146 Since they are the same as S1330 and S1330 of FIG. 13, a duplicate description will be omitted.

According to the present invention, flag information indicating whether the inter-layer reference picture set is divided into at least two sub-reference layer picture sets and added to the reference picture list may be signaled. That is, flag information for easily knowing whether the inter-layer reference picture set is divided into two or more sub-reference layer picture sets may be encoded in the encoding device and transmitted to the decoding device.

As described above, if the reference picture for the multi-view layer is divided into a plurality of sub-reference layer picture sets, the flag information is 1.

If the flag information is 1, information on the number of sub-reference layer picture sets from which the inter-layer reference picture set is separated may be additionally signaled.

In addition, when the flag information is 1, the above-described identification flag and identification index may be signaled.

As described above, according to the present invention, a method for generating a reference picture list including pictures of different layers for a layer to be currently encoded and decoded and an apparatus using the same are provided.

The inter-layer reference picture set included in the reference picture list depends on the characteristics of the picture set, for example, according to the temporal sub-layer index value of the current picture, according to the difference in the temporal sub-layer index value between layers, or the order of views. It may be adaptively added to the reference picture list according to the following.

A method and apparatus for signaling position information on a position at which an inter-layer reference picture is added to a reference picture list are also included in the scope of the present invention. The location information may be individually coded and signaled, or may be signaled as index information indicating a preset limited location.

In the exemplary system described above, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of the steps, and certain steps may occur in a different order or concurrently with the steps described above. I can. In addition, since the above-described embodiments may include examples of various aspects, a combination of each embodiment should also be understood as an embodiment of the present invention. Accordingly, the present invention will be said to cover all other replacements, modifications and changes falling within the scope of the following claims.

100: encoding device 200: decoding device
100a: encoder of an upper layer 100b: encoder of a lower layer
200a: decoding unit of the upper layer 200b: decoding unit of the lower layer

Claims (34)

In a video decoding method supporting a plurality of layers,
Decoding a corresponding picture on a reference layer for the current picture of the current layer to restore an inter-layer reference picture;
Generating an initial reference picture list including an inter-layer reference picture set including the reconstructed inter-layer reference picture;
Generating a final reference picture list by modifying the initial reference picture list;
Comprising the step of performing prediction on the current picture based on the final reference picture list,
The initial reference picture list is generated by dividing the inter-layer reference picture set into a first sub-reference layer picture set and a second sub-reference layer picture set,
The initial reference picture list includes an initial reference picture list 0 and an initial reference picture list 1,
In the initial reference picture list 0, the first sub-reference layer picture set includes a first short-term reference picture set consisting of short-term reference pictures having a smaller POC than a picture of count (POC) of the current picture, and a POC of the current picture. POC is arranged between a second short-term reference picture set composed of a large short-term reference picture, and the second sub-reference layer picture set is arranged after a long-term reference picture set,
The first short-term reference picture set, the second short-term reference picture set, and the long-term reference picture set correspond to the same layer as the current layer, and the first and second sub-reference layer picture sets correspond to a different layer from the current layer. Become,
The step of generating the inter-layer reference picture set is characterized in that the generation is based on a temporal sub-layer index (tempralId) of the current picture. delete delete delete delete delete delete delete delete The method of claim 1,
The inter-layer reference picture set is composed of a multiview reference layer picture,
The initial reference picture list includes an initial reference picture list 0,
The initial reference picture list 0 is
A first short-term reference picture set consisting of a short-term reference picture having a smaller POC than that of the current picture,
The first sub-reference layer picture set consisting of pictures having a view order smaller than that of the current picture among the multi-view reference layer pictures,
A second short-term reference picture set consisting of short-term reference pictures having a larger POC than that of the current picture,
Long-term reference picture set consisting of long-term reference pictures,
And the second sub-reference layer picture set, which is composed of pictures having a view order greater than that of the current picture, among the multi-view reference layer pictures. The method of claim 1,
The inter-layer reference picture set is composed of a multiview reference layer picture,
The initial reference picture list includes an initial reference picture list 1,
The initial reference picture list 1 is
A second short-term reference picture set consisting of short-term reference pictures having a larger POC than that of the current picture,
The second sub-reference layer picture set consisting of pictures having a view order greater than that of the current picture among the multi-view reference layer pictures,
A first short-term reference picture set consisting of a short-term reference picture having a smaller POC than that of the current picture,
Long-term reference picture set consisting of long-term reference pictures,
And the first sub-reference layer picture set, which is composed of pictures having a view order smaller than that of the current picture, among the multi-view reference layer pictures. delete delete delete The method of claim 1,
The generating of the inter-layer reference picture set comprises generating based on the number of maximum temporal sub-layers allowing inter-layer prediction. The method of claim 1,
And generating a final reference picture list by modifying the initial reference picture list. The method of claim 1,
In the performing of prediction on the current picture, a reference picture included in the inter-layer reference picture set is used as a reference picture of the current picture. In the video decoding apparatus supporting a plurality of layers,
An entropy decoding unit for decoding information for prediction and decoding of an image received through Beast Stream;
The corresponding picture on the reference layer for the current picture of the current layer is decoded to be restored as an inter-layer reference picture, and an initial reference picture list including an inter-layer reference picture set including the reconstructed inter-layer reference picture is generated, and the A prediction unit for generating a final reference picture list by modifying an initial reference picture list, and performing prediction on the current picture based on the final reference picture list,
The initial reference picture list is generated by dividing the inter-layer reference picture set into a first sub-reference layer picture set and a second sub-reference layer picture set,
The initial reference picture list includes an initial reference picture list 0 and an initial reference picture list 1,
In the initial reference picture list 0, the first sub-reference layer picture set includes a first short-term reference picture set consisting of short-term reference pictures having a smaller POC than a picture of count (POC) of the current picture, and a POC of the current picture. POC is arranged between a second short-term reference picture set composed of a large short-term reference picture, and the second sub-reference layer picture set is arranged after a long-term reference picture set,
The first short-term reference picture set, the second short-term reference picture set, and the long-term reference picture set correspond to the same layer as the current layer, and the first and second sub-reference layer picture sets correspond to a different layer from the current layer. Become,
And the prediction unit generates the initial reference picture list based on a temporal sub-layer index (tempralId) of the current picture. delete delete delete delete delete delete delete delete The method of claim 18,
The inter-layer reference picture set is composed of a multiview reference layer picture,
The initial reference picture list includes an initial reference picture list 0,
The initial reference picture list 0 is
A first short-term reference picture set consisting of a short-term reference picture having a smaller POC than that of the current picture,
The first sub-reference layer picture set consisting of pictures having a view order smaller than that of the current picture among the multi-view reference layer pictures,
A second short-term reference picture set consisting of short-term reference pictures having a larger POC than that of the current picture,
Long-term reference picture set consisting of long-term reference pictures,
And the second sub-reference layer picture set in order of a picture having a view order greater than that of the current picture among the multi-view reference layer pictures. The method of claim 18,
The inter-layer reference picture set is composed of a multiview reference layer picture,
The initial reference picture list includes an initial reference picture list 1,
The initial reference picture list 1 is
A second short-term reference picture set consisting of short-term reference pictures having a larger POC than that of the current picture,
The second sub-reference layer picture set consisting of pictures having a view order greater than that of the current picture among the multi-view reference layer pictures,
A first short-term reference picture set consisting of a short-term reference picture having a smaller POC than that of the current picture,
Long-term reference picture set consisting of long-term reference pictures,
And the first sub-reference layer picture set consisting of pictures having a view order smaller than that of the current picture among the multi-view reference layer pictures. delete delete delete The method of claim 18,
And the prediction unit generates the initial reference picture list based on the number of maximum temporal sublayers allowing inter-layer prediction. The method of claim 18,
And the prediction unit modifies the initial reference picture list to generate a final reference picture list. The method of claim 18,
And the prediction unit uses a reference picture included in the inter-layer reference picture set as a reference picture of the current picture.

Images