KR20220116107A - Method and apparatus for encoding/decoding a video signal - Google Patents

Abstract

A video signal decoding method according to the present invention may include the steps of: generating a weight prediction parameter set for a current block based on weight prediction parameters of neighboring blocks neighboring the current block; determining a weight prediction parameter for the current block based on the weight prediction parameter set and index information; and performing inter prediction on the current block using the weight prediction parameter of the current block.

Description

Video signal encoding/decoding method and apparatus

The present invention relates to a method and apparatus for encoding/decoding a video signal.

Recently, the demand for multimedia data such as moving pictures is rapidly increasing on the Internet. However, it is difficult to keep up with the rapidly increasing amount of multimedia data at the rate of development of the bandwidth of the channel. Accordingly, in February 2014, ITU-T's Video Coding Expert Group (VCEG) and ISO/IEC's MPEG (Moving Picture Expert Group) adopted HEVC (High Efficiency Video Coding) version 1, a video compression standard, in February 2014. enacted.

HEVC defines techniques such as intra prediction, inter prediction, transform, quantization, entropy encoding, and in-loop filter. Among them, inter prediction refers to performing prediction using previously reconstructed images and motion information such as a motion vector, a reference picture index, and an inter prediction indicator. it means.

In the prediction between screens, the higher the correlation between images, the higher the prediction efficiency can be obtained. However, if there is a change in brightness between images, such as fade-in or fade-out, and the correlation between images is lowered, there is a concern that the prediction result between the screens may be inaccurate. In order to solve such a problem, in the present invention, weight prediction is proposed. Here, the weighted prediction may mean that when there is a brightness change between images, the weight is estimated by the degree of the brightness change, and the estimated weight is applied to the inter prediction.

A main object of the present invention is to improve inter prediction efficiency by performing inter prediction using weights in encoding/decoding an image.

The present invention provides an apparatus and method for effectively performing inter-screen prediction even when a plurality of light sources exist in an image or a change in brightness exists only in a local area by selectively using weights in encoding/decoding an image Its main purpose is to provide

A video signal decoding method and apparatus according to the present invention generates a weighted prediction parameter set for the current block based on a weighted prediction parameter of a neighboring block adjacent to the current block, and adds the weighted prediction parameter set and index information to the weighted prediction parameter set and index information. Based on the determination of a weighted prediction parameter for the current block, inter prediction may be performed on the current block by using the weighted prediction parameter of the current block.

In the video signal decoding method and apparatus according to the present invention, the weighted prediction parameter set includes at least one of a forward weighted prediction parameter of the neighboring block and a backward weighted prediction parameter of the neighboring block according to the prediction direction of the current block. can be created on the basis of

In the video signal decoding method and apparatus according to the present invention, when the prediction direction of the current block is bidirectional, the weighted prediction parameter set includes both the forward weighted prediction parameter of the neighboring block and the backward weighted prediction parameter of the neighboring block. may include

In the method and apparatus for decoding an image signal according to the present invention, when the number of weighted prediction parameters obtainable from the neighboring block is smaller than the maximum number of weighted prediction parameters that the weighted prediction parameter set can include, the weighted prediction The parameter set may further include a combined weighted prediction parameter generated by combining the weighted prediction parameters of the neighboring blocks.

In the method and apparatus for decoding an image signal according to the present invention, a plurality of weight prediction parameters included in the weight prediction parameter set may be arranged in a predefined order.

A video signal encoding method and apparatus according to the present invention generates a weighted prediction parameter set for the current block based on a weighted prediction parameter of a neighboring block adjacent to the current block, and based on the weighted prediction parameter set, A weight prediction parameter for the current block may be determined, and index information specifying the determined weight prediction parameter may be encoded.

In the video signal encoding method and apparatus according to the present invention, the weighted prediction parameter set includes at least one of a forward weighted prediction parameter of the neighboring block and a backward weighted prediction parameter of the neighboring block according to the prediction direction of the current block. can be created on the basis of

In the video signal encoding method and apparatus according to the present invention, when the prediction direction of the current block is bidirectional, the weighted prediction parameter set includes both the forward weighted prediction parameter of the neighboring block and the backward weighted prediction parameter of the neighboring block. may include

In the video signal encoding method and apparatus according to the present invention, when the number of weighted prediction parameters obtainable from the neighboring block is smaller than the number of weighted prediction parameters that the weighted prediction parameter set can include, the weighted prediction The parameter set may further include a combined weighted prediction parameter generated by combining the weighted prediction parameters of the neighboring blocks.

In the video signal encoding method and apparatus according to the present invention, a plurality of weight prediction parameters included in the weight prediction parameter set may be arranged in a predefined order.

It is determined whether there is a change in brightness between the current image including the current block and the reference image of the current image, and when it is determined that there is a change in brightness between the current image and the reference image, weight prediction parameters for the current block Determine a candidate, determine a weighted prediction parameter for the current block based on index information specifying any one of the weighted prediction parameter candidates, and perform prediction on the current block based on the weighted prediction parameter can do.

In the method and apparatus for decoding an image signal according to the present invention, the weighted prediction parameter candidate may include a first weighted prediction parameter for the reference image.

In the video signal decoding method and apparatus according to the present invention, when at least one region from which a second weight prediction parameter can be derived is included in the current image, the weight prediction parameter candidates include at least one second weight It may further include a prediction parameter.

In the method and apparatus for decoding an image signal according to the present invention, the first weighted prediction parameter may be derived based on a prediction value of the first weighted prediction parameter and a residual value of the first weighted prediction parameter.

In the method and apparatus for decoding an image signal according to the present invention, the predicted value for the first weighted prediction parameter may be determined according to the precision of the current block.

In the method and apparatus for decoding an image signal according to the present invention, the maximum number of weight prediction parameter candidates may be adaptively determined according to the size of the current block.

In the method and apparatus for decoding an image signal according to the present invention, the weight prediction parameter candidate may include an initial weight prediction parameter having a preset weight value.

A video signal encoding method and apparatus according to the present invention determines whether there is a change in brightness between a current image including a current block and a reference image of the current image, and there is a change in brightness between the current image and the reference image If it is determined that a weight prediction parameter candidate for the current block is determined, a weight prediction parameter for the current block is determined from among the weight prediction parameter candidates, and index information specifying the determined weight prediction parameter is encoded, and Prediction may be performed on the current block based on the weighted prediction parameter.

In the video signal encoding method and apparatus according to the present invention, the weighted prediction parameter candidate may include a first weighted prediction parameter for the reference image.

In the video signal encoding method and apparatus according to the present invention, when at least one region from which a second weight prediction parameter can be derived is included in the current image, the weight prediction parameter candidates include at least one second weight It may further include a prediction parameter.

The video signal encoding method and apparatus according to the present invention may encode a difference value indicating a difference between the first weighted prediction parameter and a predicted value for the first weighted prediction parameter.

In the video signal encoding method and apparatus according to the present invention, the prediction value for the first weighted prediction parameter may be determined according to the precision of the current block.

In the video signal encoding method and apparatus according to the present invention, the maximum number of weight prediction parameter candidates may be adaptively determined according to the size of the current block.

In the video signal encoding method and apparatus according to the present invention, the weight prediction parameter candidate may include an initial weight prediction parameter having a preset weight value.

The present invention can improve inter prediction efficiency by performing inter prediction using weights when encoding/decoding an image.

In the present invention, by selectively using weights in encoding/decoding an image, even when a plurality of light sources exist in an image or a change in brightness exists only in a local area, inter prediction can be effectively performed.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.
2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present invention.
3 is a block diagram schematically illustrating a motion estimation method according to an embodiment of the present invention.
4 is an example showing the positions of neighboring blocks from which motion information to be applied to a block to be currently encoded according to an embodiment of the present invention.
5 is a diagram illustrating a change in brightness between a current image including a current block and a reference image.
6 is a flowchart illustrating a method of using a weight prediction parameter in an encoding apparatus.
7 is a diagram illustrating weight prediction parameters of neighboring blocks adjacent to the current block.
8 is a diagram illustrating an example in which only some pixels of a current block are used for weight prediction parameter estimation.
9 shows an example in which a combined weight prediction parameter is generated.
10 illustrates an example of decoding a weight prediction parameter in a decoding apparatus.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.

Referring to FIG. 1 , the image encoding apparatus 100 includes an image segmentation unit 110 , prediction units 120 and 125 , a transform unit 130 , a quantization unit 135 , a rearrangement unit 160 , and an entropy encoding unit ( 165 ), an inverse quantization unit 140 , an inverse transform unit 145 , a filter unit 150 , and a memory 155 .

Each of the constituent units shown in FIG. 1 is independently illustrated to represent different characteristic functions in the image encoding apparatus, and does not mean that each constituent unit is composed of separate hardware or one software constituent unit. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components Integrated embodiments and separate embodiments of the components are also included in the scope of the present invention without departing from the essence of the present invention.

In addition, some of the components are not essential components for performing essential functions in the present invention, but may be optional components for merely improving performance. The present invention can be implemented by including only essential components to implement the essence of the present invention except for components used for improving performance, and only having a structure including essential components excluding optional components used for improving performance Also included in the scope of the present invention.

The image dividing unit 110 may divide the input image into at least one block. In this case, the block may mean a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). The division may be performed based on at least one of a quadtree and a binary tree. A quad tree is a method in which the upper block is divided into lower blocks whose width and height are half that of the upper block. Binary tree is a method in which the upper block is divided into lower blocks whose either width or height is half that of the upper block. Through the aforementioned quad-tree or binary tree-based partitioning, a block may have a non-square shape as well as a square shape.

Hereinafter, in an embodiment of the present invention, a coding unit may be used as a unit for performing encoding or may be used as a meaning for a unit for performing decoding.

The prediction units 120 and 125 may include an inter prediction unit 120 performing inter prediction and an intra prediction unit 125 performing intra prediction. Whether to use inter prediction or to perform intra prediction for a prediction unit may be determined, and specific information (eg, intra prediction mode, motion vector, reference image, etc.) according to each prediction method may be determined. In this case, a processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different. For example, a prediction method and a prediction mode may be determined in a prediction unit, and prediction may be performed in a transformation unit.

A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130 . Also, prediction mode information, motion vector information, etc. used for prediction may be encoded by the entropy encoder 165 together with a residual value and transmitted to a decoder. When a specific encoding mode is used, the original block may be encoded and transmitted to the decoder without generating the prediction block through the predictors 120 and 125 .

The inter prediction unit 120 may predict a prediction unit based on information on at least one of a previous image or a subsequent image of the current image, and in some cases, based on information of a partial region in the current image that has been encoded. A prediction unit may be predicted. The inter prediction unit 120 may include a reference image interpolator, a motion information generator, and a motion compensator.

The reference image interpolator may receive reference image information from the memory 155 and generate pixel information of integer pixels or less from the reference image. In the case of luminance pixels, a DCT-based 8-tap interpolation filter in which filter coefficients are different to generate pixel information of integer pixels or less in units of 1/4 pixels may be used. In the case of the color difference signal, a DCT-based 4-tap interpolation filter in which filter coefficients are different to generate pixel information of integer pixels or less in units of 1/8 pixels may be used.

The motion information generator may generate motion information based on the reference image interpolated by the reference image interpolator. Here, the motion information means a motion vector, a reference image index, a prediction direction, and the like. As a method for estimating the motion vector, various methods such as Full search-based Block Matching Algorithm (FBMA), Three Step Search (TSS), and New Three-Step Search Algorithm (NTS) may be used. Also, the motion vector may have a motion vector value in units of 1/2 or 1/4 pixels based on the interpolated pixels. In inter prediction, the current prediction unit may be predicted by using a different method of generating motion information. As a method of generating motion information, various methods such as a merge method using a motion vector of a neighboring block and a motion estimation method (eg, Adaptive Motion Vector Prediction (AMVP)) may be used.

As an example, FIG. 3 is a diagram illustrating an example of generating motion information through motion estimation. Motion estimation is to determine a motion vector, a reference picture index, and an inter prediction direction of the current block according to the determination when a reference block identical to or similar to a prediction block in a reference picture that has already been encoded and decoded is determined.

When the AMVP method is used, the encoding apparatus predicts a motion vector estimated in the current block to generate a motion vector prediction (MVP), and a difference value (MVD) between the motion vector and the generated prediction motion vector: Motion Vector Difference) can be encoded.

A method of using a motion vector of a neighboring block is to apply motion information of a neighboring block adjacent to the current block to the current block. In this case, the neighboring block may include a spatial neighboring block adjacent to the current block and a temporal neighboring block existing at the same position as the current block included in the reference image. As an example, FIG. 4 illustrates a neighboring block of the current block. The encoding apparatus may determine the motion information of the current block by applying motion information of neighboring blocks (spatial neighboring blocks: A to E , temporal neighboring blocks: Col) of the current block shown in FIG. 4 to the current block. Here, Col means a block at the same or similar position as the current block existing in the reference image.

The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current image. If the neighboring block of the current prediction unit is a block on which inter prediction is performed, and the reference pixel is a pixel restored by performing inter prediction, the reference pixel included in the block on which inter prediction is performed is used for the neighboring intra prediction. It can be used by replacing it with reference pixel information of the executed block. That is, when the reference pixel is not available, the unavailable reference pixel information may be replaced with at least one reference pixel among the available reference pixels.

In intra prediction, the prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional mode in which directional information is not used when prediction is performed. A mode for predicting luminance information and a mode for predicting chrominance information may be different, and in-screen prediction mode information or predicted luminance signal information used to predict luminance information may be utilized to predict chrominance information. .

In the intra prediction method, a prediction block may be generated after applying an adaptive intra smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of the AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. When the intra prediction mode of the current prediction unit is predicted by using the intra prediction mode information predicted from the neighboring prediction unit, if the intra prediction mode of the current prediction unit and the neighboring prediction unit are the same, the current prediction unit using predetermined flag information Information that the prediction mode of the prediction unit and the neighboring prediction unit is the same may be transmitted, and if the prediction modes of the current prediction unit and the neighboring prediction unit are different, entropy encoding may be performed to encode the prediction mode information of the current block.

In addition, a residual block including residual information that is a difference value from the original block of the prediction unit and the prediction unit in which prediction is performed based on the prediction unit generated by the prediction units 120 and 125 may be generated. The generated residual block may be input to the transform unit 130 .

The transform unit 130 may transform the residual block including the residual data using a transform method such as DCT, DST, or Karhunen Loeve Transform (KLT). In this case, the transformation method may be determined based on the intra prediction mode of the prediction unit used to generate the residual block. For example, depending on the intra prediction mode, DCT may be used in a horizontal direction and DST may be used in a vertical direction.

The quantization unit 135 may quantize values transformed in the frequency domain by the transform unit 130 . The quantization coefficient may vary according to blocks or the importance of an image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160 .

The transform unit 130 and/or the quantizer 135 may be selectively included in the image encoding apparatus 100 . That is, the image encoding apparatus 100 may encode the residual block by performing at least one of transform and quantization on the residual data of the residual block, or skipping both transform and quantization. Even if either transform or quantization is not performed in the image encoding apparatus 100 or neither transform nor quantization is performed, a block entering the input of the entropy encoder 165 is generally referred to as a transform block.

The reordering unit 160 may rearrange the coefficient values on the quantized residual values.

The rearranging unit 160 may change the two-dimensional block form coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the reordering unit 160 may scan from DC coefficients to coefficients in a high frequency region using a predetermined scan type and change the scan type into a one-dimensional vector form.

The entropy encoding unit 165 may perform entropy encoding based on the values calculated by the reordering unit 160 . For entropy encoding, various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used.

The entropy encoding unit 165 receives the residual value coefficient information and block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, and motion of the coding unit from the reordering unit 160 and the prediction units 120 and 125 . Various information such as vector information, reference image information, block interpolation information, and filtering information may be encoded. In the entropy encoding unit 165 , the coefficients of the transform block include a flag indicating whether 0 in units of partial blocks within the transform block, a flag indicating whether the absolute value of the coefficient is greater than 1, and whether the absolute value of the coefficient is greater than 2 A flag indicating , etc. may be encoded. The entropy encoding unit 165 encodes the sign of the coefficient only for non-zero coefficients. And, for a coefficient whose absolute value is greater than 2, the remaining value obtained by subtracting 2 from the absolute value is encoded.

The entropy encoder 165 may entropy-encode the coefficient values of the coding units input from the reordering unit 160 .

The inverse quantizer 140 and the inverse transform unit 145 inversely quantize the values quantized by the quantizer 135 and inversely transform the values transformed by the transform unit 130 . The residual value generated by the inverse quantizer 140 and the inverse transform unit 145 may be combined with a prediction block generated for each prediction unit through the prediction units 120 and 125 to generate a reconstructed block.

The filter unit 150 may include at least one of a deblocking filter, an offset correcting unit, and an adaptive loop filter (ALF).

The deblocking filter may remove block distortion caused by the boundary between blocks in the reconstructed image. In order to determine whether to perform the deblocking filter, it may be determined whether to apply the deblocking filter to the current block based on pixels included in several columns or rows included in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter can be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be concurrently processed when performing vertical filtering and horizontal filtering.

The offset correcting unit may correct an offset from the original image in units of pixels with respect to the image on which the deblocking has been performed. In order to perform offset correction on a specific image, the method of dividing pixels included in an image into arbitrary regions, determining the region to be offset and applying the offset to the region, or calculating the offset by considering the edge information of each pixel application method can be used.

Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the corresponding group is determined to perform differential filtering for each group. Information related to whether to apply ALF may be transmitted for each coding unit (CU) in the case of a luminance signal, and the shape and filter coefficients of the ALF filter to be applied may vary according to each block. In addition, the ALF filter of the same type (fixed type) may be applied regardless of the characteristics of the application target block.

The memory 155 may store the reconstructed block or image calculated through the filter unit 150 , and the stored reconstructed block or image may be provided to the predictors 120 and 125 when inter prediction is performed.

2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present invention.

2, the image decoder 200 includes an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, and a filter unit ( 240) and a memory 245 may be included.

When an image bitstream is input from an image encoder, the input bitstream may be decoded by a procedure opposite to that of the image encoder.

The entropy decoding unit 210 may perform entropy decoding in a procedure opposite to that performed by the entropy encoding unit of the image encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied corresponding to the method performed by the image encoder. In the entropy decoding unit 210, the coefficients of the transform block include a flag indicating whether 0 in units of partial blocks within the transform block, a flag indicating whether the absolute value of the coefficient is greater than 1, and whether the absolute value of the coefficient is greater than 2 A flag indicating , etc. may be decoded. Then, the entropy decoding unit 210 decodes a sign of a non-zero coefficient. For coefficients having an absolute value greater than 2, the remaining values obtained by subtracting 2 may be decoded.

The entropy decoder 210 may decode information related to intra prediction and inter prediction performed by the encoder.

The reordering unit 215 may perform reordering based on a method of rearranging the entropy-decoded bitstream by the entropy decoding unit 210 by the encoder. Coefficients expressed in a one-dimensional vector form may be restored and rearranged in a two-dimensional block form. The reordering unit 215 may receive information related to coefficient scanning performed by the encoder and perform reordering by performing a reverse scanning method based on the scanning order performed by the corresponding encoder.

The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the reordered coefficient values of the blocks.

The inverse transform unit 225 may perform inverse transform on the inverse quantized transform coefficient using a predetermined transform method. In this case, the transformation method may be determined based on information on a prediction method (inter-picture/intra-picture prediction), a size/shape of a block, an intra prediction mode, and the like.

The prediction units 230 and 235 may generate a prediction block based on the prediction block generation related information provided from the entropy decoder 210 and the previously decoded block or image information provided from the memory 245 .

The prediction units 230 and 235 may include a prediction unit determiner, an inter prediction unit, and an intra prediction unit. The prediction unit determining unit receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and separates the prediction unit from the current coding unit. , it may be determined whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 uses information required for inter prediction of the current prediction unit provided from the image encoder, based on information included in at least one of a previous image or a subsequent image of the current image including the current prediction unit. Inter-screen prediction for the current prediction unit may be performed. Alternatively, inter prediction may be performed based on information on a pre-restored partial region in the current image including the current prediction unit.

In order to perform inter prediction, based on a coding unit, it may be determined whether a method of generating motion information of a prediction unit included in a corresponding coding unit is generated by either a merge method or a motion estimation method.

The intra prediction unit 235 may generate a prediction block based on pixel information in the current image. When the prediction unit is a prediction unit on which intra prediction is performed, intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit 235 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and can be applied by determining whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering may be performed on the reference pixel of the current block by using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on a pixel value obtained by interpolating the reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel of a pixel unit having an integer value or less. . When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter may generate the prediction block through filtering when the prediction mode of the current block is the DC mode.

The reconstructed block or image may be provided to the filter unit 240 . The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter is applied to a corresponding block or image and information on whether a strong filter or a weak filter is applied when the deblocking filter is applied may be provided from the image encoder. The deblocking filter of the image decoder may receive deblocking filter-related information provided from the image encoder, and the image decoder may perform deblocking filtering on the corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding, offset value information, and the like.

ALF may be applied to a coding unit based on information on whether ALF is applied, ALF coefficient information, etc. provided from the encoder. Such ALF information may be provided by being included in a specific parameter set.

The memory 245 may store the reconstructed image or block to be used as a reference image or reference block, and may also provide the reconstructed image to an output unit.

5 is a diagram illustrating a change in brightness between a current image including a current block and a reference image.

When inter prediction is performed on the current block, the greater the change in brightness between the current image and the reference image, the greater the change in brightness between the current block and the prediction block to be selected from the reference image. Therefore, as the error due to the inter prediction of the current block increases, it can be expected that the energy for the residual signal of the current block also increases. And, as the energy of the residual signal increases, it can be expected that the error due to quantization also increases. As a result, when there is a brightness change between the current image and the reference image, the error for the residual block will increase compared to when there is no brightness change.

Accordingly, in the present invention, a method of generating a weighted prediction parameter by estimating a brightness change between images, and performing inter prediction using the weighted prediction parameter is proposed. When the prediction block is inter-predicted, the energy of the residual block can be prevented from rapidly increasing by using the weighted prediction parameter.

The weight prediction parameter may be generated in consideration of changes in brightness of the current image and the reference image, such as fade in or fade out. However, when determining the weight prediction parameter in consideration of only the above factors, it is difficult to respond when the brightness of the current image and the reference image is changed by a plurality of light sources or when the brightness of only a local area within the current image is changed. Accordingly, the present invention proposes a method of selectively using at least one of a plurality of weight prediction parameters to effectively perform inter-screen prediction even when the brightness is changed by a plurality of light sources or when the brightness of only a local area is changed We would like to suggest a way to do this.

Hereinafter, inter prediction using a weight prediction parameter will be described in detail with reference to the drawings.

6 is a flowchart illustrating a method of using a weight prediction parameter in an encoding apparatus.

The encoding apparatus may generate a weighted prediction parameter set for the current block based on the weighted prediction parameters of blocks coded prior to the current block to be encoded ( S601 ). Here, the weighted prediction parameter set may include weighted prediction parameters of blocks for which inter prediction is used among blocks encoded before the current block.

The encoding apparatus may generate a weight prediction parameter set based on weight prediction parameters used in neighboring blocks adjacent to the current block among blocks encoded before the current block. Here, the neighboring blocks adjacent to the current block may include spatial neighboring blocks and temporal neighboring blocks. As an example, the spatial neighboring block may include an upper-left block, an upper block, an upper-right block, a left block, and a lower-left block adjacent to the current block, and the temporal neighboring block of the current block includes the current block and the current block in the reference image. It may include a collocated block existing in the same location.

The encoding apparatus may configure a weighted prediction parameter set based on a reconstructed block encoded prior to the current block and a weighted prediction parameter for a prediction block corresponding to the reconstructed block.

The weight prediction parameter may be set differently according to the prediction direction between pictures. For example, the forward weight prediction parameter may be used when a block to be encoded performs forward prediction, and the backward weight prediction parameter may be used when a block to be encoded performs backward prediction. When a block is encoded by performing bi-prediction, both a forward weight prediction parameter and a backward weight prediction parameter may be used. In this case, the forward direction refers to a reference image that is past than the current image (ie, a reference image having a POC smaller than a picture order count (POC) of the current image), and the reverse direction refers to a reference image that is future than the current image (that is, of the current image). reference image having a POC greater than the POC).

The encoding apparatus may configure a weighted prediction parameter set of the current block by using at least one of a forward weighted prediction parameter and a backward weighted prediction parameter for a neighboring block based on the inter prediction direction of the current block.

7 is a diagram illustrating weight prediction parameters of neighboring blocks adjacent to the current block.

In the example shown in FIG. 7 , A to E denote spatial neighboring blocks of the current block, and Col denotes temporal neighboring blocks of the current block. The positions of A to E and Col refer to the example shown in FIG. 4 .

Referring to FIG. 7 , the neighboring blocks include at least one of a forward weight prediction parameter (ie, a weight parameter allocated to list 0) and a backward weight prediction parameter (ie, a weight parameter allocated to list 1) according to the inter prediction direction. shown to have one.

The encoding apparatus may determine weighted prediction parameters of neighboring blocks to be included in the weighted prediction parameter set according to the prediction direction of the current block. For example, if the current block is encoded through forward prediction, a weighted prediction parameter set is configured using forward weighted prediction parameters of neighboring blocks, and if the current block is encoded through backward prediction, the backward weights of neighboring blocks A weighted prediction parameter set may be constructed using the prediction parameters. If the current block is encoded through bi-prediction, a weight prediction parameter set may be constructed using forward weight prediction parameters and backward weight parameters of neighboring blocks.

For example, in the example shown in FIG. 7 , if the current block is encoded by forward prediction, the forward weight prediction parameters of neighboring blocks, (65,1), (64,2), (66,-2), (61,7) and (59,-2) may be included in the weight prediction parameter set.

Here, the weighted prediction parameter may include at least one of a multiplication parameter multiplied by the prediction sample and an addition parameter added to the prediction sample. In FIG. 7, the weight prediction parameters are expressed in the form of (w, o). In this case, w may indicate a multiplication parameter, and o may indicate an addition parameter.

The encoding apparatus may select an optimal weighted prediction parameter for the current block from among weighted prediction parameters included in the weighted prediction parameter set ( S602 ). When the optimal weight prediction parameter for the current block is selected, the encoding apparatus may encode information (eg, index information) specifying the selected weight prediction parameter.

When the optimal weighted prediction parameter is selected, inter prediction for the current block may be performed based on the selected weighted prediction parameter ( S603 ). As an example, inter prediction of the current block may be performed by multiplying a prediction pixel obtained through motion compensation by a multiplication parameter and then adding an addition parameter.

When a prediction block is generated as a result of inter prediction for the current block, the current block may be reconstructed based on the generated prediction block and the residual block after inverse quantization and inverse transformation ( S604 ).

Assuming that the block to be encoded next to the current block is encoded by inter prediction, the weight prediction parameter set of the next block may be generated using the weight prediction parameter of the current block. In this case, the weight prediction parameter of the current block that can be used by the next block may be any one selected from the set of weight prediction parameters of the current block.

As another example, the weight prediction parameter of the current block that can be used by the next block may be estimated from the reconstructed block and the prediction block of the current block. In this case, the encoding apparatus may perform weight prediction parameter estimation on the current block so that a weight prediction parameter set can be configured in a block next to the current block ( S605 ).

The reconstructed block has a quantization error compared to the original block, but the brightness change characteristic of the reconstructed block has a pattern similar to that of the original block. Accordingly, in order to estimate the weight prediction parameter in the decoding apparatus in the same way as in the encoding apparatus, the weight prediction parameter for the current block may be estimated by using the reconstruction block and the prediction block for the current block.

Specifically, when a reconstructed block is generated for the current block, the weighted prediction parameter of the current block for the next block may be estimated by using regression analysis on the reconstructed block and the prediction block to which the weighted prediction parameter is not applied. Equation 1 below represents an example of a regression analysis model.

In Equation 1, Y is the data of the reconstruction block, X is the data of the prediction block, w is the slope of the regression line, o is the intercept value of the regression line, and e is the error of prediction of the regression line. Specifically, Y denotes a pixel value of a reconstructed block, and X denotes a pixel value of a prediction block.

The weight prediction parameter of the current block may be obtained by partial differentiation of Equation 1 with w and o, respectively. As an example, when Equation 1 is partially differentiated by w and o, respectively, w and o at which the square of the error e is minimized may be derived as a multiplication parameter and an addition parameter, respectively.

The weight prediction parameter calculated based on Equation 1 may have a real value. The weight prediction parameter of the current block may be set as a real value calculated based on Equation 1, or may be set as a value obtained by integerizing a real value calculated based on Equation 1. As an example, the weight prediction parameter may be derived as an integer value derived by multiplying a real value calculated based on Equation 1 by 2 ^N. A variable N used for integerizing a weight prediction parameter may be encoded through a bisstream. Alternatively, the weight prediction parameter may be integerized by using a preset N value having the same value in the encoding apparatus and the decoding apparatus.

In Equation 1, all pixels in the reconstruction block may be set as the input range for X, and all pixels in the prediction block may be set as the input range for Y. As another example, instead of using the entire block, only some pixels through subsampling may be used when estimating the weight prediction parameter of the current block.

As an example, FIG. 8 is a diagram illustrating an example in which only some pixels of a current block are used for weight prediction parameter estimation. If it is assumed that the reconstructed block and the prediction block for the current block are subsampled by 1/2 in the horizontal and vertical directions, one of the four samples (2x2) may be used as an input of X or Y. For example, in the example shown in FIG. 8 , only pixels marked in black may be used for regression analysis.

In the example shown in FIG. 8 , the sample located at the upper left of the 2x2 samples is shown to be used for the regression analysis, but the upper right pixel, the lower left pixel, or the lower right pixel among the 2x2 pixels may be set to be used for the regression analysis. . Alternatively, whether the regression analysis is used using any of the above-mentioned four positions may be encoded through a bitstream using an index.

The encoding apparatus and the decoding apparatus may perform subsampling according to a predefined value, or may perform subsampling according to a value derived under a predefined condition. Alternatively, the encoding apparatus may encode the subsampling unit M, encode it through a bitstream, and inform the decoding apparatus that subsampling is performed with 1/M.

The weighted prediction parameter estimated for the current block may be used to construct the weighted prediction parameter set of the next block.

The maximum number of weight prediction parameters that the weight prediction parameter set of the current block can include may have a fixed number or a variable number according to the size, shape, or information signaled through the bitstream of the current block. . As an example, the maximum number P of maximum weight prediction parameters that the weight prediction parameter set may include may be encoded through a bitstream.

In FIG. 7 , blocks used to construct the weight prediction parameter set of the current block are limited to A to E and Col, but blocks located at different positions may also be used to configure the weight prediction parameter set of the current block.

As an example, the encoding apparatus may configure the weight prediction parameter set of the current block by using a block at a preset position. Alternatively, the encoding apparatus constructs a weighted prediction parameter set of a current block using an arbitrary location block, and encodes information specifying a location or block of a block used to construct a weighted prediction parameter set of the current block through a bitstream. You may. In this case, the location of the block or information specifying the block may be encoded through the bitstream.

When the weight prediction parameter set includes a plurality of weight prediction parameters, the weight prediction parameters may be arranged according to a preset order. As an example, the weight prediction parameters may be arranged in the order of the spatial neighboring block and the temporal neighboring block. Accordingly, in FIG. 7 , the weight prediction parameters of A to E may be first added to the weight prediction parameter set, and then the weight prediction parameter of Col may be added to the weight prediction parameter set. As another example, the weight prediction parameters may be arranged in an order most similar to motion information (eg, motion vector) of the current block. Alternatively, it is possible to prioritize and add to the weighted prediction parameter set by increasing the priority of the weighted prediction parameter derived based on the neighboring reconstructed block and the corresponding prediction block. It is also possible to increase and add to the weighted prediction parameter set.

The weighted prediction parameter set may include an initial weighted prediction parameter set. The initial weight prediction parameter means that the multiplication parameter and the addition parameter are set as initial values. Here, the initial values of the multiplication parameter and the addition parameter may be 1 and 0, respectively. As another example, the initial value of the multiplication parameter may be set to 1<<N, and the initial value of the addition parameter may be set to zero.

Information about the initial weight prediction parameter may be encoded through a bitstream and transmitted to a decoder. Alternatively, the initial weight prediction parameter may be set to have a fixed index in the weight prediction parameter set. As an example, the initial weight prediction parameter may be set to have index 0 in the weight prediction parameter set.

The weight prediction parameter set may include weight prediction parameters derived in units of images. Here, the weight prediction parameter set derived for each image may be generated based on a change in brightness between the current image and a reference image of the current image. As an example, the weight prediction parameter derived in units of images may be derived by Equation 1 above. In this case, all pixels in the current image may be set as the input range for X, and all pixels in the reference image may be set as the input range for Y.

Information on the weight prediction parameter of the image unit may be encoded through a bitstream and transmitted to a decoder. Here, the information on the weight prediction parameter may include a weight prediction parameter value or a residual value of the weight prediction parameter. For example, in the decoder, the weight prediction parameter of the image unit may be obtained by summing the prediction value and the residual value of the weight prediction parameter. In this case, the predicted value of the weight prediction parameter may be determined by the precision N of the current block, and the residual value may be determined based on information decoded from the bitstream.

In this case, the precision N of the current block may be encoded through the bitstream and transmitted to the decoding apparatus through the bitstream.

Alternatively, it is possible to determine the precision N of the current block by the same method in the encoding apparatus and the decoding apparatus. For example, the precision N of the current block may be adaptively determined according to the size, shape, etc. of the current block.

When the image unit weight prediction parameter is included in the weight prediction parameter set, the index assigned to the image unit weight prediction parameter may be encoded through a bitstream or may have a preset value. As an example, if it is assumed that the weighted prediction parameter set consists of up to six weighted prediction parameters, the initial weighted prediction parameter may be set to have index 0, and the image unit weighted prediction parameter may be set to have index 1. Indexes 2 to 5 may be used for weight prediction parameters derived from neighboring blocks.

When the number of weighted prediction parameters derived from neighboring blocks is smaller than the maximum number that the weighted prediction parameter set can include, the initial weighted prediction parameter or the image unit weighted prediction parameter may be added to the weighted prediction parameter set. As an example, the number of weight prediction parameters that can be used is 4, but when a neighboring block adjacent to the current block is encoded with intra prediction, or the weight prediction parameter of the neighboring block cannot be used because the current block is at an image boundary, etc. When four weight prediction parameters cannot be derived from neighboring blocks for the reason of , at least one of an initial weight prediction parameter or an image unit weight prediction parameter may be included in the weight prediction parameter set.

The weighted prediction parameter set may include a combined weighted prediction parameter generated by combining two or more weighted prediction parameters. For example, when the number of weighted prediction parameters that can be derived from a neighboring block is smaller than the number of usable weighted prediction parameters, the combined weighted prediction parameters generated by combining two or more weighted prediction parameters are added to the weighted prediction parameter set. can do.

Assuming that the weighted prediction parameter is composed of (w, o), the combined weighted prediction parameter may be generated in consideration of the prediction mode of the neighboring blocks or the prediction direction of the neighboring blocks.

As an example, FIG. 9 shows an example in which a combined weight prediction parameter is generated.

In FIG. 9 , A to D indicate neighboring blocks adjacent to the current block, and Lists 0 and 1 indicate weight prediction parameters of neighboring blocks when a forward reference image and a backward reference image are used, respectively.

In FIG. 9 , A has only a forward weighted prediction parameter as it performs forward prediction, and D has only a backward weighted prediction parameter as it performs backward prediction. C is shown to have both a forward weight prediction parameter and a backward weight prediction parameter as it performs bidirectional prediction. B is shown as having neither a forward weighted prediction parameter nor a backward weighted prediction parameter. As an example, if B is encoded with intra prediction, B will not have a weighted prediction parameter.

Accordingly, in the example shown in FIG. 9 , the forward weighted prediction parameter (ie, the weighted prediction parameter for List 0) is obtained only from blocks A and C, and the backward weighted prediction parameter (ie, the weighted prediction parameter for List 1) ) can be obtained only from blocks C and D.

Assuming that the weighted prediction parameter set of the current block can include four forward weighted prediction parameters, the remaining two forward weighted prediction parameters except for the two forward weighted prediction parameters derived from neighboring blocks are two or more forward weighted prediction parameters. It can be created by combining parameters. Alternatively, assuming that the weight prediction parameter set of the current block may include four backward weight prediction parameters, the remaining two backward weight prediction parameters except for the two backward weight parameters derived from neighboring blocks are two or more backward weight prediction parameters. It can be created by combining parameters. In FIG. 9, by combining the forward weight prediction parameters of A and C, the forward weight prediction parameters of B and D are generated, and the backward weight prediction parameters of A and B are generated by combining the backward weight prediction parameters of C and D. was exemplified as

When the combined weight prediction parameter is added to the weight prediction parameter set, an index assigned to the combined weight prediction parameter may be determined according to a preset priority.

If all of the predetermined weighted prediction parameter sets are not generated despite combining the weighted prediction parameters in the neighboring blocks, all of the remaining weighted prediction parameters may be set as initial weighted prediction parameters.

10 shows an example of decoding a weight prediction parameter in a decoding apparatus. Referring to FIG. 10 , first, the decoding apparatus may generate a weight prediction parameter set based on weight prediction parameters of neighboring blocks adjacent to the current block among blocks decoded prior to the current block ( S1001 ). Since the generation of the weight prediction parameter set has been previously described in detail in the operation of the encoding apparatus, a detailed description thereof will be omitted.

When the weight prediction parameter set for the current block is configured, the decoding apparatus may determine the weight prediction parameter for the current block based on index information specifying any one of the weight prediction parameters included in the weight prediction parameter set (S1002). ). Here, the index information may be decoded from the bitstream.

When the weight prediction parameter for the current block is determined, inter prediction for the current block may be performed using the determined weight prediction parameter ( S1003 ). As an example, inter prediction of the current block may be performed by multiplying a prediction sample obtained through motion compensation by a multiplication parameter and then adding an addition parameter.

When a prediction block is generated as a result of inter prediction for the current block, the current block may be reconstructed based on the generated prediction block and the residual block ( S1004 ).

When the restoration of the current block is completed, a weight prediction parameter for the current block may be estimated so that a block to be decoded next to the current block may use it ( S1005 ). Since the estimation of the weight prediction parameter set of the current block has been described in detail in the operation of the encoding apparatus, a detailed description thereof will be omitted.

Example methods of the present disclosure are expressed as a series of operations for clarity of description, but this is not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, other steps may be included in addition to the illustrated steps, other steps may be included except some steps, or additional other steps may be included except some steps.

Various embodiments of the present disclosure do not list all possible combinations but are intended to describe representative aspects of the present disclosure, and matters described in various embodiments may be applied independently or in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure includes software or machine-executable instructions (eg, operating system, application, firmware, program, etc.) that cause an operation according to the method of various embodiments to be executed on a device or computer, and such software or and non-transitory computer-readable media in which instructions and the like are stored and executed on a device or computer.

Claims (4)

determining a weight prediction parameter for a current block by using a predetermined reference region; Here, the reference area is an area reconstructed before the current block, and includes a neighboring area adjacent to the current block, and
Predicting the current block based on the weight prediction parameter,
Information specifying a location of a surrounding area used to determine the weight prediction parameter is obtained from a bitstream,
The surrounding area is specified by information obtained from the bitstream,
The peripheral region includes a lower left peripheral region or an upper right peripheral region of the current block. The method of claim 1,
The weight prediction parameter is determined based on at least one of a size and a shape of the current block. determining a weight prediction parameter for a current block by using a predetermined reference region; Here, the reference region is a region coded before the current block and includes a neighboring region adjacent to the current block, and
Predicting the current block based on the weight prediction parameter,
Information specifying the location of the surrounding area used to determine the weight prediction parameter is encoded,
The peripheral region includes a lower-left peripheral region or an upper-right peripheral region of the current block. A computer-readable recording medium storing a bitstream generated by an encoding method, comprising:
The encoding method may include: determining a weight prediction parameter for a current block by using a predetermined reference region; Here, the reference region is a region coded before the current block and includes a neighboring region adjacent to the current block, and
Predicting the current block based on the weight prediction parameter,
Information specifying the location of the surrounding area used to determine the weight prediction parameter is encoded,
The peripheral area includes a lower left peripheral area or an upper right peripheral area of the current block.

Images