WO2022039635A1 - Method and apparatus for hadamard transform domain filtering of the video signal with various bit depth - Google Patents
Method and apparatus for hadamard transform domain filtering of the video signal with various bit depth Download PDFInfo
- Publication number
- WO2022039635A1 WO2022039635A1 PCT/RU2021/050266 RU2021050266W WO2022039635A1 WO 2022039635 A1 WO2022039635 A1 WO 2022039635A1 RU 2021050266 W RU2021050266 W RU 2021050266W WO 2022039635 A1 WO2022039635 A1 WO 2022039635A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- bit depth
- depth value
- luma
- block
- luma bit
- Prior art date
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 148
- 238000000034 method Methods 0.000 title claims abstract description 122
- 241000023320 Luma <angiosperm> Species 0.000 claims abstract description 178
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 claims abstract description 178
- 238000001228 spectrum Methods 0.000 claims description 96
- 238000012545 processing Methods 0.000 claims description 35
- 238000013139 quantization Methods 0.000 claims description 27
- 238000010606 normalization Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 32
- 238000010586 diagram Methods 0.000 description 22
- 238000009825 accumulation Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 238000004891 communication Methods 0.000 description 10
- 241001125929 Trisopterus luscus Species 0.000 description 9
- 230000009286 beneficial effect Effects 0.000 description 9
- 230000009466 transformation Effects 0.000 description 8
- 230000001360 synchronised effect Effects 0.000 description 7
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 230000000295 complement effect Effects 0.000 description 5
- 238000013500 data storage Methods 0.000 description 4
- 238000009795 derivation Methods 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000007620 mathematical function Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
- H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
Definitions
- the present invention relates to the field of video coding. More specifically, the present invention relates to a filter for video coding and to a method for filtering reconstructed image blocks, a program implementing such method, as well as an encoding apparatus and a decoding apparatus comprising said filter for video coding.
- Digital video has been widely used since the introduction of DVD-discs. Before transmission, the video is encoded and transmitted using a transmission medium. The viewer receives the video and uses a viewing device to decode and display the video. Over the years the quality of video has improved, for example, because of higher resolutions, color depths and frame rates. This has led to larger data streams that are nowadays commonly transported over internet and/or over mobile communication networks.
- the High Efficiency Video Coding is an example of a video coding standard that is commonly known to persons skilled in the art.
- a coding unit CU
- PU prediction units
- TUs transform units
- the Versatile Video Coding (VVC) next generation standard is the most recent joint video project of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG) standardization organizations, working together in a partnership known as the Joint Video Exploration Team (JVET).
- JVET Joint Video Exploration Team
- VVC is also referred to as ITU-T H.266/Next Generation Video Coding (NGVC) standard.
- NGVC Next Generation Video Coding
- Image filtering is frequently used to emphasize certain features of an image or to enhance the objective or perceptual quality of the filtered image. Image filtering has to deal with various sources of noise. Accordingly, various approaches for quality enhancement have been proposed and are currently in use.
- each reconstructed frame is divided into a set of small macro-blocks (super- pixels) and each macroblock is filtered by the adaptive loop filter in that each pixel of the filtered reconstructed frame is a weighted sum of several pixels in the connected area of the pixel from the reconstructed frame around the position of the generating filtered pixel.
- Weighting coefficients also referred to as filter coefficients
- Edges often have a big size and therefore the number of transmitted weighting coefficients can become too large for an efficient processing.
- a large number of weighting coefficients requires a complex rate-distortion optimization (RDO) at the encoder side for decreasing the number of weighting coefficients for transmission.
- RDO rate-distortion optimization
- ALF requires implementation of universal multipliers and these multipliers should be reloaded for each 2x2 pixel block.
- Embodiments of the invention provide an improved filter and method allowing to improve the filtering efficiency with limited complexity and, thus, increase the video coding efficiency.
- the proposed method can be applied for wide range of video signal representation within encoder implementations.
- the proposed method allow to utilize the filtering process for various bit depths in integer range from 8 to 16 bits.
- the invention relates to a method of coding implemented by a decoding device, comprising: receiving a bitstream; obtaining from the bitstream, a first luma bit depth value of luma samples of a current picture; performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table (LUT) based on the obtained first luma bit depth value.
- HTDF Hadamard Transform Domain Filtering
- performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table based on the obtained first luma bit depth value comprises: obtaining a second luma bit depth value by adding 8 to the first luma bit depth value obtained from the bitstream; and performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table (LUT) based on the obtained second luma bit depth value.
- HTDF Hadamard Transform Domain Filtering
- the obtained first luma bit depth value from the bitstream is different from the second luma bit depth value.
- the second luma bit depth value is the real luma bit depth value so there is a mapping between the first and second luma bit depth values and the second luma bit depth value.
- the second luma bit depth value is not obtaining from the bitstream directly.
- the second luma bit depth value is obtained by adding 8 to the first luma bit depth value obtained from the bitstream.
- the step of performing an HTDF post reconstruction filtering of a block of the current picture by using a lookup table based on the obtained second luma bit depth value comprises: if second luma bit depth value is less than 10, left shifting an input value of the LUT by [10 - second luma bit depth value] bits; otherwise, right shifting the input value of the LUT by [second luma bit depth value - 10] bits.
- the step of performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table based on the obtained second luma bit depth value can be done performed by left shifting the input value of the LUT by a number of bits calculated as [10-second luma bit depth value] if the second luma bit depth value is less than 10, and right shifting the input value of the LUT by a number of bits calculated as [second luma bit depth value-10] otherwise, in particular, if the second luma bit depth value is larger than 10.
- HTDF Hadamard Transform Domain Filtering
- the lookup table is predefined or designed for a third luma bit depth value.
- the lookup table is used for a plurality of obtained second luma bit depth values, in particular for obtained second luma bit depth values smaller than the third luma bit depth value and/or not smaller than the third luma bit depth value.
- the third obtained luma bit depth value is 10.
- the obtained second luma bit depth value is one of 8-16 bits.
- the lookup table is used for 10 bits luma bit depth value, 8 bits luma bit depth value, 12 bits luma bit depth value and/or other bit depth value.
- the performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using the lookup table based on the obtained first luma bit depth value comprises: when the obtained second luma bit depth value is not a third luma bit depth value, performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of the block of the current picture by using the lookup table, wherein an input of the lookup table is normalized based on the obtained second luma bit depth value, and an output of the lookup table is normalized based on the obtained second luma bit depth value, wherein the obtained second luma bit depth value is obtained based on the obtained first luma bit depth value; and/or when the obtained first luma bit depth value is the third luma bit depth value, performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using the lookup table, wherein no normalization is performed for an input of the
- an input (e.g. LUT element index) of the lookup table(s) is normalized to the first bit depth representation (such as 10 bits bit depth) and an output (e.g. the filtered Hadamard spectrum components) of the lookup table(s) is normalized to the obtained internal bit depth value (such as 8 bits bit depth or 12 bits bit depth).
- the step of performing an HTDF post reconstruction filtering of a block of the current picture comprises: when at least one condition is satisfied, performing an HTDF post reconstruction filtering of a block of the current picture using one or more predefined HTDF lookup tables based on a formula which is associated with the first luma bit depth value; or when at least one condition is satisfied, performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using lookup table(s) based on the obtained first luma bit depth value.
- HTDF Hadamard Transform Domain Filtering
- the condition is based on a comparison result between a scaled Hadamard spectrum component and a predefined threshold value; or when the condition is satisfied, comprises: when a scaled Hadamard spectrum component is less than a predefined threshold value.
- the step of performing an HTDF post reconstruction filtering of a block of the current picture comprises: loading a current pixel of a reconstructed block or an extended reconstructed block including a current pixel and neighboring pixels of the current pixel according to a predefined scan template; obtaining Hadamard spectrum components (fHad) by performing a Hadamard transform for the current pixel or the current pixel and the neighboring pixels; obtaining filtered Hadamard spectrum components (fHadFilt) based on the obtained Hadamard spectrum components by using the lookup table based on the obtained second luma bit depth value, wherein an input of the lookup table(s) is normalized based on the obtained second luma bit depth value, and an output of the lookup table is normalized based on the obtained second luma bit depth value, to obtain the filtered Hadamard spectrum components (fHadFilt); obtaining filtered pixels by performing an inverse Hadamard transform transform for the filtered Hadamard spectrum components; and
- the step of obtaining filtered Hadamard spectrum components comprises:
- the lookup table is obtained based on a quantization parameter (Qp) of the block.
- the following formula is used in the following way depending on the bit depth (bitDepth y ) value:
- the invention relates to a decoding apparatus for decoding a block from a received bitstream, wherein the decoding apparatus comprises processing circuitry for carrying out the method according to the first aspect or any implementation form thereof.
- the invention relates to a computer program product comprising program code for performing the method according to the first aspect of any implementation form thereof when executed on a computer or a processor.
- the invention relates to a decoder, comprising: one or more processors; and a non-transitory computer-readable storage medium coupled to the processors and storing programming for execution by the processors, wherein the programming, when executed by the processors, configures the decoder to carry out the method according to the first aspect or any implementation form thereof.
- the invention relates to a non-transitory computer-readable medium carrying a program code which, when executed by a computer device, causes the computer device to perform the method according to the first aspect or any implementation form thereof.
- the invention relates to a decoder, comprising: receiving means for receiving a bitstream; obtaining means for obtaining from the bitstream, a first luma bit depth value of luma samples of a current picture; and filtering means for performing an Hadamard Transform Domain Filtering post reconstruction filtering of a block of the current picture by using a lookup table based on the obtained first luma bit depth value.
- Fig. 1 shows a schematic diagram illustrating an encoding apparatus according to an embodiment comprising a filter according to an embodiment
- Fig. 2 shows a schematic diagram illustrating a decoding apparatus according to an embodiment comprising a filter according to an embodiment
- FIG. 3A shows schematic diagram illustrating aspects of a filtering process implemented in a filter according to an embodiment
- Fig. 3B shows a schematic diagram illustrating aspects of a filtering process implemented in a filter according to an embodiment
- FIG. 4 illustrates templates for different pixel position inside square reconstructed block
- Fig. 5 illustrates equivalent filter shape for one pixel
- FIG. 6 is a schematic diagram illustrating an exemplary structure of an apparatus according to an embodiment
- Fig. 7 is a flow diagram illustrating a decoding method according to an embodiment
- Fig. 8 is a flow diagram illustrating an encoding method according to an embodiment
- Fig. 9 is a flow diagram illustrating a method according to an embodiment
- Fig. 10 shows a schematic diagram illustrating an encoding apparatus according to another embodiment comprising a filter according to another embodiment
- Fig. 11 shows a schematic diagram illustrating a decoding apparatus according to another embodiment comprising a filter according to another embodiment
- Fig. 12 is a block diagram showing an example structure of a content supply system 3100 which realizes a content delivery service.
- Fig. 13 is a block diagram showing a structure of an example of a terminal device.
- Fig. 14 is a block diagram showing a decoder according to an embodiment.
- Fig. 15 is a block diagram of a decoder according to another embodiment.
- a disclosure in connection with a described method may also hold true for a corresponding device or system configured to perform the method and vice versa.
- a corresponding device may include a unit to perform the described method step, even if such unit is not explicitly described or illustrated in the figures.
- the features of the various exemplary aspects described herein may be combined with each other, unless specifically noted otherwise.
- Figure 1 shows an encoding apparatus 100 according to an embodiment comprising a filter 120 according to an embodiment.
- the encoding apparatus 100 is configured to encode a block of a frame of a video signal comprising a plurality of frames (also referred to as pictures or images herein), wherein each frame is dividable into a plurality of blocks and each block comprises a plurality of pixels.
- the blocks could be macro blocks, coding tree units, coding units, prediction units and/or prediction blocks.
- block in this disclosure is used for any type block or for any depth block, for example, the term “block” is included but not limited to root block, block, sub-block, leaf node, and etc.
- the blocks to be coded do not necessarily have the same size.
- One picture may include blocks of different sizes and the block rasters of different pictures of video sequence may also differ.
- block in this disclosure may refer to an MxN (M-column by N-row) array of samples, or an MxN array of transform coefficients.
- a frame of video may be regarded as a (digital) picture which can be regarded as a two- dimensional array or matrix of samples with intensity values.
- a sample in the array may also be referred to as pixel (short form of picture element) or a pel.
- the number of samples in horizontal and vertical direction (or axis) of the array or picture define the size and/or resolution of the picture.
- typically three color components are employed, i.e. the picture may be represented or include three sample arrays.
- RBG format or color space a picture comprises a corresponding red, green and blue sample array.
- each pixel is typically represented in a luminance and chrominance format or color space, e.g.
- YCbCr which comprises a luminance component indicated by Y (sometimes also L is used instead) and two chrominance components indicated by Cb and Cr.
- the luminance (or short luma) component Y represents the brightness or grey level intensity (e.g. like in a grey-scale picture), while the two chrominance (or short chroma) components Cb and Cr represent the chromaticity or color information components.
- a picture in YCbCr format comprises a luminance sample array of luminance sample values (Y), and two chrominance sample arrays of chrominance values (Cb and Cr).
- Pictures in RGB format may be converted or transformed into YCbCr format and vice versa, the process is also known as color transformation or conversion. If a picture is monochrome, the picture may comprise only a luminance sample array. It is understood that the present disclosure is applicable to samples (pixels) of any one or more (or all) color components.
- video coding generally applies for video encoding and/or video decoding.
- the encoding apparatus 100 is implemented in the form of a hybrid video coding encoder.
- the first frame of a video signal is an intra frame, which is encoded using only intra prediction.
- the embodiment of the encoding apparatus 100 shown in figure 1 comprises an intra prediction unit 154 for intra prediction.
- An intra frame can be decoded without information from other frames.
- the intra prediction unit 154 can perform the intra prediction of a block on the basis of information provided by the intra estimation unit 152. It is noted that in another example, the intra estimation unit 152 and intra prediction unit 154 may be not separately presented in the encoding apparatus 100, for example, the intra estimation unit 152 and intra prediction unit 154 may be integrated together.
- the intra prediction unit 154 may be responsible for reference samples derivation that comprises logic for checking availability of neighboring reconstructed samples and retrieving the neighboring reconstructed samples (e.g. form the line buffer 116 in some specific implementations) if they are available, and substituting them by other available (neighboring) samples if they are not available. These reference samples are used as an input for intra prediction.
- the blocks of subsequent frames following the first intra frame can be coded using inter or intra prediction, as selected by a mode selection unit 160.
- the encoding apparatus 100 shown in figure 1 further comprises an inter prediction unit 144.
- the inter prediction unit 144 can be configured to perform motion compensation of a block based on motion estimation provided by the inter estimation unit 142. It is noted that in another example, the inter estimation unit 142 and inter prediction unit 144 may be not separately presented in the encoding apparatus 100, for example, the inter estimation unit 142 and inter prediction unit 144 may be integrated together.
- a residual calculation unit 104 determines the difference between the original block and its prediction, i.e. the residual block defining the prediction error of the intra/inter picture prediction.
- This residual block is transformed by the transformation unit 106 (for instance using a DCT) and the transformation coefficients are quantized by the quantization unit 108.
- the output of the quantization unit 108 as well as the coding or side information provided, for instance, by the intra prediction unit 154, the inter prediction unit 144 and the filter 120 are further encoded by an entropy encoding unit 170.
- a hybrid video encoder usually duplicates the decoder processing such that both will generate the same predictions.
- the inverse quantization unit 110 and the inverse transformation unit perform the inverse operations of the transformation unit 106 and the quantization unit 108 and duplicate the decoded approximation of the residual block.
- the decoded residual block data is then added to the results of the prediction, i.e. the prediction block, by the reconstruction unit 114.
- the output of the reconstruction unit 114 can be provided to a line buffer 116 to be used for intra prediction and is further processed by the filter 120, which will be described in more detail below.
- the final picture is stored in the decoded picture buffer 130 and can be used for the inter prediction of subsequent frames.
- Figure 2 shows a decoding apparatus 200 according to an embodiment comprising a filter 220 according to an embodiment.
- the decoding apparatus 200 is configured to decode a block of a frame of an encoded video signal.
- the decoding apparatus 200 is implemented as a hybrid decoder.
- An entropy decoding unit 204 performs entropy decoding of the encoded picture data, which generally can comprise prediction errors (i.e. residual blocks), motion data and other side information, which are needed, in particular, for an intra prediction unit 254 and an inter prediction unit 244 as well as other components of the decoding apparatus 200, such as the filter 220.
- prediction errors i.e. residual blocks
- motion data i.e. residual blocks
- other side information which are needed, in particular, for an intra prediction unit 254 and an inter prediction unit 244 as well as other components of the decoding apparatus 200, such as the filter 220.
- the intra prediction unit 254 and the inter prediction unit 244 of the decoding apparatus 200 shown in figure 2 are selected by a mode selection unit 260 and function in the same way as the intra prediction unit 154 and the inter prediction unit 144 of the encoding apparatus 100 shown in figure 1 , so that identical predictions can be generated by the encoding apparatus 100 and the decoding apparatus 200.
- the intra prediction unit 154, 254 may be responsible for reference samples derivation that comprises logic for checking availability of neighboring reconstructed samples and retrieving neighboring reconstructed samples (e.g. form the line buffer 216, 116 in some specific implementation) if they are available and substitution them by other available neighboring samples if they are not available. These reference samples are used as an input for intra prediction.
- a reconstruction unit 214 of the decoding apparatus 200 is configured to reconstruct the block on the basis of the filtered predicted block and the residual block provided by the inverse quantization unit 210 and the inverse transformation unit 212.
- the reconstructed block can be provided to a line buffer 216 used for intra prediction and the filtered block/frame can be provided to a decoded picture buffer 230 by the filter 220 for inter prediction.
- the filter 120, 220 may be used at a frame level, for example, the filter 120, 220 may be configured to process a reconstructed frame from a decoded reconstructed video stream for generating a filtered reconstructed frame, where the reconstructed frame includes a plurality of blocks.
- the filter 120, 220 may be also used at a block level after block reconstruction (or post-reconstruction filter) without waiting for a whole frame, for example, the filter 120, 220 may be configured to process a reconstructed block for generating a filtered reconstructed block and also providing filtered reconstructed samples to the line buffer 1016, 1116, as it exemplary illustrated on Fig. 10 or filtered reconstructed samples to the line buffer 1 116, as it exemplary illustrated on Fig .11 , where the reconstructed block includes a plurality of pixels.
- Figure 10 shows an example of an encoding apparatus 1000 according to another embodiment, where the filter is applied as a post-reconstruction filter 1018 after block reconstruction, such as immediately after block reconstruction.
- Result of a reconstruction unit 1014 can be provided to the post-reconstruction filter 1018.
- the result of the postreconstruction filter 1018 can be provided to a line buffer 1016 and is further processed by a loop filter 1020 or directly provided to a decoded picture buffer 1030.
- Figure 11 shows an example of a decoding apparatus 1100 according to another embodiment, where the filter is applied as a post-reconstruction filter 11 18 after block reconstruction, such as immediately after block reconstruction.
- Result of a reconstruction unit 11 14 can be provided to the post-reconstruction filter 11 18.
- the result of the post-reconstruction filter 1 118 can be provided to a line buffer 1116 and is further processed by a loop filter 1120 or directly provided to a decoded picture buffer 1130.
- the filter 120, 220, 1018, 1118 comprises, in an embodiment, one or more processors (or one or more processing units or processing circuitry).
- the one or more processors is/are configured to: performing an HTDF post reconstruction filtering of a block of the current picture based on a formula which is associated with a luma bit depth value, where the bit depth value of luma samples of a current picture is obtained from the bitsream.
- all pixels inside a reconstructed block can be processed independently from each other.
- neighboring pixels are used for example, as illustrated on Figure 3A, pixels r(1) to r(7) are used, and pixels r(0) to r(7) form one processing group.
- Fig. 3A or 3B shows a schematic diagram 300 (or 300’) illustrating aspects of a filtering process implemented in a filter according to an embodiment.
- a current pixel and its neighboring pixels from a reconstructed block are loaded to a linear buffer, such as a linear buffer 1 16, 216, 1016, 11 16 according to a predefined scan template.
- a 1 D transform is performed for pixel r(0) and its neighboring pixels r(1) to r(7) in the linear buffer to obtain spectrum components R:
- the 1 D transform may be a Hadamard transform.
- the set of gain coefficients for all spectrum components is a frequency impulse response of the filter.
- the gain coefficient G(i, ⁇ ) depends on the corresponding spectrum component R(i) and a filtering parameter, such as ⁇ .
- the gain coefficient G(i, ⁇ ) depends on one or more filtering parameters and one or more of the corresponding spectrum components.
- the respective gain coefficient may depend on the one or more filtering parameters, and the corresponding spectrum component as well as neighboring spectral components to the left and to the right of the spectrum component.
- each gain coefficient G(i, ⁇ ) is a function of a spectrum component of the reconstructed block and the filtering parameter
- the gain coefficient G(i, ⁇ ) can be described by the following formula as an example: where (i) is an index of a spectrum component, R(i) is the spectrum component corresponding to (i) index, G(i, ⁇ ) is the gain coefficient corresponding to R(i) , a is the filtering parameter, and m is a normalization constant equal to number of spectrum components.
- m corresponds to the length of the 1 D transformation.
- An exemplary and limiting value for m is 4. However, the present disclosure is applicable to any size of the 1 D transformation.
- Different spectrum components may have a same gain coefficient, or may have different gain coefficients.
- QP codec quantization parameter
- Different spectrum components may have a same filtering parameter, or may have different filtering parameters.
- gain coefficient for each frequency is derived from spectrum component of the reconstructed pixels. Therefore, the method 300 (or 300’) does not need transmission of filtering parameters and can be applied for any reconstructed block without additional signaling.
- Spectrum gain coefficient is less 1 , so filtering can be implemented based on a short look up table (LUT) reading according to the following formula: where LUT (i) is an index of a spectrum component, R(i) is the spectrum component corresponding to index (i), ⁇ ii the filtering parameter, and THR is a threshold, m is normalization constant equal to number of spectrum components.
- F(i, ⁇ ) represents an (i)- th filtered spectrum component, filtered with a filter parameterized with the parameters o.
- THR may be calculated from following formula, where C is a value close to 1 , for example, 0.9.
- the result of the inverse 1 D transform is placed to linear buffer of filtered reconstructed pixels.
- a filtered reconstructed block is generated based on the filtered pixels estimated in previous processing steps.
- the filtered pixel f(0) is placed to its original position according to the predefined scan template.
- Other filtered samples f(1) - f(7) are not used.
- more than one filtered pixels for example, all filtered pixels from the linear buffer of filtered samples are added to an accumulation buffer according to the predefined scan template used at step 302 (or 302’) of FIG. 3A or 3B.
- the accumulation buffer should be initialized by zero before the filtering step.
- final filtered pixels are obtained as accumulated values in the accumulation buffer divided by number of pixels added to a current position of the accumulation buffer, in other words, number of pixels values added to current position of accumulation buffer on previous processing steps. Then the filtered reconstructed block is generated based on the final filtered pixels.
- the accumulation buffer should be initialized by unfiltered pixel values multiplied by maximum number of pixel values to be added in the block.
- the maximum number of pixel values to be added in the block is defined based on scan template. Indeed scan template defines number of pixel values added for each position. Based on that the maximum number from all positions in the block can be selected and used during accumulation buffer initialization. Then during each accumulation step unfiltered pixel value is subtracted from corresponding filtered value and added to accumulation buffer:
- This embodiment allows to avoid storing number of pixels added to current position and allows to replace division and multiplication by shift operation at the last normalization step and accumulation buffer initialization step correspondingly if the maximum number of pixel values added is power of 2 e.g. 2, 4, 8 etc.
- a scan template is chosen based on position of filtering pixel inside reconstructed block for steps 302 and 310.
- Scan template is chosen to guarantee all pixels be inside reconstructed CU and placed (located) close to processing pixel.
- Arbitrary scan order can be used inside template.
- the predefined scan template is defined as set of spatial or raster offsets relative to a position of the current pixel inside the reconstructed block, where offsets point to neighbor pixels are inside the reconstructed block. It is noted that for the related part of the method 300’ as illustrated in fig. 3B, reference may be made to the method 300 as illustrated in fig. 3A.
- Fig. 4 illustrates templates fordifferent pixel position inside square reconstructed block (square CU reconstructed block). According to this figure, boundary pixels can be filtered based on 4 point transform and central pixels can be filtered based on 8 point transform.
- the described filter can be selectively applied depending on conditions: • for reconstructed blocks with non-zero residual signal;
- Filter parameter sigma and scan pattern may vary depending on conditions listed above.
- Fig. 5 illustrates equivalent filter shape considering one pixel inside of current block for exemplary scan template ( 0,0 ), ( 0,1 ), ( 1 ,0 ), ( 1 ,1 ).
- current pixel is marked by dark-gray color in the center of 3x3 square.
- Filtered pixel is obtained by combining transform domain filtered samples from four 2x2 processing groups, in other words, filtered pixel is obtained by combining transform domain filtered samples from four processing groups, where each processing group being the shape/size of 2x2. It can be understood that if current pixel is located in block border (e.g.
- top border top left and top right 2x2 processing groups are unavailable and only two 2x2 processing groups (bottom left and bottom right) can be used for filtering. Furthermore, if current pixel is located in block corner (e.g. top-left corner) only one 2x2 processing group (bottom right) can be used for filtering.
- the current block can be padded by additional samples.
- Fig. 6 is a block diagram of an apparatus 600 that can be used to implement various embodiments.
- the apparatus 600 may be the encoding apparatus as shown in Fig .1 or 10 and the decoding apparatus as shown in Fig. 2 or 11 . Additionally, the apparatus 600 can host one or more of the described elements.
- the apparatus 600 is equipped with one or more input/output devices, such as a speaker, microphone, mouse, touchscreen, keypad, keyboard, printer, display, and the like.
- the apparatus 600 may include one or more central processing units (CPUs) 610, a memory 620, a mass storage 630, a video adapter 640, and an I/O interface 660 connected to a bus.
- the bus is one or more of any type of several bus architectures including a memory bus or memory controller, a peripheral bus, a video bus, or the like.
- the CPU 610 may have any type of electronic data processor.
- the memory 620 may have, or be, any type of system memory such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), a combination thereof, or the like.
- the memory 620 may include ROM for use at boot-up, and DRAM for program and data storage for use while executing programs.
- the memory 620 is non-transitory.
- the mass storage 630 includes any type of storage device that stores data, programs, and other information and to make the data, programs, and other information accessible via the bus.
- the mass storage 630 includes, for example, one or more of a solid state drive, hard disk drive, a magnetic disk drive, an optical disk drive, or the like.
- the video adapter 640 and the I/O interface 660 provide interfaces to couple external input and output devices to the apparatus 600.
- the apparatus 600 may provide SQL command interface to clients.
- examples of input and output devices include a display 690 coupled to the video adapter 640 and any combination of mouse/keyboard/printer 670 coupled to the I/O interface 660.
- Other devices may be coupled to the apparatus 600, and additional or fewer interface cards may be utilized.
- a serial interface card (not shown) may be used to provide a serial interface for a printer.
- the apparatus 600 also includes one or more network interfaces 650, which includes wired links, such as an Ethernet cable or the like, and/or wireless links to access nodes or one or more networks 680.
- the network interface 650 allows the apparatus 600 to communicate with remote units via the networks 680.
- the network interface 650 may provide communication to database.
- the apparatus 600 is coupled to a local-area network or a wide-area network for data processing and communications with remote devices, such as other processing units, the Internet, remote storage facilities, or the like.
- Reconstructed video signal filtering process can be integrated into the RDO process and performing on a CU-based basis, such approach is called Post-reconstruction filtering.
- Hadamard Transform Domain Filter (HTDF) is an example of the post reconstruction filtering.
- HTDF process a subject CU is firstly divided onto a set of overlapped blocks 2 x 2 luma samples, then 1 D Hadamard transform is performed for each block, and then for each frequency coefficient R i starting from 1 st (0 th coefficient is bypassed) HTDF performs filtering according to the equation below.
- ⁇ is a quantization noise estimation.
- the value of quantization noise in a coded image tends to be roughly proportional to the quantization step size which is normally defined by exponential equation based on codec quantization parameter (QP).
- the original efficient LUT implementation of HTDF post-reconstruction filter supposed to have predefines set of LUTs optimized for 10bits internal video signal representation, which means that bit depth of the samples of the luma array within encoding and/or decoding processes is 10 bits regardless of original input video signal bitdepth.
- bit_depth_luma_minus8 syntax element and decoder typically operates with BitDepthy variable defined as
- BitDepthy 8 + bit_depth_luma_minus8
- bit_depth_luma_minus8 corresponds to the first luma bit depth value
- bitDepthy corresponds to the second bit depth value as defined in the method according to the first aspect and its implementations forms.
- the “obtained first luma bit depth value” is different from the “second luma bit depth value”.
- BitDepthY i.e. the second luma bit depth value is the real bit depth.
- Fig. 7 shows a flow diagram illustrating steps of a decoding method 700 according to an embodiment.
- the reconstructed block comprises a plurality of pixels (samples).
- the method 700 comprises the following steps: receiving 710 a bitstream; obtaining 730 from the bitsream, a bit depth value of luma samples of a current picture; and performing 750 an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table(s) based on the obtained luma bit depth value, or performing 750 an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture based on a formula which is associated with the luma bit depth value.
- the step 750 may be introduced in details as shown in Fig. 9.
- the method 700 can be performed by the decoding apparatus as shown in FIG. 2 or 11 .
- Detailed information 300, 300’ of FIG. 3A or 3B may be also applied to method 700 as shown in Fig. 7.
- Fig. 8 shows a flow diagram illustrating steps of an encoding method 800 according to an embodiment.
- the reconstructed block comprises a plurality of pixels (samples).
- the method 800 comprises the following steps: determining 810 a bit depth value of luma samples of a current picture; performing 830 an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table(s) based on the obtained luma bit depth value, or performing 830 an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture based on a formula which is associated with the luma bit depth value.
- the step 830 may be introduced in details as shown in Fig. 9.
- the method 800 can be performed by the encoding apparatus as shown in Fig.1 or 10. Detailed information 300, 300’ of FIG. 3A or 3B may be also applied to method 800 as shown in Fig. 8.
- the bit depth value of luma samples of a current picture is obtained from an encoder configuration file which is specified based on specific use case and determined by the user.
- Fig. 9 shows a flow diagram illustrating steps of a corresponding (in-loop or post) filtering method 1300 according to an embodiment.
- the reconstructed block comprises a plurality of pixels (samples).
- the method 1300 comprises the following steps: obtaining 1310 an extended reconstructed block by extending the current reconstructed block to include one or more padding samples obtained based on at least one sample from the current reconstructed block; loading 1320 a current pixel of the extended reconstructed block(orthe current reconstructed block) and its neighboring pixels of the current pixel to a linear buffer according to a predefined scan template; obtaining 1330 Hadamard spectrum components (fHad) by performing Hadamard transform for pixels in the linear buffer; obtaining 1340 filtered Hadamard spectrum components (fHadFilt) based on the obtained Hadamard spectrum components by using the lookup table(s) based on the obtained luma bit depth value, wherein an input of the lookup table(s) is normalized to a first
- step 1310 is an optional step.
- the disclosure describes an in-loop filter for lossy video codec which performs filtering of a reconstructed block from a reconstructed frame.
- the reconstructed frame is divided into a set of rectangular macro-blocks (CU blocks).
- each reconstructed macro-block (reconstructed CU block) is filtered in a frequency domain independently from other reconstructed macro-blocks, in particular independently from the filtering of the other reconstructed macro-blocks.
- the filter can also be applied after transform and reconstruction, and the filtered result is used both for output as well as for spatial and temporal prediction.
- the filtering process operates with input video signal with various bit depth in an integer range from 8 to 16 bits inclusively.
- the method 1300 can be performed by the encoding apparatus as shown in Fig.1 or 10 and the decoding apparatus as shown in FIG. 2 or 11 .
- Detailed information 300, 300’ of FIG. 3A or 3B may be also applied to method 1300 as shown in Fig. 9.
- the filtering method can be used in image encoding or image decoding.
- the bit depth value of luma samples of a current picture is obtained from an encoder configuration file which is specified based on specific use case and determined by the user.
- the present invention allows to support various bit depth video signal by HTDF post reconstruction filter by modifying filtering formula without changes in the given LUTs.
- the HTDF filtering formula (*) is modified to allow scaling of the input value (fHad[ i ]) and further clearly specify cases of each filtering option usage in the following way:
- 8 bits video signal representation support is added to the current design of HTDF filtering process. It is proposed to use a modified formula (**) as given below, if the bit depth is set to 8 bit.
- bitDepthy is a variable represent internal bitdepth of luma component of the video signal internal representation, which can be any integer number from 8 to 16 inclusively.
- filtering formula (**) can be modified in order to support variable bit depths in the following way.
- bitDepth Y corresponds to the second bit depth value and is the real luma bit depth value.
- the four abovementioned embodiments allow utilize HTDF beneficial effect for a wide range or different bit depths without storing any additional LUTs specifically designed for each supported bit depth value.
- the example of post-reconstruction filter process is as follows.
- nCbW and nCbH specifying the width and the height of the current luma coding block
- the output of this process is the modified reconstructed picture sample array S L .
- An (nCbW + 2)x(nCbH + 2) array of padded reconstructed luma samples recSamplesPad, with location ( xCb, yCb ) specifying the top-left sample of the current luma block, size ( nCbW, nCbH ) of the current block and array S L specifying the reconstructed picture sample array as inputs.
- inFilt[ i ] recSamplesPad[ x + scanTmpl[ i ][ 1 ] ][ y + scanTmpl[ i ][ 0 ] ]
- array scanTmpl is equal to ⁇ ( 0, 0 ), ( 0, 1 ), ( 1 , 0 ), ( 1 , 1 ) ⁇
- fHad[ 0 ] inFilt[ 0 ] + inFilt[ 2 ] + inFilt[ 1 ] + inFilt[ 3 ]
- fHad[ 1 ] inFilt[ 0 ] + inFilt[ 2 ] - inFilt[ 1 ] - inFilt[ 3 ]
- fHad[ 2 ] inFilt[ 0 ] - inFilt[ 2 ] + inFilt[ 1 ] - inFilt[ 3 ]
- fHad[ 3 ] inFilt[ 0 ] - inFilt[ 2 ] + inFilt[ 1 ] + inFilt[ 3 ]
- Input to this process is a luma quantization parameter Qp Y of the current block.
- Outputs of this process are:
- the look-up-table and corresponding parameters used for filtering of the block are selected from the set of the look-up-tables based on luma quantization parameter Qp Y .
- a corresponding neighboring block is available for a current block in the case that the current and neighboring blocks belonging to same slice, tile, tile group. In another example, a corresponding neighboring block is not available for a current block in the case that the location of neighboring block beyond picture boundary.
- the block coding order may be regarded as block encoding and/or block decoding order which may be regarded as block reconstruction order, since both encoding and decoding apply reconstruction to blocks.
- the block reconstruction order corresponds to block scanning order, i.e. to the order in which the blocks of a picture are scanned for processing (encoding, decoding, reconstruction).
- neighboring samples availability checking for padding process may be different for filtering of inter and intra blocks. For inter blocks it may further check whether available samples belong to intra or inter predicted blocks. If samples belong to inter predicted blocks, it can be used for padding, otherwise it cannot be used for padding. It may be beneficial for some hardware implementation when intra and inter blocks are reconstructed in different pipelines with using of different operating memory. Using neighboring samples coming only from inter blocks during inter blocks post-reconstruction filtering allows to avoid dependency between different pipelines, that simplifies logic and speedups overall processing pipeline.
- FIG. 12 is a block diagram showing a content supply system 3100 for realizing content distribution service.
- This content supply system 3100 includes capture device 3102, terminal device 3106, and optionally includes display 3126.
- the capture device 3102 communicates with the terminal device 3106 over communication link 3104.
- the communication link may include the communication channel 13 described above.
- the communication link 3104 includes but not limited to WIFI, Ethernet, Cable, wireless (3G/4G/5G), USB, or any kind of combination thereof, or the like.
- the capture device 3102 generates data, and may encode the data by the encoding method as shown in the above embodiments.
- the capture device 3102 may distribute the data to a streaming server (not shown in the Figures), and the server encodes the data and transmits the encoded data to the terminal device 3106.
- the capture device 3102 includes but not limited to camera, smart phone or Pad, computer or laptop, video conference system, PDA, vehicle mounted device, or a combination of any of them, or the like.
- the capture device 3102 may include the source device 12 as described above.
- the video encoder 20 included in the capture device 3102 may actually perform video encoding processing.
- the data includes audio (i.e., voice)
- an audio encoder included in the capture device 3102 may actually perform audio encoding processing.
- the capture device 3102 distributes the encoded video and audio data by multiplexing them together.
- the encoded audio data and the encoded video data are not multiplexed.
- Capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106 separately.
- the terminal device 310 receives and reproduces the encoded data.
- the terminal device 3106 could be a device with data receiving and recovering capability, such as smart phone or Pad 3108, computer or laptop 3110, network video recorder (NVR)/ digital video recorder (DVR) 3112, TV 3114, set top box (STB) 3116, video conference system 3118, video surveillance system 3120, personal digital assistant (PDA) 3122, vehicle mounted device 3124, or a combination of any of them, or the like capable of decoding the above-mentioned encoded data.
- the terminal device 3106 may include the destination device 14 as described above.
- the encoded data includes video
- the video decoder 30 included in the terminal device is prioritized to perform video decoding.
- an audio decoder included in the terminal device is prioritized to perform audio decoding processing.
- the terminal device can feed the decoded data to its display.
- NVR network video recorder
- DVR digital video recorder
- TV 3114 TV 3114
- PDA personal digital assistant
- vehicle mounted device 3124 the terminal device can feed the decoded data to its display.
- NVR network video recorder
- DVR digital video recorder
- TV 3114 TV 3114
- PDA personal digital assistant
- vehicle mounted device 3124 the terminal device can feed the decoded data to its display.
- a terminal device equipped with no display for example, STB 3116, video conference system 3118, or video surveillance system 3120, an external display 3126 is contacted therein to receive and show the decoded data.
- the picture encoding device or the picture decoding device can be used.
- FIG. 13 is a diagram showing a structure of an example of the terminal device 3106.
- the protocol proceeding unit 3202 analyzes the transmission protocol of the stream.
- the protocol includes but not limited to Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG-DASH, Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any kind of combination thereof, or the like.
- RTSP Real Time Streaming Protocol
- HTTP Hyper Text Transfer Protocol
- HLS HTTP Live streaming protocol
- MPEG-DASH Real-time Transport protocol
- RTP Real-time Transport protocol
- RTMP Real Time Messaging Protocol
- the encoded audio data and the encoded video data are not multiplexed.
- the encoded data is transmitted to video decoder 3206 and audio decoder 3208 without through the demultiplexing unit 3204.
- video elementary stream (ES), audio ES, and optionally subtitle are generated.
- the video decoder 3206 which includes the video decoder 30 as explained in the above mentioned embodiments, decodes the video ES by the decoding method as shown in the above-mentioned embodiments to generate video frame, and feeds this data to the synchronous unit 3212.
- the audio decoder 3208 decodes the audio ES to generate audio frame, and feeds this data to the synchronous unit 3212.
- the video frame may store in a buffer (not shown in FIG. 15) before feeding it to the synchronous unit 3212.
- the audio frame may store in a buffer (not shown in FIG. 15) before feeding it to the synchronous unit 3212.
- the synchronous unit 3212 synchronizes the video frame and the audio frame, and supplies the video/audio to a video/audio display 3214.
- the synchronous unit 3212 synchronizes the presentation of the video and audio information.
- Information may code in the syntax using time stamps concerning the presentation of coded audio and visual data and time stamps concerning the delivery of the data stream itself.
- the subtitle decoder 3210 decodes the subtitle, and synchronizes it with the video frame and the audio frame, and supplies the video/audio/subtitle to a video/audio/subtitle display 3216.
- the present invention is not limited to the above-mentioned system, and either the picture encoding device or the picture decoding device in the above-mentioned embodiments can be incorporated into other system, for example, a car system.
- FIG. 14 shows a decoder 1400, comprising one or more processors 1401 and a non- transitory computer-readable storage medium1402 coupled to the processors 1401 and storing programming for execution by the processors 1401 , wherein the programming, when executed by the processors 1401 , configures the decoder to carry out the method according to the first aspect or any one of the implementation forms thereof.
- Fig. 15 is a block diagram of a decoder according to another embodiment.
- the decoder 1500 comprises receiving means 1501 for receiving a bitstream; obtaining means 1502 for obtaining from the bitstream, a first luma bit depth value of luma samples of a current picture; and filtering means 1503 for performing an Hadamard Transform Domain Filtering post reconstruction filtering of a block of the current picture by using a lookup table based on the obtained first luma bit depth value.
- the invention relates to a method of coding implemented by a decoding device, comprising: receiving a bitstream; obtaining from the bitsream, a bit depth value of luma samples of a current picture(e.g. a luma bit depth value); performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table(s) based on the obtained luma bit depth value.
- HTDF Hadamard Transform Domain Filtering
- the lookup table(s) are predefined or designed for a first bit depth representation.
- the first bit depth representation is 10 bits bit depth representation.
- the lookup table(s) is used for a plurality of luma bit depth values.
- the lookup table(s) is used for 10 bits bit depth, 8 bits bit depth, 12 bits bit depth and/or other bit depth value.
- the performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using the lookup table(s) based on the obtained luma bit depth value comprises: when the luma bit depth value is not the first bit depth representation, performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using the lookup table(s), wherein an input of the lookup table(s) is normalized to a first bit depth representation, and an output of the lookup table(s) is normalized to the obtained luma bit depth value; and/or when the luma bit depth value is the first bit depth representation, performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using the lookup table(s), wherein an input of the lookup table(s) is the first bit depth representation, and no normalization is required for an output of the lookup table(s).
- HTDF Hadamard Transform Domain Filtering
- an input (e.g. LUT element index) of the lookup table(s) is normalized to the first bit depth representation (such as 10 bits bit depth) and an output (e.g. the filtered Hadamard spectrum components) of the lookup table(s) is normalized to the obtained internal bit depth value(such as 8 bits bit depth or 12 bits bit depth).
- the step of performing an HTDF post reconstruction filtering of a block of the current picture comprises: when at least one condition is satisfied, performing an HTDF post reconstruction filtering of a block of the current picture using one or more predefined HTDF lookup tables based on the formula which is associated with the luma bit depth value; or when at least one condition is satisfied, performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using lookup table(s) based on the obtained luma bit depth value.
- HTDF Hadamard Transform Domain Filtering
- the condition is based on a comparison result between a scaled Hadamard spectrum component and a predefined threshold value; or when the condition is satisfied, comprises: when a scaled Hadamard spectrum component is less than a predefined threshold value.
- the step of performing an HTDF post reconstruction filtering of a block of the current picture comprises: loading a current pixel of a reconstructed block or an extended reconstructed block and its neighboring pixels of the current pixel according to a predefined scan template; obtaining Hadamard spectrum components(fHad) by performing a Hadamard transform for the pixels; obtaining filtered Hadamard spectrum components (fHadFilt) based on the obtained Hadamard spectrum components by using the lookup table(s) based on the obtained luma bit depth value, wherein an input of the lookup table(s) is normalized to a first bit depth representation(e.g.
- the step of obtain filtered Hadamard spectrum components comprises:
- the lookup table(s) is obtained based on a quantization parameter (Qp) of the block.
- the invention relates to a method of coding implemented by a decoding device.
- the method includes: receiving a bitstream;
- bit depth value e.g. a luma bit depth value
- a bit depth value of luma samples of a current picture(or sequence) or deriving a bit depth value of luma samples of a current picture from the bitsream
- HTDF Hadamard Transform Domain Filtering
- the performing an HTDF post reconstruction filtering may include performing an HTDF post reconstruction filtering of the block by predefined lookup table(s) usage based on the bit depth value which corresponds to a respective formula(such as, among respective formulas).
- the (predefined) one or more lookup table(s) are designed for 10 bits internal bit depth representation, however the one or more lookup tables (e.g. a same LUT or a same set of LUTs) may be used for a plurality of luma bit depth values, such as, not only 10 bits internal bit depth, but also 8 bits internal bit depth and/or 12 bits internal bit depth and other bit depth value.
- an input (e.g. LUT element index) of the lookup table(s) is normalized to 10bits representation and an output (e.g. the filtered Hadamard spectrum components) of the lookup table(s) is normalized to the obtained internal bit depth value.
- the step of performing an HTDF post reconstruction filtering of a block of the current picture comprises: loading a current pixel of a reconstructed block and its neighboring pixels of the current pixel according to a predefined scan template; obtaining Hadamard spectrum components(fHad) by performing a Hadamard transform for the pixels; obtaining filtered Hadamard spectrum components (fHadFilt) based on the obtained Hadamard spectrum components by using the lookup table(s) according to the formula which is associated with the luma bit depth value; or obtaining filtered Hadamard spectrum components (fHadFilt) based on the obtained Hadamard spectrum components by using the lookup table(s), wherein an input (e.g.
- LUT element index) of the lookup table(s) is normalized to a first bit depth representation ⁇ . g. 10bits representation), and an output(e.g. output filtered Hadamard spectrum components) of the lookup table(s) is normalized to the obtained luma bit depth value, to obtain the filtered Hadamard spectrum components (fHadFilt); obtaining filtered pixels by performing an inverse Hadamard transform transform for the filtered Hadamard spectrum components; and generating a filtered reconstructed block based on the filtered pixels.
- fHadFilt filtered Hadamard spectrum components
- bit_depth_luma_minus8 syntax element and decoder typically operates with BitDepthy variable defined as
- BitDepthy 8 + bit_depth_luma_minus8.
- the lookup table is obtained based on a quantization parameter (Qp) of the current reconstructed block. This facilitates efficient implementation without necessity for complex calculations.
- Qp quantization parameter
- the invention relates to a method of coding implemented by an encoding device.
- the method includes:
- bit depth value of luma samples of a current picture e.g. a luma bit depth value
- HTDF Hadamard Transform Domain Filtering
- the luma bit depth value is obtained from an encoder configuration file. Further features and implementation forms of the method according to the second example of the invention correspond to the features and implementation forms of the method according to the first example of the invention.
- the apparatus comprises:
- an entropy decoding unit 1104 configured for receiving a bitstream; and obtaining from the bitsream, a bit depth value of luma samples of a current picture;
- a post reconstruction filter 1118 configured for performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table(s) based on the obtained luma bit depth value, or performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block(i.e. a reconstructed block) of the current picture based on a formula which is associated with the luma bit depth value;
- HTDF Hadamard Transform Domain Filtering
- an implementation form of the apparatus comprises the feature(s) of the corresponding implementation form of the method according to the first example.
- the invention relates to an encoding apparatus.
- the apparatus comprises: ...
- a determining unit 1017 (not shown in figure 10), configured for determining a bit depth value of luma samples of a current picture
- a post reconstruction filter 1018 configured for performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture by using a lookup table(s) based on the obtained luma bit depth value, or performing an Hadamard Transform Domain Filtering (HTDF) post reconstruction filtering of a block of the current picture based on a formula which is associated with the luma bit depth value.
- HTDF Hadamard Transform Domain Filtering
- an implementation form of the apparatus comprises the feature(s) of the corresponding implementation form of the method according to the second example.
- the invention relates to an apparatus for decoding a video stream includes a processor and a memory.
- the memory is storing instructions that cause the processor to perform the method according to any of the preceding examples.
- the invention relates to an apparatus for encoding a video stream includes a processor and a memory.
- the memory is storing instructions that cause the processor to perform the method according to any of the preceding examples.
- a computer-readable storage medium having stored thereon instructions that when executed cause one or more processors configured to code video data is proposed.
- the instructions cause the one or more processors to perform a method according to any of the preceding examples or any possible embodiment of any of the preceding examples.
- the invention relates to a computer program product comprising program code for performing the method according to any of the above-mentioned examples, embodiments and examples when executed on a computer.
- the filter is provided allowing improving the efficiency for video coding in a wide range of applications. More specifically, the improved filter according to embodiments of the invention can be used for video compression that represents video signal with bit depth samples values from 8 bits per sample to 16 bits per same using the only one set or same set of look up tables.
- 7 / 4 and -71 -4 are truncated to 1 and -7 / 4 and 7 / -4 are truncated to -1.
- ⁇ Used to denote division in mathematical equations where no truncation or rounding is intended. Used to denote division in mathematical equations where no truncation or rounding is intended.
- na When a relational operator is applied to a syntax element or variable that has been assigned the value "na” (not applicable), the value "na” is treated as a distinct value for the syntax element or variable. The value “na” is considered not to be equal to any other value.
- Bit-wise "or" When operating on integer arguments, operates on a two's complement representation of the integer value. When operating on a binary argument that contains fewer bits than another argument, the shorter argument is extended by adding more significant bits equal to 0.
- x y..z x takes on integer values starting from y to z, inclusive, with x, y, and z being integer numbers and z being greater than y.
- Asin( x ) the trigonometric inverse sine function, operating on an argument x that is in the range of -1 .0 to 1 .0, inclusive, with an output value in the range of - ⁇ -S-2 to ⁇ -S-2, inclusive, in units of radians
- Ceil( x ) the smallest integer greater than or equal to x.
- Clip1 Y ( x) Clip3( 0, ( 1 « BitDepth Y ) - 1 , x )
- Clip1c( x) Clip3( 0, ( 1 « BitDepthc ) - 1 , x)
- Cos( x ) the trigonometric cosine function operating on an argument x in units of radians.
- Round( x ) Sign( x ) * Floor( Abs( x ) + 0.5 )
- the table below specifies the precedence of operations from highest to lowest; a higher position in the table indicates a higher precedence.
- statement 1 If one or more of the following conditions are true, statement 1 :
- embodiments of the invention have been primarily described based on video coding, it should be noted that embodiments of the coding system 10, encoder 20 and decoder 30 (and correspondingly the system 10) and the other embodiments described herein may also be configured for still picture processing or coding, i.e. the processing or coding of an individual picture independent of any preceding or consecutive picture as in video coding.
- inter-prediction units 244 (encoder) and 344 (decoder) may not be available in case the picture processing coding is limited to a single picture 17. All other functionalities (also referred to as tools or technologies) of the video encoder 20 and video decoder 30 may equally be used for still picture processing, e.g.
- residual calculation 204/304 transform 206, quantization 208, inverse quantization 210/310, (inverse) transform 212/312, partitioning 262/362, intra-prediction 254/354, and/or loop filtering 220, 320, and entropy coding 270 and entropy decoding 304.
- Embodiments, e.g. of the encoder 20 and the decoder 30, and functions described herein, e.g. with reference to the encoder 20 and the decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on a computer-readable medium or transmitted over communication media as one or more instructions or code and executed by a hardware-based processing unit.
- Computer- readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol.
- computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave.
- Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure.
- a computer program product may include a computer-readable medium.
- such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.
- any connection is properly termed a computer-readable medium.
- a computer-readable medium For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium.
- DSL digital subscriber line
- Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
- processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry.
- DSPs digital signal processors
- ASICs application specific integrated circuits
- FPGAs field programmable logic arrays
- processors may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein.
- the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques could be fully implemented in one or more circuits or logic elements.
- the techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set).
- IC integrated circuit
- a set of ICs e.g., a chip set.
- Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit or provided by a collection of interoperative hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Processing (AREA)
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
BR112023003118A BR112023003118A2 (en) | 2020-08-18 | 2021-08-18 | METHOD AND APPARATUS FOR DOMAIN FILTERING BY HADAMARD TRANSFORMATION OF VIDEO SIGNAL WITH VARIOUS BIT DEPTH |
EP21769824.0A EP4189956A1 (en) | 2020-08-18 | 2021-08-18 | Method and apparatus for hadamard transform domain filtering of the video signal with various bit depth |
CN202180050924.0A CN116830575A (en) | 2020-08-18 | 2021-08-18 | Method and apparatus for hadamard transform domain filtering of video signals having different bit depths |
US18/172,174 US20230199227A1 (en) | 2020-08-18 | 2023-02-21 | Method and apparatus for hadamard transform domain filtering of the video signal with various bit depth |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
RU2020000439 | 2020-08-18 | ||
RUPCT/RU2020/000439 | 2020-08-18 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/172,174 Continuation US20230199227A1 (en) | 2020-08-18 | 2023-02-21 | Method and apparatus for hadamard transform domain filtering of the video signal with various bit depth |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022039635A1 true WO2022039635A1 (en) | 2022-02-24 |
Family
ID=77739116
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/RU2021/050266 WO2022039635A1 (en) | 2020-08-18 | 2021-08-18 | Method and apparatus for hadamard transform domain filtering of the video signal with various bit depth |
Country Status (5)
Country | Link |
---|---|
US (1) | US20230199227A1 (en) |
EP (1) | EP4189956A1 (en) |
CN (1) | CN116830575A (en) |
BR (1) | BR112023003118A2 (en) |
WO (1) | WO2022039635A1 (en) |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020055288A2 (en) * | 2018-09-16 | 2020-03-19 | Huawei Technologies Co., Ltd. | Apparatus and method for filtering in video coding with look-up table selected based on bitstream information |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10104385B2 (en) * | 2014-04-17 | 2018-10-16 | Qualcomm Incorporated | Signaling reference layers for 3D color prediction for color gamut scalability |
SG11202013198PA (en) * | 2018-07-02 | 2021-01-28 | Huawei Tech Co Ltd | Apparatus and method for filtering in video coding |
US20200288158A1 (en) * | 2019-03-05 | 2020-09-10 | Qualcomm Incorporated | Prediction-domain filtering for video coding |
EP3734973B1 (en) * | 2019-05-02 | 2023-07-05 | Sick IVP AB | Method and encoder relating to encoding of pixel values to accomplish lossless compression of a digital image |
-
2021
- 2021-08-18 WO PCT/RU2021/050266 patent/WO2022039635A1/en active Application Filing
- 2021-08-18 BR BR112023003118A patent/BR112023003118A2/en unknown
- 2021-08-18 EP EP21769824.0A patent/EP4189956A1/en active Pending
- 2021-08-18 CN CN202180050924.0A patent/CN116830575A/en active Pending
-
2023
- 2023-02-21 US US18/172,174 patent/US20230199227A1/en active Pending
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020055288A2 (en) * | 2018-09-16 | 2020-03-19 | Huawei Technologies Co., Ltd. | Apparatus and method for filtering in video coding with look-up table selected based on bitstream information |
Non-Patent Citations (1)
Title |
---|
"Text of ISO/IEC FDIS 23094-1 Essential Video Coding", no. n19229, 30 April 2020 (2020-04-30), XP030285492, Retrieved from the Internet <URL:http://phenix.int-evry.fr/mpeg/doc_end_user/documents/130_Alpbach/wg11/w19229.zip w19229_ISO_IEC_FDIS_23094-1.pdf> [retrieved on 20200430] * |
Also Published As
Publication number | Publication date |
---|---|
US20230199227A1 (en) | 2023-06-22 |
BR112023003118A2 (en) | 2023-04-04 |
EP4189956A1 (en) | 2023-06-07 |
CN116830575A (en) | 2023-09-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR102431537B1 (en) | Encoders, decoders and corresponding methods using IBC dedicated buffers and default value refreshing for luma and chroma components | |
AU2020297260B2 (en) | Apparatus and method for filtering in video coding | |
US11375187B2 (en) | Apparatus and method for filtering in video coding with look-up table selected based on bitstream information | |
US11070799B2 (en) | Encoder, decoder and corresponding methods for intra prediction | |
EP4344202A2 (en) | An encoder, a decoder and corresponding methods for performing chroma deblocking for blocks which use joint chroma coding | |
JP2023103292A (en) | Encoder, decoder and corresponding intra prediction method | |
JP7366149B2 (en) | An encoder, decoder, and corresponding method for harmonizing matrix-based intra-prediction and quadratic transform core selection | |
US11388422B2 (en) | Encoder, a decoder and corresponding methods related to intra prediction mode | |
JP7553007B2 (en) | METHOD AND APPARATUS FOR CHROMA INTRA PREDICTION IN VIDEO CODING - Patent application | |
JP2023162243A (en) | Encoder, decoder, and corresponding method using high-level flag with dct2 enabled | |
JP7397878B2 (en) | Method and apparatus for intra-subpartition coding mode | |
KR102475963B1 (en) | Encoders, decoders and corresponding methods used for the conversion process | |
KR20220065880A (en) | Use of DCT-based interpolation filters and enhanced bilinear interpolation filters in affine motion compensation | |
US20230199227A1 (en) | Method and apparatus for hadamard transform domain filtering of the video signal with various bit depth | |
KR20210145824A (en) | Method and apparatus for intra prediction without division | |
EP3991435A1 (en) | Method and apparatus of sample adaptive offset in-loop filter with application region size constraint | |
EP3895418A1 (en) | An encoder, a decoder and corresponding methods for local illumination compensation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21769824 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202180050924.0 Country of ref document: CN |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112023003118 Country of ref document: BR |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202317013847 Country of ref document: IN |
|
ENP | Entry into the national phase |
Ref document number: 2021769824 Country of ref document: EP Effective date: 20230301 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 112023003118 Country of ref document: BR Kind code of ref document: A2 Effective date: 20230217 |