GB2539210A - Decoding method and corresponding device - Google Patents

Abstract

Decoding a picture parameter set used for decoding images from a bitstream. In a first embodiment a colour component parameter which indicates whether the image is monochrome and is involved in at least two steps of the decoding method is decoded and at least one step of the decoding method is skipped or amended if the parameter indicates that the image is monochrome. Preferably, this step is skipped rather than amended and relates to the application of a residual adaptive colour transform. In a second embodiment the decoding method includes a step related to the residual adaptive colour transform and a step for defining a palette predictor initialiser. The defining step is processed based on the value of a colour component parameter which indicates whether or not the image is monochrome and the colour component parameter is inferred from the execution of the residual adaptive colour transform. These embodiments rely on a redundancy in the parameters encoded in the extension to the picture parameter set defined in the HEVC Screen Content Coding draft specification A third embodiment has a step of initialising a palette predictor, the step being governed by rules when switching between images based on the number of colour components in each image.

Description

DECODING METHOD AND CORRESPONDING DEVICE

FIELD OF THE INVENTION

The present invention concerns a method and a device for processing at least one image, e.g. for encoding or decoding the images into or from a bitstream, using a palette prediction mode. It particularly concerns the palette mode encoding as introduced in HEVC Screen Content Coding (SCC) Extension.

BACKGROUND OF THE INVENTION

It applies more particularly to a mode of coding where blocks of pixels are each encoded based on a respective block of indexes encoded with or built from a so-called palette.

A palette in this document is defined as a look up table having entries associating an index with a value of a pixel. Generally each entry comprises one or three elements as mentioned later. Each element concerns the pixel value for a color component. For example, if the image is monochrome each entry of the palette comprises one element for the unique color component.

In other words, typically, but not necessarily, the value of a pixel is constituted by the value of each colour component associated with the pixel, resulting in a colour palette. However, the value of a pixel may be made of a single pixel component (named "element"), resulting in a monochrome palette.

This mode of encoding a block of pixel is generally referred to as Palette coding mode. It is contemplated to adopt this mode, for example, in the Screen Content Coding (SCC) Extension of the High Efficiency Video Coding international standard (see 25 document JCTVC-S1005).

When encoding an image in a video sequence, the image is first divided into coding entities (also known as "coding structures") of pixels of equal size referred to as Coding Tree Blocks (CTBs). The CTBs may be grouped into other coding structures having a higher hierarchical level, such as slices and/or tiles. In other words, the image is recursively divided into hierarchical coding structures or coding entities.

The size of a Coding Tree Block is typically 64 by 64 pixels. Each Coding Tree Block may then be broken down into a hierarchical tree of smaller blocks whose size may vary and which are the actual blocks of pixels to encode. These smaller blocks to encode are referred to as Coding Units (CUs).

The encoding of a particular Coding Unit involves competition between predictive coding modes, including the well-known INTRA coding mode, the well-known INTER coding mode, and the Palette coding mode.

With the Palette coding mode, it is possible to define a representative block for a given Coding Unit as a block of indexes from a palette: for each pixel location in the Coding Unit, the said representative block contains the index associated with a pixel value in the Palette which is the closest to the value of the pixel having the same location (i.e. colocated) in the coding unit. However, this palette-based algorithm of selecting the closest palette entry is encoder-only in HEVC SCC: there is no need to know said algorithm in order to parse or decode a bitstream. Typically, closest means with the lowest distance using a particular metric distance such as the sum of absolute, or the square of, differences of component values. In particular, in case of lossless coding, this means the palette entry should be selected as equal to the pixel by the encoder. In the following, "correspond to" or "match" is used to mean either "is equal" when in lossless coding, or "is the closest" otherwise.

In the recent version of HEVC SCC, no residual between the original pixel block and the corresponding palette-based representative pixel block is provided. To avoid high quality decreasing in the encoded image, an "escape-coded" feature has been introduced to encode the pixels, the values of which do not match a pixel value of an entry of the Palette. This means, in lossless coding, that no palette entry is equal to the pixel value. In such case, a specific index in the Palette is used to signal an "escape-coded" pixel; and the quantized value itself of the escape-coded pixel is directly encoded in the bitstream, the quantization depending on a quantizer step transmitted at the CU-level. In case of lossless coding, the quantizer step is 0, meaning no quantization. The quantization is what is defined in the HEVC standard as the transform-bypass quantization, and the quantized values are encoded using truncated binary codes.

The Palette coding mode thus uses a current palette to build a block of indexes representative of a current coding unit or block of pixels. Entry indexes in the Palette are also known as "levels".

When using the Palette mode, the palette and the block of indexes or "levels" is often transmitted in the bitstream encoding the image. This represents a high cost of signalling because a palette, which may comprise tens of entries, needs to be transmitted for each Coding Unit.

In Applicant's contribution in JCT-VC (No. JCTVC-Q0063 entitled "AhG10: palette predictor stuffing", 17th Meeting: Valencia, ES, 27 March -4 April 2014), it has been proposed to predict the current Palette for a current Coding Unit using a palette predictor, for instance the last Palette used (for the last processed Coding Unit). This approach aims at reducing the coding costs, since a palette is no longer fully explicitly transmitted for each Coding Unit.

However, some coding specificities may break the palette prediction scheme throughout the Coding Units of the image. This is the case of coding structures like Slices and Tiles. As a consequence, in Applicant's contribution in JCT-VC (No. JCTVC-T1005 entitled "HEVC Screen Content Coding Draft Text 3", 21st Meeting: Geneva, CH, 19 -26 February 2015), it has been proposed to transmit a palette predictor initializer in the Picture Parameter Set extension for the HEVC SCC Extension. According to a particular embodiment, when the entries of that initializer are monochrome, the monochrome context and/or the bitdepths of the entries components are signalled.

However signalling the colour format introduces either redundancies or potentially some incompatibilities in parameter values.

SUMMARY OF THE INVENTION

The present invention has been devised to overcome all or part of the foregoing drawbacks. In particular, it seeks to simplify the signalling of configuration or used tools in the presence of the palette mode.

An aspect of the invention relates to a method of decoding a picture parameter set used for decoding images from a bitstream, the image comprising pixels having at least one color-component, the method being broken down into several steps. The method comprises: decoding one color-component parameter defined in the picture parameter set and involved in at least two steps of the decoding method, indicating if said image is monochrome, and; skipping or amending at least one step of the decoding method if the decoded color-component parameter indicates that the image is monochrome.

This aspect of the invention aims at proposing a syntax without any redundancy which used to be source of faults. This aspect of the invention is more efficient and prevents contradictory configurations.

In an embodiment at least two different steps are either skipped or amended, at least one of the steps comprising decoding pixel values having been encoded using a palette mode.

In an embodiment, one step comprises determining the number of elements for at least one entry or each entry of a palette predictor initializer. As defined above each element concerns the pixel value for a color component. For example, if the image is monochrome each entry of the palette comprises one element for the unique color component In an embodiment, each entry of a palette predictor initializer is a singleton if the decoded color-component parameter is set to a predetermined value, else a triplet.

In an embodiment, one of the steps is related to the residual adaptive colour transform, which is skipped if the color-component parameter indicates that the image is monochrome.

In an embodiment, the parameter is a flag taking the value "1" when the image is monochrome.

Another aspect of the invention relates to a method of decoding a picture parameter set used for decoding images from a bitstream, the image comprising pixels having at least one color-component, said image being encoded by using one mode among a plurality of modes including the palette mode. The method comprises: -a step related to the residual adaptive colour transform; and -a step for defining a palette predictor initializer, the defining step being processed based on the value of a color-component parameter indicating if the image is 20 monochrome or not.

The value of the color-component parameter is inferred from the execution of the residual adaptive colour transform.

This other aspect of the invention aims at proposing a syntax without any redundancy which used to be source of faults. This aspect of the invention is more efficient and prevents contradictory configurations.

In an embodiment, if the residual adaptive colour transform is executed, the value of the color-component parameter is inferred to indicate that the image is not monochrome.

In an embodiment, the color-component parameter is a flag whose value is inferred to be "0" if the residual adaptive colour transform is executed.

Another aspect of the invention relates to a method of decoding a picture parameter set used for decoding images from a bitstream, the image comprising pixels having at least one color-component, at least one image being encoded by using the palette mode, the method comprising a step for initializing a palette predictor with a palette predictor initializer having the same number of entries, the palette predictor initializer and the palette predictor having one or more elements per entry.

The initialization of the palette predictor by the palette predictor initializer is governed by predetermined rules when switching from an image having pixels with a given number of color-components to an image having another number color-components.

In an embodiment, a predetermined rule comprises for at least one entry of the palette predictor, setting at least two of the elements to the same element's value of the palette predictor initializer.

In an embodiment, a predetermined rule comprises for at least one entry of the palette predictor, setting at least one of the elements to a predetermined default value.

In an embodiment, the predetermined rule is applied when switching from an image having one color-component to an image having three color-components.

According to another aspect of the invention there is provided a device for decoding a picture parameter set related to an image from a bitstream, said device being configured to implement a decoding method according to one of the embodiments described above.

According to another aspect of the invention there is provided a non-transitory computer-readable medium storing a program which, when executed by a microprocessor or computer system in a device for decoding at least one image, causes the device to perform the method according to one of the embodiments described above.

In other aspects, the invention relates to a method of and apparatus for encoding a sequence of images into a bitstream, wherein said bitstream is capable of being decoded by the decoding methods and devices of the above described aspects. Encoding may include encoding in said bitstream a color-component parameter indicating whether an image being decoded is a monochrome image or not. When the color-component parameter indicates that the image is monochrome this indicates to a decoder that at least one step of the decoding method that uses said color-component parameter can be skipped. In an embodiment, it is determined whether the property of the image being monochrome or not can be inferred from other parameters and, if so, then the color-component parameter (e.g. the flag) is not directly encoded in the bitstream. For example, whether the image is monochrome or not may be inferred from whether a residual adaptive colour transform is executed. The color-component parameter is preferably defined in a picture parameter set used for decoding images and preferably takes the form of a flag. Preferably, at least portions of the image are encoded using a palette mode.

At least parts of the methods according to the invention may be computer implemented. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system". Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

Since the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described, by way of example only, and with reference to the following drawings in which: Figure 1 illustrates the HEVC encoder architecture; Figure 2 illustrates the HEVC decoder architecture; Figure 3 illustrates the concept of the causal area; Figure 4 illustrates the Coding Tree Block splitting into Coding Units and the scan order decoding of these Coding Unit; Figure 5 illustrates the principle of Palette mode at the decoder side under investigation in the SECC Extension of HEVC, together with prediction of said palette; Figure 6 illustrates an example of a coding unit with its corresponding block of levels and the associated palette; Figure 7 illustrates the same block of levels and the set of syntax elements used for the encoding of this block of levels; Figure 8 illustrates the decoding process of the syntax elements relating to the Palette coding mode; Figure 9 illustrates the reconstruction process to build the block of levels at the decoding side; Figure 10 illustrates an example of signalling the palette prediction initializer, as proposed in the JCTVC -T1005 document, in the PPS SCC extension, as written by the encoder and parsed by the decoder; Figure 11 illustrates a first embodiment of the invention; Figure 12 illustrates a second embodiment of the invention; and Figure 13 is a schematic block diagram of a computing device for implementation of one or more embodiments of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

Figure 1 illustrates the HEVC encoder architecture. In the video encoder, an original sequence 101 is divided into blocks of pixels 102. A coding mode is then assigned to each block. There are two families of coding modes typically used in HEVC: the modes based on spatial prediction or INTRA modes 103 and the modes based on temporal prediction or INTER modes based on motion estimation 104 and motion compensation 105. An extension of HEVC being currently designed, known as HEVC SCC, adds additional coding modes, in particular the Palette coding mode, which competes with INTRA and INTER coding modes to encode blocks of pixels. This Palette coding mode is described in more details below, in particular with reference to Figures 5 to 9. One skilled in the art may also obtain details about the Palette coding mode in document JCTVC-S1005 (HEVC Screen Content Coding Draft Text 2), the latest as of writing. To be noted that the invention is not limited to the implementation of the Palette coding mode as described in the HEVC SCC extension, but may apply to any palette predictive scheme.

An INTRA Coding Unit is generally predicted from the encoded pixels at its causal border by a process called INTRA prediction.

Temporal prediction of an INTER coding mode first consists in finding in a previous or future frame called the reference frame 116 the reference area of which is the closest to the Coding Unit, in a motion estimation step 104. This reference area constitutes the predictor block. Next this Coding Unit is predicted using the predictor block to compute the residue in a motion compensation step 105.

In both cases, spatial and temporal prediction, a residual is computed by subtracting the Coding Unit from the original predictor block.

In the INTRA prediction, a prediction direction is encoded. In the temporal prediction, at least one motion vector is encoded. However, in order to further reduce the bitrate cost related to motion vector encoding, a motion vector is not directly encoded.

Indeed, assuming that motion is homogeneous, it is particularly advantageous to encode a motion vector as a difference between this motion vector, and a motion vector in its surroundings. In the H.264/AVC coding standard for instance, motion vectors are encoded with respect to a median vector computed between 3 blocks located above and on the left of the current block. Only a difference, also called residual motion vector, computed between the median vector and the current block motion vector is encoded in the bitstream. This is processed in module "Mv prediction and coding" 117. The value of each encoded vector is stored in the motion vector field 118. The neighbouring motion vectors, used for the prediction, are extracted from the motion vector field 118.

Next, the mode optimizing the rate distortion performance is selected in module 106. In order to further reduce the redundancies, a transform, typically a DCT, is applied to the residual block in module 107, and a quantization is applied to the coefficients in module 108. The quantized block of coefficients is then entropy coded in module 109 and the result is inserted into the bitstream 110.

The encoder then performs a decoding of the encoded frame for the future motion estimation in modules 111 to 116. This is a decoding loop at the encoder. These steps allow the encoder and the decoder to have the same reference frames. To reconstruct the coded frame, the residual is inverse quantized in module 111 and inverse transformed in module 112 in order to provide the "reconstructed" residual in the pixel domain. According to the encoding mode (INTER or INTRA), this residual is added to the INTER predictor 114 or to the INTRA predictor 113.

Next, this first reconstruction is filtered in module 115 by one or several kinds of post filtering. These post filters are integrated into the decoding loop. This means that they need to be applied to the reconstructed frame at the encoder and decoder in order to use the same reference frames at the encoder and decoder. The aim of this post filtering is to remove compression artefacts.

The principle of an HEVC decoder has been represented in Figure 2. The video stream 201 is first entropy decoded in a module 202. The residual data are then inverse quantized in a module 203 and inverse transformed in a module 204 to obtain pixel values. The mode data are also entropy decoded and depending on the mode, an INTRA type decoding or an INTER type decoding is performed. In the case of INTRA mode, the INTRA prediction direction is decoded from the bitstream. The prediction direction is then used to locate the reference area 205. If the mode is INTER, the motion information is decoded from the bitstream 202. This is composed of the reference frame index and the motion vector residual. The motion vector predictor is added to the motion vector residual to obtain the motion vector 210. The motion vector is then used to locate the reference area in the reference frame 206. Note that the motion vector field data 211 is updated with the decoded motion vector in order to be used for the prediction of the next decoded motion vectors. This first reconstruction of the decoded frame is then post filtered 207 with exactly the same post filter as used at the encoder side. The output of the decoder is the de-compressed video 209.

Figure 3 illustrates the causal principle resulting from block-by-block encoding as in HEVC.

At a high-level, an image is divided into Coding Units that are encoded in raster scan order. Thus, when coding block 3.1, all the blocks of area 3.3 have already been encoded, and can be considered available to the encoder. Similarly, when decoding block 3.1 at the decoder, all the blocks of area 3.3 have already been decoded and thus reconstructed, and can be considered as available at the decoder. Area 3.3 is called the causal area of the Coding Unit 3.1. Once Coding Unit 3.1 is encoded, it will belong to the causal area for the next Coding Unit. This next Coding Unit, as well as all the next ones, belongs to area 3.4 illustrated as a dotted area, and cannot be used for coding the current Coding Unit 3.1. It is worth noting that the causal area is constituted by reconstructed blocks. The information used to encode a given Coding Unit is not the original blocks of the image for the reason that this information is not available at decoding. The only information available at decoding is the reconstructed version of the blocks of pixels in the causal area, namely the decoded version of these blocks. For this reason, at encoding, previously encoded blocks of the causal area are decoded to provide this reconstructed version of these blocks.

It is possible to use information from a block 3.2 in the causal area when encoding a block 3.1. In the HEVC Extension draft specifications, a displacement vector 3.5, which can be transmitted in the bitstream, may indicate this block 3.2.

Figure 4 illustrates a splitting of a Coding Tree Block into Coding Units and an exemplary scan order to sequentially process these Coding Units. In the HEVC standard, the block structure is organized by Coding Tree Blocks (CTBs). A frame contains several non-overlapped and square Coding Tree Blocks. The size of a Coding Tree Block can range in size from 64x64 to 16x16. This size is determined at sequence level. The most efficient size, in term of coding efficiency, is the largest one: 64x64. Note that all Coding Tree Blocks have the same size except for the image border, meaning that they are arranged in rows. The size of the border CTBs is adapted according to the amount of remaining pixels.

Each Coding Tree Block contains one or more square Coding Units (CU). The Coding Tree Block is split based on a quad-tree structure into several Coding Units. The processing (coding or decoding) order of each Coding Unit in the Coding Tree Block follows the quad-tree structure based on a raster scan order. Figure 5 shows an example of the processing order of Coding Units. In this figure, the number in each Coding Unit gives the processing order of each corresponding Coding Unit of this Coding Tree Block. In HEVC, several methods are used to code the different syntax elements, for example block residuals, information on predictor blocks (motion vectors, INTRA prediction directions, etc.). HEVC uses several types of entropy coding such as the Context based Adaptive Binary Arithmetic Coding (CABAC), Golomb-Rice Code, or simple binary representation called Fixed Length Coding. Most of the time a binary encoding process is performed to represent the different syntax elements. This binary encoding process is also very specific and depends on the different syntax element.

The HEVC Screen Content Coding Extension, also commonly called HEVC SCC, is an extension that is currently being drafted for the new video coding standard HEVC. It is derived from the HEVC Range Extension, also commonly called HEVC RExt. An aim of this extension is to provide additional tools to encode video sequences in particular for the 4:4:4 colour format with 8 bits of bit-depth, and possibly losslessly, containing contents such as graphical user interfaces captures, computer-graphic generated content, etc. (known as Screen Contents).

A colour image is generally made of three colour components R, G and B. These components are generally correlated, and it is very common in image and video compression to de-correlate the colour components prior to processing the images. The most common format that de-correlates the colour components is the YUV colour format.

YUV signals are typically created from RGB representation of images, by applying a linear transform to the three inputs R, G and B input frames. Y is usually called Luma component, U and V are generally called Chroma components. The term 'YCbCr' is also commonly used in place of the term 'YUV'.

Moreover some SCC's tools may be compatible with other colour formats. In particular, the palette mode has been made compatible with 4:2:0 by adjusting the 4:4:4 case which has been set to handle monochrome data by operating on a single component instead of three components. Different bitdepths can also been handled. HEVC SCC, beside lossy compression, is also able to provide a lossless encoding of the input sequences; this is to have a decoded output 209 strictly identical to the input 101. To achieve this, a number of tools have been modified or added, compared to the conventional HEVC RExt lossy codec.

Additional tools for HEVC SCC are currently being designed to efficiently encode "screen content' video sequences in addition to natural sequences. As briefly introduced above, the "screen content" video sequences refer to particular video sequences which have a very specific content corresponding to those captured from a personal computer of any other device, containing for example text, PowerPoint presentation, Graphical User Interface, tables (e.g. screen shots). These particular video sequences have quite different statistics compared to natural video sequences. In video coding, performance of conventional video coding tools, including HEVC, proves sometimes to be underwhelming when processing such "screen content".

The tools currently discussed on in HEVC SCC to process "screen content" video sequences include the Adaptive Color Transform, the Infra Block Copy mode and the Palette mode. Prototypes for these modes have shown good coding efficiency compared to the conventional method targeting natural video sequences. The present application focuses on the Palette coding mode.

The Palette coding mode of HEVC SCC is a coding mode, meaning that the information directly codes pixel data. As currently drafted, the Palette coding mode does not use residual data, but uses an "escape coding" when a pixel does not match with any entry of the palette currently used. In particular, in case of lossless coding, this means the palette entry should be selected as equal to the pixel by the encoder, or that the escape coded pixel is not quantized, the quantizer value being transmitted at the CU level.

A palette is generally represented by a table containing a finite set of N-tuple of colours, each colour being defined by its components in a given colour space (see for example 603 in Figure 6 based on YUV colour space). For example, in a typical RGB format, the palette is composed of a list of P elements of N-tuple (where N=3 for a RGB). More precisely, each element corresponds to a fixed triplet of colour components in the RGB format. Of course this is not limited to a RGB or YUV colour format. Any other colour format can be represented by a palette and can use a smaller or a higher number of colour components, meaning that N may be different from 3.

At the encoder side, the Palette mode, under consideration in HEVC SCC, consists in transforming pixel values of a given input coding unit into indexes called levels. The levels identify the entries in an associated palette, the pixel values of which match the pixel values of the input coding unit. However, when a pixel value of the input coding unit cannot be represented by a level (i.e. it does not match), e.g. because the distortion would be too large (greater than 0 in case of lossless coding), then said pixel is represented by a specific level, indicating "escape coding". For each pixel being represented by this specific "escape coding" level, quantized pixel values are furthermore transmitted.

After the transformation, the resulting coding unit is composed of a block of levels and a block of quantized values (for the escape-coded pixels). It is then transmitted to the decoder with the associated palette, generally a table having a finite number of triplets of colours used to represent the coding unit. Since the palette defines a finite number of colours, the transformation into a block of indexes usually approximates the original input coding unit in lossy coding, but strictly corresponds to the original input coding unit in lossless coding.

To apply the Palette mode at the encoder side, an exemplary way to transform a coding unit of pixels is performed as follows: -find the P triplets best describing the coding unit of pixels to encode, for example by minimizing overall distortion; -then associate with each pixel of the coding unit the matching colour among the P triplets: the value to encode (or level) (which thus forms part of the block of indexes) is then the index corresponding to the entry of the associated matching colour. The block of indexes is thus obtained from the palette by comparing the entries of the palette to each pixel of the coding unit, in order to identify, for each pixel, the entry which defines the matching colour. If no entry matches, then the level indicating escape coding, as well as quantized pixel values, is associated with the pixel (in the block of quantized values).

For each coding unit, the palette (i.e. the P triplets found), the block of indexes or levels and the block of quantized pixel values are coded in the bitstream 110 and sent to the decoder.

Furthermore, specific flags may be provided in some sets of parameters in the bitstream to specify whether or not the Palette coding mode is activated (for instance in the Sequence Parameter Set, or "SPS"). Those sets of parameters are also referred to as syntax structures.

Also, at the coding unit level, a flag may specify whether or not the coding unit has escape-coded values, to force the palette to include the above-mentioned specific "escape coding" level.

At the decoder, the Palette coding mode consists in performing the conversion in the reverse way. This means that each decoded index associated with each pixel of the coding unit is replaced by the corresponding colour in the palette decoded from the bitstream, in order to reconstruct the corresponding colour for each pixel of the coding unit. Note that if a pixel is associated with the "escape coding" level, then the corresponding quantized pixel value is decoded and inverse quantized from the block of quantized pixel values (i.e. of escape-coded pixels). This is the reconstruction of the block of indexes in the colour space (i.e. of the coding unit predictor).

Figure 5 further illustrates the principle of the Palette coding mode at the decoder. When decoding a slice, frame or tile, the decoding process loops over the CUs from the bitstream, starting from a first coding unit. Then, the prediction mode for the current coding unit is extracted at step 502 from the bitstream 501. Currently, the Palette mode is identified by a flag located after the skip flag and the intra block copy flag in the bitstream (the other coding modes have been described above with reference to Figures 1 and 2). This flag is CABAC coded using a single context. If this mode is not the Palette mode 503 then conventional decoding occurs on step 520. Otherwise, the related syntax of the Palette mode 505, i.e. the information on the palette, the block of levels and the block of escape-coded pixels, is extracted and decoded 504 from the bitstream 501. Next, during step 506, the following elements are built from the decoded data: the palette 509, the block of escape-coded pixels 507 and the block of levels 508. In particular, HEVC SCC provides that the palette is predicted from a palette predictor 510. From the block of levels, the associated palette and the block of escape-coded pixels, the reconstructed coding unit in pixel domain 514 is built. This means that for each level of the block of levels, a colour (RGB or YUV) is associated with each pixel.

Palette 509 is then used to update palette predictor 510 for use in decoding other palette-coded CUs.

To be noted that the prediction of palette 509 using palette predictor 510 may not use all the entries of palette predictor 510. Information on the used entries of palette predictor 510 or the non-used entries, as well as information on the size of the last used palette may be stored. Such information is reused as described below.

Figure 6 illustrates the principle of the Palette coding mode at the encoder.

The current coding unit 601 is converted into a block 602 of the same size which contains a level for each pixel instead of three colour values (Y, U, V) or (R, G, B). For means of illustration, pixel 611 of 601 is actually escape-coded and therefore, its associated level 612 indicates the escape coding level 613 (value "3") of the palette. As a consequence, block 604 of escape-coded pixels contains the quantized pixel value of a single pixel 620. The palette 603 associated with block of levels 602 is built based on coding unit overall distortion minimization and associates at each entry, an entry index or level with corresponding pixel colour values. Note that for monochrome application, the pixel value can contain only one component.

As mentioned above in relation to Figure 5, the palette (as well as the block of escape-coded pixels) is coded and inserted into the bitstream for each coding unit. In the same way, the block of levels (corresponding to the coding unit) is coded and inserted into the bitstream and an example of the coding is given below with reference to Figure 7. In this example, the block of levels is scanned in a horizontal order.

The block of levels 71 is exactly the same as the one illustrated in Figure 6 under reference 602. The tables 72 and 73 describe the successive syntax elements used to code the block of levels 71. Table 73 should be read as the continuation of table 72. The syntax elements in the table correspond to the encoding of the groups of levels surrounded by bold lines in the block 71.

The block of levels is encoded by group of successive pixels in scan order.

Each group is encoded using a first syntax element giving a prediction direction, a second element giving the repetition, and an optional third element giving the value of the pixel, namely the level. The repetition corresponds to the number of pixels in the group.

These two tables represent the current syntax associated with the Palette coding mode. These syntax elements correspond to the encoded information inserted in the bitstream for the block of levels 71. In these tables, three main syntax elements are used to fully represent the operations of the Palette coding mode and are used as follows when successively considering the levels of the block of levels 71.

A first syntax element, called "Pred mode" makes it possible to distinguish between two encoding modes. In a first mode corresponding to "Pred mode" flag equal to "0", a new level is used for the current pixel. The level is immediately signalled after this flag in the bitstream. In a second mode corresponding to "Pred mode" flag equal to "1", a "copy up" mode is used. More specifically, this means that the current pixel level corresponds to the pixel level located at the line immediately above starting on the same position for a raster scan order. In that case of "Pred mode" flag equal to "1", there is no need to signal a level immediately after the flag because the value of the level is known by reference to the value of the level of the pixel just above in the block of levels 71.

A second syntax element called "Level" indicates the level value of the palette for the current pixel only in the first mode of "Pred mode", or the level value for escape-coding of the pixel.

A third syntax element, called "Run", is used to encode a repetition value in both modes of "Pred mode". Considering that the block of levels 71 is scanned from the top left corner to the bottom right corner, row by row from left to right and top to bottom, the Run syntax element gives the number of successive pixels in block 71 having the same encoding.

This "Run" syntax element has a different meaning which depends on the "pred mode" flag. When Pred mode is 0, "Run" element is the number of successive pixels of the block of indexes having the same level value. For example, if Run=8 this means that the current "Level" is applied to the current pixel and to the following 8 pixels which corresponds to 9 identical successive samples in raster scan order.

When Pred mode is 1, "Run" element is the number of successive pixels of the block of indexes having a level value corresponding to the level value of their above pixel in block 71, i.e. where the "copy up" mode is applied. For example, if Run=26 this means that the level of the current pixel is copied from the pixel of the line above as well as the following 26 pixels which corresponds to 27 pixels in total.

Tables 72 and 73 represent the nine steps to represent the block 71 by using the Palette coding mode. Each step starts with the coding of the "Pred mode" flag which is followed by the "Level" syntax element when "Pred mode" flag equals "0", or by the "Run" syntax element when "Pred mode" flag equals "1". The "Level" syntax element is always followed by a "Run" syntax element.

When the prediction mode decoded for the current block is the palette mode, the decoder first decodes the syntax relating to this block and then applies the reconstruction process for the coding unit.

Figure 8 illustrates the decoding process of syntax elements relating to the Palette coding mode. First, the size of the palette is extracted and decoded 802 from the bitstream 801. The exact size of the palette (Palette_size) is obtained by adding 1 to this size value decoded at step 802. Indeed, the size is coded by using a unary code for which the value 0 has the smallest number of bits (1 bit) and the size of the palette cannot be equal to 0, otherwise no pixel value can be used to build the block predictor.

Next the process corresponding to the palette values decoding starts. A variable i corresponding to the index of the palette is set equal to 0 at step 804 next a test is performed at step 805 to check whether i is equal to the palette size (Palette_size) or not. If it is different from the palette size at step 805, one palette element is extracted from the bitstream 801 (in case the palette is directly encoded in the bitstream) and decoded at step 806 and is then added to the palette with the associated level/index equal to i. Then the variable i is incremented through step 807. If i is equal to the palette size at step 805, the palette has been completely decoded.

Next the process corresponding to the decoding of the block of levels 71 is performed. First, the variable j, corresponding to a pixel counter, is set to 0 as well as the variable syntax_i 808. Then a check is performed to know whether the pixel counter corresponds to the number of pixels contained in the block. If the answer is yes at step 809 the process ends at step 817, otherwise the value of the flag "Pred mode" corresponding to one prediction mode is extracted from the bitstream 801 and decoded 810.

The value of "Pred mode" is added to a table at the index syntax i containing all "Pred mode" value decoded. If the value of this "Pred mode" is equal to 0, step 811, the syntax element corresponding to "Level" is extracted from the bitstream 801 and decoded 812. This variable "Level" is added to a table at the index syntax_i containing all levels decoded. The variable j corresponding to the pixel counter is incremented by one 813.

Next the "Run" syntax element is decoded at step 814. If the syntax element "Pred Mode" is equal to 1, step 811, the "Run" value is also decoded at step 814. This syntax element "Run" is added to a table at the index syntax_i containing all the runs 25 decoded.

Next, at step 818, for each pixel that is escape-coded as indicated by its level (i.e. if the extracted Level equals the "escape coding" level, for instance "3" in the example of Figures 6 and 7), the associated quantized pixel value is parsed and dequantized from the block of escape-coded pixels. The dequantized pixel value is for instance stored in a corresponding table at the index syntax_i.

Note that step 818 may be performed just after step 812. In a variant, the whole block of escape-coded pixels may be extracted from the bitstream and dequantized in a step between step 809 and step 817.

Next at step 815, the value j is incremented by the value of the run decoded at step 814. The variable syntax_i is incremented by one to consider the next set of syntax elements. If the counter j is equal to the number of pixels in the block then the syntax to build the block of levels 71 is finished (817). At the end of this process, the decoder knows the palette, and the tables containing the list of all the "Pred mode", "Level" and "Run" syntax elements associated with the Palette coding mode of this coding unit, and also knows the table of dequantized pixel values for the escape-coded pixels. The decoder can then proceed with the reconstruction process of the coding unit as described through Figure 5.

In a slight variant of this embodiment of Figure 8, the "Pred mode" element is not provided for the first line of pixels at the top of the block of levels 71. This is because, since these pixels are deprived of levels at a line above, the "copy up" mode cannot be executed. Therefore, as long as j is less than the block width at step 809, no "Pred mode" element is provided and steps 810-811 are shortcut, thereby directly performing step 812. Note that this slight variant decreases the size of the encoded block of levels.

In an embodiment that can be combined with either the above embodiment of Figure 8 or its slight variant, several blocks of levels may be generated instead of only one. This means that several levels are used for all or parts of the pixels. For instance, a first block of levels may be built for a first colour component (Y for example), while another block of levels may be built for the at least one remaining component (U and V for example). Of course, three blocks of levels for the three colour components may be contemplated. The choice to have several blocks of level and their correspondence with the colour components may be signalled in the bitstream using specific flags. In a variant, this may be implied by the colour format of the image.

Referring back to the palette, each palette element, constituted by three values in the above examples, is generally encoded using three binary codes. The length of the binary codes corresponds to the bit-depth of each colour component. The "Pred mode" element is encoded using one bit. The "Level" element is encoded using binary code with binary code length equal to b, where 2b is the smallest integer equal or above the palette size.

Figure 9 illustrates the reconstruction process to build the block of levels 91.

The input data of this process are the tables obtained using the process of Figure 8 above, and containing the list of "Pred mode", "Level" and "Run".

An additional item of input data to the "Pred mode", "Level" and "Run" elements is the size of the coding unit 801 (which is the same as the size of the block of levels 602/71) known from the quadtree (Figure 4) signalled in the bitstream.

In a first step 901, a variable i, representing a pixel counter, is set equal to 0 and a variable j, to successively consider each set of syntax elements, is also set equal to 0. At step 904, the element Pred_mode[j] extracted from the table of "Pred mode" at index j is checked against 0.

If it is equal to 0, a new level is encoded for the current pixel i. As a consequence, the value of the pixel at position i is set equal to the level at the index j from the table of levels; Block[i] =Level[j]. This is step 905. The variable i is incremented by one at step 906 to consider the next pixel, and the variable k, dedicated to count the pixels already processed in the current Run, is set equal to 0 at step 907.

A check is performed at step 908 to determine whether or not k is equal to the "Run" element of the table of runs at the index j: k = Run[j] 2. If not equal, the level of the pixel at position i is set equal to the level value of the pixel at position Block[i] = Block[i-1]. This is step 909. The variable i and the variable k are then incremented by one at respectively steps 910 and 911. If k = Run[j] at step 908, the propagation of the left level value is finished and step 920 is performed (described below).

If Pred_mode[j] is different from 0 at step 904, the "copy up" mode starts with the variable k set equal to 0 at step 912. Next, step 913 checks whether or not (k-1) is equal to the "Run" element of the table of runs at the index j: k = Run[j]+1? If not equal, the level value of the pixel at position i is set equal to the level value of the pixel at position i of the above line: Block[i] = Block[i-width], where "width" is the width of the block of levels (the same as the coding unit) as deduced from the input size of the coding unit. This is step 914. Next, the variable i and the variable k are incremented by one at respectively steps 915 and 916. If k = Run[j]+1 at step 913, the prediction mode 'copy up' is completed and the process goes on at step 920.

At step 920, a check is performed to determine whether or not the variable i is equal to the amount of pixels in the block 71/CU 601. If not equal, the variable j is incremented by one at step 921 to consider the next set of syntax elements and the process loops back to step 904 described above.

If all the pixels have been processed at step 920, the final block of levels 71 is obtained at step 922: this corresponds to table Block[].

Next, a final step 923 consists in converting each level in colour values using the palette 603 decoded using the process of Figure 8 and using block 604 of dequantized pixel values for the escape-coded pixels. This final step affects pixel values (Y, U, V) or (R, G, B) at each block position according to the level of this position in the block and either the corresponding entry in the palette 603 if any, or the corresponding dequantized pixel value in block 604.

As described above, the Palette coding mode as currently designed in HEVC SCC requires a palette to be transmitted for each coding unit. This represents a large amount of data in the bitstream, and thus a coding cost. In order to reduce that cost, some proposed mechanisms provide the current palette for a current coding unit to be predicted using a palette predictor.

As proposed in Applicant's contribution JCTVC-Q0063, a reference palette predictor can be transmitted in the bitstream to be used by each coding unit of a slice for instance; or the palette predictor can be built using pixels neighboring the coding unit processed; or the predictor can be built from two or more palettes already existing.

The prediction process thus modifies step 806 of forming the palette from the bitstream Figure 10 illustrates the existing embodiment for declaring or defining the palette predictor initializer within the standard specification as described in JCTVC -R 1005. This embodiment relies on an existing extension section of the PPS syntax structure in the HEVC standard and its SCC Extension (see document JCTVC-T1005). The presence of the corresponding extension section (namely "pps_scc_extensions") is indicated in the PPS, by the flag pps_scc_extensions_flag.

It firstly contains information relating to the "residual adaptive colour transform" tool (refered to as "ACT" from now on), which applies a revertible predetermined colour transform to the decoded output block. This tool can be disabled at the PPS level by setting the flag residual_adaptive_colour transform_enabled_flag to 0.

If it is not disabled, then addition information follows. A flag pps_slice_act_qp_offsets_present_flag indicates whether quantiser step offsets (compared to the ones set for the slice that uses current PPS) are present, in which case these offsets, pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5, and pps_act_cr qp_offset_plus3 are transmitted as variable-length elements.

Then a first flag related to the palette mode palette_predictor initializer present_flag is used. It indicates whether the palette predictor initializer is present (i.e. actually defined in the PPS SCC extension as mentioned above).

If it is present, then information regarding the entries of this initializer are transmitted. Firstly, the colour format information. It is usually a three components or a monochrome format. A flag monochrome_palette_flag indicates if the format comprises only one color component if set to a predetermined value, for example 1. If not set to this value, the format is not monochrome.

Then, the bitdepth of the first (and potentially only) component is transmitted on a variable number of bits by the syntax element luma_bit_depth_entry_minus8. As the bitdepth of any component cannot be lower than 8, only the difference of said bitdepth with 8 needs to be transmitted (thereby saving a few bits). Then, if the entries are not monochrome, the bitdepth for chroma is transmitted through the syntax element chroma_bit_depth_entry_minus8.

Then, as it is known palette entries are transmitted, their number is known, and is at least 1. Its value is transmitted through the variable-length syntax element: num_palette_predictor initializer minusl. It is the size of the palette predictor initializer.

Next, the values of the entries of the palette predictor initializer are defined. To be noted that the number of components numComps of the entries is known and is 1 when monochrome_palette_flag is 1 and 3 otherwise. Also, the number of bits for each component is also known, as they can be derived from the syntax elements luma_bit_depth_entry_minus8 and chroma_bit_depth_entry_minus8 included in the SPS.

The proposed syntax is redundant and may even be source of faults. By authorizing contradictory configurations, the current syntax is not efficient enough.

As proposed below, the information in the PPS extension can be improved regarding the use of the palette mode. Figure 11 illustrates an example of such an improvement.

According to an embodiment, a parameter like a flag called here "monochrome_flag" is sent before either the palette or ACT tool information is transmitted. In other words, the proposed "monochrome_flag" is defined at the PPS level.

As a variant, the monochrome flag may be defined at a sequence level. If the monochrome_flag is set to a predetermined value, for example "1", then when considering the palette predictor initializer, the palette entries are directly adjusted for a monochrome format (in other words that the step for defining the palette predictor initializer is amended given the palette predictor initializer comprises only one element per entry). There is no more need of a parameter, specific to the palette predictor initializer, for signalling the colour format in order to adapt the number of entries. The invention according to one of its aspect, proposes to use a parameter defined at the picture (or image) level of at a higher level.

In a preferred embodiment, when the "monochrome_flag" is set to "1", then the ACT tool is disabled (in other words the step of reading the value of the "residual adaptive coulour transform enabled flag" is skipped, disabling the ACT tool), given that tool become useless. In that case the flag residual_adaptive_colour_transform_enabled_flag is inferred to be 0. On the opposite, if monochrome_flag's value is 0, then the ACT tool may or may not be enabled, therefore its corresponding flag is transmitted. Similarly, the palette entries number of components numComps is 3.

According to another embodiment, a simpler syntax is proposed so that the monochrome_flag impacts only one mode at a time, preferably the palette mode. Such an example would be in case the palette mode is known to be deactivated by another mean.

An example of such an embodiment is illustrated in Figure 12. In that case, a condition (for example a single condition) is introduced before reading flag monochrome_palette_flag presented in Figure 10. If the ACT tool is activated (as indicated by the value of its corresponding flag residual_adaptive_colour_transform_enabled_flag), then the pixel colour format is deduced as being not monochrome: there are three color-components. The flag monochrome_palette_flag's value can be directly inferred to be "0". On the contrary, if the ACT tool is disabled, the format may or may not be monochrome.

In another embodiment it is proposed an encoder-only constraint, so as to not modify the decoder logic. The two following conditions must be satisfied: if residual_adaptive_colour_transform_enabled_flag is 1, then monochrome_palette_flag must be 0. And if monochrome_palette_flag is 1, residual_adaptive_colour_transform_enabled_flag must be 0.

In another embodiment it is proposed to decorrelate the actual pixel format from the number of components of the entries. In the previous embodiment, the palette predictor initializer is set to have entries having a different number of components according to the pixel format. In this embodiment the same palette predictor initializer may be able to be used with monochrome data and RGB or YUV data. As a consequence, both palette predictor and palette predictor initializer may have entries with one to three elements depending on the color format. The number of elements per entry may be different for the palette predictor and palette predictor initializer. In such case, when initializing the palette predictor with the predictor initializer, various rules may be applied. For instance, when going from one component to three, the palette predictor entries predictorfi] are initialized with default greyscale values or existing palette predictor initializer entries. For instance, For RGB, predictor[i][2], predictor[i][1] and predictor[i][0] are set equal to palette_predictor_initializers[i][0]; - For YUV, predictor[i][0] is set equal to palette_predictor_initializers[i][0], and predictor[i][2] and predictor[i][1] are set equal to a specific value of the component (i.e. 0, 128 or 255 for a component whose values are between 0 and 255).

Other variations as possible, i.e. when going from three to one component, when coding a given color plane for a component of index comp (for instance U if YUV format is considered), predictor[i] is set to palette_predictor_initializers[i][comp].

Conventional mechanisms to extract the entries of the initializer from the PPS SCC extension can be used.

Several ways to define a simplified syntax removing redundancy between the palette mode and other existing tools on the decoder side have been described.

Figure 13 is a schematic block diagram of a computing device 1300 for implementation of one or more embodiments of the invention. The computing device 1300 may be a device such as a micro-computer, a workstation or a light portable device. The computing device 1300 comprises a communication bus connected to: -a central processing unit 1301, such as a microprocessor, denoted CPU; - a random access memory 1302, denoted RAM, for storing the executable code of the method of embodiments of the invention as well as the registers adapted to record variables and parameters necessary for implementing the method for encoding or decoding an image according to embodiments of the invention, the memory capacity thereof can be expanded by an optional RAM connected to an expansion port for example; -a read only memory 1303, denoted ROM, for storing computer programs for implementing embodiments of the invention; - a network interface 1304 is typically connected to a communication network over which digital data to be processed are transmitted or received. The network interface 1304 can be a single network interface, or composed of a set of different network interfaces (for instance wired and wireless interfaces, or different kinds of wired or wireless interfaces). Data packets are written to the network interface for transmission or are read from the network interface for reception under the control of the software application running in the CPU 1301; -a user interface 1305 may be used for receiving inputs from a user or to display information to a user; -a hard disk 1306 denoted HD may be provided as a mass storage device; -an I/O module 1307 may be used for receiving/sending data from/to external devices such as a video source or display.

The executable code may be stored either in read only memory 1303, on the hard disk 1306 or on a removable digital medium such as for example a disk. According to a variant, the executable code of the programs can be received by means of a communication network, via the network interface 1304, in order to be stored in one of the storage means of the communication device 1300, such as the hard disk 1306, before being executed.

The central processing unit 1301 is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to embodiments of the invention, which instructions are stored in one of the aforementioned storage means. After powering on, the CPU 1301 is capable of executing instructions from main RAM memory 1302 relating to a software application after those instructions have been loaded from the program ROM 1303 or the hard-disk (HD) 1306 for example. Such a software application, when executed by the CPU 1301, causes the steps of the flowcharts shown in Figures 11 or 12 to be performed.

Any step of the algorithms shown in Figures 11 or 12 may be implemented in software by execution of a set of instructions or program by a programmable computing machine, such as a PC ("Personal Computer"), a DSP ("Digital Signal Processor") or a microcontroller; or else implemented in hardware by a machine or a dedicated component, such as an FPGA ("Field-Programmable Gate Array") or an ASIC ("Application-Specific Integrated Circuit').

Although the present invention has been described hereinabove with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications will be apparent to a skilled person in the art which lie within the scope of the present invention.

Many further modifications and variations will suggest themselves to those versed in the art upon making reference to the foregoing illustrative embodiments, which are given by way of example only and which are not intended to limit the scope of the invention, that being determined solely by the appended claims. In particular the different features from different embodiments may be interchanged, where appropriate.

In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used.

Claims (17)

1. CLAIMS1. A method of decoding a picture parameter set used for decoding images from a bitstream, the image comprising pixels having at least one color-component, the method being broken down into several steps, comprising: decoding one color-component parameter defined in the picture parameter set and involved in at least two steps of the decoding method, indicating if said image is monochrome; and skipping or amending at least one step of the decoding method if the decoded color-component parameter indicates that the image is monochrome.
2. 2. The method according to claim 1, wherein at least two different steps are either skipped or amended, at least one of the steps comprising decoding pixel values having been encoded using a palette mode.
3. 3. The method of claim 2, wherein one step comprising determining the number of elements for at least one entry or each entry of a palette predictor initializer.
4. 4. The method of claim 3, wherein each entry of a palette predictor initializer is a singleton if the decoded color-component parameter is set to a predetermined value, else a triplet.
5. 5. The method according to any of the previous claims wherein one of the steps is related to the residual adaptive colour transform, which is skipped if the color-component parameter indicates that the image is monochrome.
6. 6. The method according to any of the previous claims, wherein the parameter is a flag taking the value "1" when the image is monochrome.
7. 7. A method of decoding a picture parameter set used for decoding images from a bitstream, the image comprising pixels having at least one color-component, said image being encoded by using one mode among a plurality of modes including the palette mode, the method comprising: - a step related to the residual adaptive colour transform; and - a step for defining a palette predictor initializer, the defining step being processed based on the value of a color-component parameter indicating if the image is monochrome or not, wherein the value of the color-component parameter is inferred from the execution of the residual adaptive colour transform.
8. 8. The method of claim 7, wherein if the residual adaptive colour transform is executed, the value of the color-component parameter is inferred to indicate that the image is not monochrome.
9. 9. The method of claim 7 or 8, wherein the color-component parameter is a flag whose value is inferred to be "0" if the residual adaptive colour transform is executed.
10. 10. A method of decoding a picture parameter set used for decoding images from a bitstream, the image comprising pixels having at least one color-component, at least one image being encoded by using the palette mode, the method comprising a step for initializing a palette predictor with a palette predictor initializer having the same number of entries, the palette predictor initializer and the palette predictor having one or more elements per entry, wherein the initialization of the palette predictor by the palette predictor initializer is governed by predetermined rules when switching from an image having pixels with a given number of color-components to an image having another number color-15 components.
11. 11. The method of claim 10, wherein a predetermined rule comprises for at least one entry of the palette predictor, setting at least two of the elements to the same element's value of the palette predictor initializer.
12. 12. The method of claim 10, wherein a predetermined rule comprises for at least one entry of the palette predictor, setting at least one of the elements to a predetermined default value.
13. 13. The method of claims 10 to 12, wherein the predetermined rule is applied when switching from an image having one color-component to an image having three color-components.
14. 14. A device for decoding a picture parameter set related to an image from a bitstream, said device being configured to implement a decoding method according to anyone of claims 1 to 13.
15. 15. A non-transitory computer-readable medium storing a program which, when executed by a microprocessor or computer system in a device for decoding at least one image, causes the device to perform the method of any of the claims 1 to 13.
16. 16. A method of decoding a picture parameter set used for decoding images from a bitstream, substantially as herein described with reference to, and as shown in Figure 10 to Figure 12 of the accompanying drawings.
17. 17. A device for decoding a picture parameter set used for decoding images from a bitstream, substantially as herein described with reference to, and as shown in Figure 13 of the accompanying drawings.