CN110235446B

CN110235446B - Video encoding method, video decoding method and related devices

Info

Publication number: CN110235446B
Application number: CN201780084713.2A
Authority: CN
Inventors: 郑萧桢; 李蔚然
Original assignee: SZ DJI Technology Co Ltd
Current assignee: SZ DJI Technology Co Ltd
Priority date: 2017-06-08
Filing date: 2017-06-08
Publication date: 2024-04-12
Anticipated expiration: 2037-06-08
Also published as: CN110235446A; WO2018223353A1

Abstract

A video encoding method, a video decoding method and related devices are provided. The video encoding method includes: when a current image to be coded can be used as a reference image, and when an image block can be used for updating a specific reference image, updating a specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be coded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be coded.

Description

Video encoding method, video decoding method and related devices

Technical Field

The present invention relates to the field of image processing, and more particularly, to a video encoding method and apparatus, and a video decoding method and apparatus.

Background

In order to reduce the bandwidth occupied by video storage and transmission, it is necessary to perform a coding compression process on video data. The encoding compression process includes prediction, transformation, quantization, and entropy encoding processes. Wherein the prediction includes both intra prediction and inter prediction types, and aims to remove redundant information of a current image block to be encoded by using prediction block information. Intra prediction obtains prediction block data using information of a current frame image. Inter prediction obtains prediction block data by using information of a reference frame, and the process includes searching a reference image for a block most matched with a current image block or a sub-image block as a prediction block in units of image blocks or sub-image blocks obtained by dividing the image blocks; thereafter, the image block or sub-image block is subtracted from the corresponding pixel values of the prediction block to obtain a residual.

When the reference frame is more similar to the current image to be coded, the residual error generated by the inter-frame prediction is smaller when the inter-frame prediction is performed, so that the coding efficiency of the inter-frame prediction can be improved. In order to increase the similarity of the reference frame to the current image to be encoded, this object may be achieved by constructing a specific reference frame. In video content, there is typically a specific class of coded scenes in which the background does not change substantially, but only the foreground in the video changes or moves. For example, video surveillance belongs to this type of scenario. In a video monitoring scene, a monitoring camera is usually fixed or only slowly moves, and the background is considered to be basically unchanged. In contrast, objects such as persons and vehicles photographed in video monitoring lenses are often moved or changed, and the foreground is considered to be constantly changed. In such a scene, if a specific reference frame can be constructed, which contains only high quality background information, the background portion of the current encoded frame can be used to reduce residual information of inter prediction by referring to the high quality specific reference frame when inter prediction is performed, thereby improving encoding efficiency.

Constructing the specific reference frame can refresh the reconstructed pixel information of the image blocks considered to belong to the background into the image blocks which are positioned in the same position as the image blocks in the specific reference frame by analyzing the coding content of the coding frame in the coding process, the updating operation is carried out every time one frame of image is coded, the reconstructed pixel information of a plurality of image blocks in one frame of image is refreshed into the specific reference frame, and the construction of one specific reference frame can be completed after a plurality of images are coded.

The prior art does not consider whether an image block is located in an image that is not to be used as a reference frame when refreshing a particular reference frame with reconstructed pixel information for that image block. In video technology, images may be divided into images that can be reference frames and images that are not reference frames. For images that are not taken as reference frames, the information of their own images during the encoding and decoding processes is not used for the encoding and decoding processes of other images. In the prior art, whether the image block is positioned in the image which is not used as the reference frame is not considered when the reconstructed pixel information of the image block is used for refreshing the specific reference frame, so that the construction process of the specific reference frame is in violation of the definition of the image which is not used as the reference frame.

In addition, the image which is not used as the reference frame is not used as the decoding process of other images when being decoded, so the image which is not used as the reference frame can not be decoded when being decoded, the decoding speed of the video code stream can be increased, and the variable frame rate playing of the video content can be realized. In the prior art, whether the image block is positioned in the image which is not used as the reference frame is not considered when the reconstructed pixel information of the image block is used for refreshing the specific reference frame, so that the image which is not used as the reference frame cannot be directly discarded in the decoding process, and the functions cannot be realized.

Disclosure of Invention

The embodiment of the invention provides a video coding method and device and a video decoding method and device.

In a first aspect, a video encoding method is provided, including:

when a current image to be coded can be used as a reference image, and when an image block can be used for updating a specific reference image, updating a specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be coded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be coded.

In a second aspect, there is provided a video decoding method, comprising:

when a current image to be decoded can be used as a reference image, and when an image block can be used for updating a specific reference image, updating a specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be decoded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be decoded.

In a third aspect, there is provided a video decoding apparatus comprising:

at least one memory for storing computer-executable instructions;

at least one processor configured to, individually or collectively: accessing the at least one memory and executing the computer-executable instructions to perform the operations of:

In a fourth aspect, a machine readable storage medium is provided, suitable for use in a terminal, the machine readable storage medium having stored thereon computer instructions that, when executed, perform the method of processing video encoding according to the first aspect.

In a fifth aspect, there is provided a video decoding apparatus comprising:

at least one memory for storing computer-executable instructions;

In a sixth aspect, a machine-readable storage medium is provided, suitable for use with a terminal, having stored thereon a number of computer instructions that when executed perform the video decoding method of the second aspect.

In a seventh aspect, a method of video encoding is provided, comprising:

when a current image to be coded is available for updating a specific reference image, and when an image block is available for updating the specific reference image, updating a specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be coded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be coded.

An eighth aspect provides a video decoding method, comprising:

when a current image to be decoded is available for updating a specific reference image, and when an image block is available for updating the specific reference image, updating a specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be decoded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be decoded.

A ninth aspect provides a video encoding apparatus, comprising:

at least one memory for storing computer-executable instructions;

In a tenth aspect, there is provided a video decoding apparatus comprising:

at least one memory for storing computer-executable instructions;

In an eleventh aspect, there is provided a machine-readable storage medium adapted for use with a terminal, the machine-readable storage medium having stored thereon computer instructions which, when executed, perform the video encoding method of the seventh aspect.

In a twelfth aspect, a machine-readable storage medium is provided, adapted for use in a terminal, the machine-readable storage medium having stored thereon computer instructions that when executed perform the video decoding method of the eighth aspect.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an embodiment of a video encoding method of the present invention;

FIG. 2 illustrates the relationship of image blocks in a current image to be encoded to image blocks in a particular reference image;

FIG. 3 is a schematic diagram of the relationship of multiple images in a video to a particular reference image;

FIG. 4 is a schematic diagram of a further embodiment of the method of video encoding of the present invention;

FIG. 5 is a schematic diagram of a video decoding method according to another embodiment of the present invention;

FIG. 6 is a flow chart of yet another embodiment of a method of video decoding according to the present invention;

fig. 7 is a schematic block diagram of a video encoding apparatus according to an embodiment of the present invention;

fig. 8 is another schematic block diagram of a video encoding device according to an embodiment of the present invention;

fig. 9 is a schematic block diagram of a video decoding apparatus according to an embodiment of the present invention;

fig. 10 is a schematic block diagram of a video decoding apparatus according to an embodiment of the present invention.

Detailed Description

The methods, apparatus and terminals described herein provide a number of ways to update a particular reference image.

Video is made up of multiple images. When the video is encoded, different images in the video can adopt different prediction modes. Depending on the prediction mode adopted for the picture, the picture can be divided into an intra-predicted picture and an inter-predicted picture, wherein the inter-predicted picture includes a forward predicted picture and a bi-predicted picture. The I-picture is an intra-predicted picture, also called a key frame; the P picture is a forward predictive picture, i.e. a P picture or an I picture that has been previously encoded is used as a reference picture; the B picture is a bi-predictive picture, i.e., a picture before and after is used as a reference picture. One implementation is that the encoding end encodes a plurality of pictures to generate a group of pictures (GOP), which is a group of pictures consisting of an I picture, and a plurality of B pictures (or bi-predictive pictures) and/or P pictures (or forward predictive pictures). The decoding end reads the GOP section by section for decoding and then reads the picture for rendering and displaying when playing.

As each image is encoded, the image may be initially divided into a plurality of image blocks. In some embodiments, an image may be divided into a plurality of image blocks, which are referred to in some coding standards as macro blocks or largest coding units (LCUs, largest Coding Unit). The image blocks may or may not have any overlapping portions. The image may be divided into any number of image blocks. For example, the image may be divided into an array of m x n image blocks. The image blocks may have a rectangular shape, a square shape, a circular shape, or any other shape. The image blocks may have any size, such as p x q pixels. In modern video coding standards, pictures of different resolutions may be encoded by first dividing the picture into a plurality of small blocks. For h.264, a picture block is called a macroblock, which may be 16×16 pixels in size, and for HEVG, a picture block is called a maximum coding unit, which may be 64×64 in size. Each image block may be the same size and/or shape. Alternatively, two or more tiles may have different sizes and/or shapes. In some embodiments, an image block may also be not a macroblock or a maximum coding unit, but rather comprise a portion of a macroblock or a maximum coding unit, or comprise at least two complete macroblocks (or maximum coding units), or comprise at least one complete macroblock (or maximum coding unit) and a portion of a macroblock (or maximum coding unit), or comprise at least two complete macroblocks (or maximum coding units) and portions of some macroblocks (or maximum coding units). In this way, after an image is divided into a plurality of image blocks, the image blocks in the image data can be encoded separately.

In the process of encoding and decoding video, the contents of the specific reference images are updated by utilizing each image in the video at the same time to construct a specific reference image containing the background content of a scene, so that the background part of the current image to be encoded or the current image to be decoded can be used for reducing the residual information of the inter-frame prediction by referring to the high-quality specific reference image when the inter-frame prediction is carried out, thereby improving the encoding efficiency. That is, the specific reference picture is a reference picture that is inter prediction. Specifically, the specific reference image may be a long-term reference frame (long term reference) in standards such as H.264/AVC or H.265/HEVC, a background frame (background picture) in standards such as AVS1-P2, AVS2-P2, IEEE 1857.9-P4, or a gold frame (golden frame) in standards such as VP8 and VP 9.

The method for updating a particular reference image for an image block in an image is explained below in connection with fig. 1. Fig. 1 is a flowchart of an embodiment of a video encoding method according to the present invention. The method is performed by an image processing apparatus, which may be various types of chips for image processing, an image processor, or the like. As shown in fig. 1, the method includes:

101. When the current image to be coded can be used as a reference image, and when the image block can be used for updating the specific reference image, updating the specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be coded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be coded.

In some embodiments, determining that the current image to be encoded can be used as a reference image can occur prior to encoding the image to be encoded. In this way, it is possible to judge that each image block satisfies the condition for updating the specific reference image at the time of encoding each image block in the image to be encoded or after encoding, based on the result of the determination. Or determining that the current image to be encoded can be used as a reference image, or can occur when each image block in the image to be encoded is encoded or after encoding, namely, when each image block is encoded or after encoding, firstly determining that the image in which the image block is currently located can be used as the reference image, and when the image block is determined to be used as the reference image, determining that the image block can be used for updating the specific reference image.

In some embodiments, since the I-picture and the P-picture (or the forward predictive picture) may be reference pictures for inter prediction of other pictures, when it is determined that the current picture to be encoded is the I-picture or the P-picture (or the forward predictive picture), the current picture to be encoded is determined as the reference picture. In some embodiments, a partial B-picture (or bi-predictive picture) may also be used as a reference picture for inter-prediction of other pictures, e.g., in the hierarchical B (Hierarchical B) technique, a B-picture located at a lower level may be used as a reference frame, and thus, when it is determined that the current picture to be encoded is this partial B-picture, it may also be determined that the current picture to be encoded may be used as a reference picture.

In some embodiments, after determining that the current picture to be encoded can be used as the reference picture, the encoding side also writes a parameter or flag bit in at least one of a video parameter set (VPS, video parameter set), a sequence parameter set (SPS, sequence parameter set), a sequence header, a picture header, a slice header, a reference picture set (RPS, reference picture set), and a reference picture configuration set (RCS, reference configuration set) to indicate that the current picture to be encoded can be used as the reference picture.

In some embodiments, a judgment image block may be used to update a particular reference image, and the judgment may be made by pixel information of the image block. For example, when it is determined that the difference in content between an image block and a block at the same position in a previously encoded image is small, the image block is considered to contain background content, which can be used to update a specific reference image. Of course, there are other methods that may be used to determine that an image block may be used to update a particular reference image.

In some embodiments, the position of the particular image block in the particular reference image is the same as the position of the image block in the current image to be encoded. As shown in fig. 2, fig. 2 illustrates a relationship between an image block in a current image to be encoded and an image block in a specific reference image. The image blocks 210 and 220 in the current image to be encoded shown in fig. 2 are image blocks that satisfy a preset condition, where the position of the image block 210 in the current image to be encoded is the same as the position of the image block 110 in the specific reference image, and the position of the image block 220 in the current image to be encoded is the same as the position of the image 120 in the specific reference image.

In some embodiments, the position in the particular reference image is offset from the position of the image block in the current image to be encoded by a preset offset value.

In some embodiments, a particular image block in a particular reference image is updated from image block, in particular replacing the current content of the particular reference block with the pixel value of the image block. In some embodiments, updating the specific image block in the specific reference image according to the image block may be to replace the current pixel value of the specific reference image after processing the pixel value of the image block, where the processing may be to average the pixel value of the image block with the pixel value of one image block in the specific reference image, or to average the pixel value of the image block with the pixel value of the image block in the specific reference image in a weighted manner, where the coefficient of the weighted average is a preset value or is obtained by parsing from a code stream. When the pixel value of the image block is adopted, the original pixel value of the image block can be adopted, and the reconstructed pixel value of the image block can be adopted.

In video technology, image information of an image that is not taken as a reference image itself is not used in encoding and decoding processes of other images in the encoding and decoding processes. In this embodiment, when it is determined that the current image to be encoded can be used as the reference image, the image block in the current image to be encoded is considered to update the specific reference image, so that the situation that the construction process of the specific reference image is violated with the definition of the image which is not used as the reference image is avoided. In addition, the image which is not used as the reference image is not used as the decoding process of other images when being decoded, so the image which is not used as the reference image can not be decoded when being decoded, the decoding speed of the video code stream can be increased, and the variable frame rate playing of the video content can be realized. In this embodiment, when it is determined that the current image to be encoded can be used as the reference image, the image block in the current image to be encoded is considered to update the specific reference image, so that the situation that the discarded part cannot be used as the reference image when the image block in the image which cannot be used as the reference image is used to update the specific reference image is avoided.

In some embodiments, when the current image to be encoded is not an image that can be used as a reference image, it is determined that the image block is not used to update the particular reference image.

There are various ways to determine that an image block can be used to update a particular reference image. For example, it may be determined that an image block is available for updating a particular reference image based on pixel values of the image block and pixel values of an encoded block, where the encoded block refers to an image block located at a particular position in the encoded image prior to the current image to be encoded. For example, the encoded block refers to being located in an encoded image of a previous frame (or two previous frames) of the current image to be encoded. For example, the specific position may be the same as the position of the image block in the current image to be encoded, or the position of the image block in the current image to be encoded is added with a preset offset value.

In some embodiments, when a judgment is made that an image block is available for updating a particular reference image based on pixel information of the image block and pixel information of the encoded block, the judgment is made that the image block is available for updating the particular reference image based on at least one of:

pixel value differences of the luminance component between the image block and the encoded block;

the total number of pixels of the luminance component of the image block and/or the encoded block;

Pixel value differences of chrominance components between the image block and the encoded block;

the total number of pixels of the chrominance component of the image block and/or the encoded block.

The difference between the pixel values of the luminance components of the image block and the encoded block may be a distribution of differences between the pixel values of the luminance components of the image block and the second encoded block at the same positions, a sum of differences between the pixel values of the luminance components of the image block and the second encoded block at the same positions, or a difference between an average value of the luminance components of the image block at the same positions and an average value of the luminance components of the encoded block at the same positions.

The difference between the pixel values of the chrominance components between the image block and the encoded block may be a distribution of differences between the pixel values of the chrominance components of the image block and the second encoded block at the same positions, a sum of differences between the pixel values of the chrominance components of the image block and the second encoded block at the same positions, or a difference between an average value of the chrominance components of the image block at the same positions and an average value of the chrominance components of the encoded block at the same positions.

Wherein the determination of the image block from the difference in pixel values of the luminance component between the image block and the encoded block may be used to update the specific reference image, in particular the determination of the image block from the sum of absolute values of the difference in pixel values of the luminance component between the image block and the second encoded block may be used to update the specific reference image.

Wherein the determination of the image block from the difference in pixel values of the chrominance components between the image block and the encoded block may be used to update the specific reference image, in particular the determination of the image block from the sum of absolute values of the difference in pixel values of the chrominance components between the image block and the second encoded block may be used to update the specific reference image.

In an embodiment in which it is determined that an image block is available for updating a particular reference image based on a difference in pixel values of a luminance component between the image block and an encoded block, optionally, when it is determined that the image block is available for updating the particular reference image, the conditions that the image block is required to satisfy include: the number of specific pixels in the image block is smaller than the first threshold. Wherein the specific pixel is a pixel whose difference between the pixel values of the first color channel and the pixel at the same position in the encoded block is not less than the second threshold value.

Specifically, for example, an image is stored in three components, Y (luminance), U (chromaticity 1), V (chromaticity 2), respectively. The first color channel is a Y channel. The coded block is an image block which is positioned in the coded image of the previous frame (or the previous two frames) of the current image to be coded and has the same position as the image block, or an image block which is offset by a preset value in the current image to be coded compared with the image block. The pixel difference value of the Y component of the encoded block and the Y component of the image block at any one identical position is disty. When Dist y is not smaller than the second threshold, the pixels in the image block corresponding to Dist y are specific pixels. When it is determined that the image block satisfies the condition that "the number of specific pixels of the image block is not less than the first threshold", the first threshold may be a preset value, or may be a product of the total number of pixels of the Y component of the image block and a preset ratio, which is not limited herein.

In an embodiment in which it is determined that an image block is available for updating a particular reference image based on a difference in pixel values of a luminance component between the image block and an encoded block, optionally, when it is determined that the image block is available for updating the particular reference image, the conditions that the image block is required to satisfy include: the difference in pixel values of the luminance component between the image block and the encoded block is less than a third threshold.

When it is determined that the image block satisfies the condition that the difference between the pixel values of the luminance components between the image block and the encoded block is smaller than the third threshold, the third threshold may be a preset value, or may be a product of the total number of pixels of the Y component of the image block and a preset ratio, which is not limited herein.

In an embodiment in which it is determined that an image block is available for updating a particular reference image based on a difference in pixel values of chrominance components between the image block and the encoded block, optionally, when it is determined that the image block is available for updating the particular reference image, the conditions that the image block is required to satisfy include: the difference in pixel values of the chrominance components between the image block and the encoded block is less than a fourth threshold.

Wherein the difference in pixel value of the chrominance component between the image block and the encoded block is less than a fourth threshold, it may be that the difference in pixel value of the U component between the image block and the encoded block is less than the fourth threshold, or that the difference in pixel value of the V component between the image block and the second encoded block is less than the fourth threshold, or that the difference in pixel value of the U component between the image block and the encoded block is less than a preset value, and the difference in pixel value of the V component between the image block and the encoded block is less than another preset value.

When it is determined that the image block satisfies the condition that the difference in pixel value of the chrominance component between the image block and the encoded block is smaller than the fourth threshold value, the fourth threshold value may be a preset value, or may be a product of the total number of pixels of the luminance component (or the chrominance component) of the image block and a preset ratio, which is not limited herein.

Some of the conditions that need to be met by the image blocks that can be used to update a particular reference image are described above. In some embodiments, when the current picture to be encoded is an I-picture or a random access point (RAP, random access point), or the current picture to be encoded is both an I-picture and a random access point, all picture blocks of the particular reference picture are updated according to all picture blocks of the current picture to be encoded. The whole image block of the image to be encoded may refer to the whole image block after the encoding reconstruction of the current image to be encoded, or may refer to the original whole image block of the current image to be encoded. For example, all image blocks in a particular reference image may be replaced with all image blocks of the current image to be encoded. Alternatively, all image blocks in the specific reference image are replaced after a certain process is performed on all image blocks of the current image to be encoded, where the process may be to average or weight average pixel values of all image blocks of the current image to be encoded, which is not limited herein.

In some embodiments, the number of image blocks in the current image to be encoded that can be used to update a particular reference image may be 1 or greater than 1. In some embodiments, the number of image blocks in the current image to be encoded that can be used to update the particular reference image may be unlimited, i.e., all image blocks in the current image to be encoded that satisfy the conditions that can be used to update the particular reference image are used to update the particular reference image.

In some embodiments, the number of image blocks available for updating the specific reference image is large, and from the standpoint of complexity of implementation of the codec system, the number of image blocks available for updating the specific reference image in the current image to be encoded may be limited to be no greater than M, where M is an integer no less than 1.

Thus, after determining all the image blocks available for updating the specific reference image in the current image to be encoded, if the number of the determined image blocks is not greater than M, updating the specific reference image according to each determined image block. The method for updating the specific reference image according to each image block may refer to the above description, and will not be described herein. If the number of the determined image blocks is greater than M, selecting M image blocks from the determined image blocks, and updating the specific reference image according to the M image blocks.

There are various determination methods for the value of M corresponding to the current image to be encoded. For example, the value of M corresponding to the current image to be encoded is determined based on the type of the current image to be encoded.

In some embodiments, when the type of the current image to be encoded is different, the value of M corresponding to the current image to be encoded is different. For example, when the current image to be encoded is an I image, the current image to be encoded has at most a first preset number of image blocks for updating a specific reference image; when the current image to be coded is a P image (or a forward predictive image), the current image to be coded has at most a second preset numerical image block for updating a specific reference image; when the current image to be coded is a B image (or a bi-directional predictive image), the current image to be coded is provided with at most a third preset value image block for updating the specific reference image, wherein the first preset value, the second preset value and the third preset value are different.

In some embodiments, the value of M corresponding to the current image to be encoded is determined based on the total number of images of the type to which the current image to be encoded belongs. For example, the M value corresponding to the I picture is half of the total number of pictures of the I picture in the video; the corresponding M value of the P picture (or forward predictive picture) is one fourth of the total number of pictures of the P picture (or forward predictive picture); the B picture (or bi-predictive picture) that can be used as the reference picture corresponds to an M value that is one eighth of the total number of pictures of the B picture (or bi-predictive picture).

There are various methods for selecting M image blocks from among all the image blocks determined from the current image to be encoded that can be used to update a specific reference image. For example, in the case where images are stored in terms of components of different color channels, M image blocks of which costs (costs) are smaller than a preset value and which have the smallest costs are selected from all image blocks available for updating a specific reference image, which are determined from the current image to be encoded, for updating the specific reference image.

Wherein the cost (cost) of an image block is the sum of pixel differences of the image block in each color channel; the pixel difference of the image block in each color channel is the sum of the differences between the pixel values of the color channel and the pixels with the same position in the third coded block; the third encoded block refers to a block of an image located at a specific position in the encoded image preceding the current image to be encoded. For example, the third encoded block refers to being located in an encoded image of a previous frame (or two previous frames) of the current image to be encoded. For example, the specific position may be the same as the position of the image block in the current image to be encoded.

For example, an image is stored in three components of YUV. The pixel value difference values of the image block and the third coding block on the YUV three components are Dist Y, dist U and Dist V respectively, and the total number of pixels of the image block on the brightness component is PixCount. Then the cost of the tile (cost) = (disty+distu+distv)/PixCount. When determining that the current image to be coded can be used for updating a specific reference image, determining all image blocks with costs smaller than a preset value in the current image to be coded, and if the number of the determined image blocks is not greater than M, all the image blocks can be used for updating the specific reference image; if the number of the determined image blocks is greater than M, determining that M image blocks with the minimum cost in the current image to be coded can be used for updating the specific reference image.

In some embodiments, a flag bit of the image block is also encoded, the flag bit being used to identify whether the image block is used to update the particular reference image. The code stream sent to the decoding end by the encoding end also comprises the zone bit of each image, wherein the zone bit of each image is used for indicating whether each image block in the image is used for updating a specific reference image.

The method of video encoding is illustrated below in connection with specific examples.

As shown in fig. 3, fig. 3 is a schematic diagram of the relationship between a plurality of images in a video and a specific reference image. In fig. 3, 4 of the images in the video are sequentially encoded, where the first three images are encoded images, and the 4 th image is the current image to be encoded. Of the first three images, the first and third encoded images may be reference images and the second encoded image may not be reference images. Wherein image blocks 11 and 12 in the first encoded image are used for updating a specific reference image. The image block 31, the image block 32, the image block 33 and the image block 34 in the third encoded image are used for updating the specific reference image. For example, the pixel values in image blocks 11, 12, 31, 32, 33, 34 are used to replace the pixel values of image blocks at the same position in a particular reference image, respectively.

A video encoding method when encoding a current image to be encoded is exemplified as follows. Before encoding the current image to be encoded, determining that the current image to be encoded can be used as a reference image according to the type of the current image to be encoded. For example, since the current picture to be encoded is an I picture, it is determined that the current picture to be encoded can be used as a reference picture. Thus, when each image block in the current image to be encoded is encoded separately, it is determined for each image block whether that image block is available for updating a particular reference image.

The method of determining that an image block is available for updating a particular reference image is described below in connection with two examples.

Example one

The image is stored in three components, Y, U, V. Any image block in the current image to be coded is called as image block 1, and the image block with the same position as the image block 1 in the coded image of the previous frame of the current image to be coded is called as image block 2.

In the Y component of image block 1 and image block 2, the sum of the differences in pixel values of pixels at the same positions is Dist Y. In the U components of image block 1 and image block 2, the sum of differences in pixel values of pixels at the same positions is Dist U. In the V component of image block 1 and image block 2, the sum of the differences in pixel values of pixels at the same positions is Dist V.

The luminance component total pixel of image block 1 is PixCount and the large error point count between image block 1 and image block 2 is large dist1. The initial value of large dist1 is set to 0, and large dist1 is accumulated to 1 when a pixel having a difference value of more than a preset value (for example, 20) from the same position where one of the luminance components of the image block 1 and the luminance component of the image block 2 appears.

When the following 4 conditions are simultaneously satisfied, image block 1 is considered to be an alternative block to updating a specific reference image.

a) LargeDist1 is less than a preset proportion (e.g., 1%) of PixCount;

b) Dist Y is less than a preset multiple (e.g., 4 times) of PixCount;

c) Dist U is less than a preset multiple (e.g., 0.5 times) of PixCount;

d) DistV is less than a preset multiple (e.g., 0.5 times) of PixCount.

Among the candidate blocks in the current image to be encoded, which are in line with the update of the specific reference image, when the number of the candidate blocks is not more than M, all the candidate blocks are employed as the update of the specific reference image.

In the alternative blocks which accord with the updating of the specific reference image in the current image to be coded, when the number of the alternative blocks is more than M, recording the cost of each image block as follows: cost= (disty+distu+distv)/PixCount. And selecting M candidate blocks with minimum cost from the current image to be coded for updating the specific reference image.

Example two

The image is stored in three components, Y, U, V. Any image block in the current image to be coded is called as an image block 1, the image block with the same position as the image block 1 in the coded image of the previous frame of the current image to be coded is called as an image 2, and the image block with the same position as the image block 1 in the coded image of the previous two frames of the current image to be coded is called as an image 3.

In the Y component of image block 1 and image block 3, the sum of the differences in pixel values of pixels at the same positions is Dist Y'. In the U components of image block 1 and image block 2, the sum of the differences in pixel values of pixels at the same positions is Dist U'. In the V component of image block 1 and image block 2, the sum of the differences in pixel values of pixels at the same positions is Dist V'.

The luminance component total pixel of image block 1 is PixCount, the large error point count between image block 1 and image block 2 is large dist1, and the large error point count between image block 1 and image block 3 is large dist2. The initial values of large dist1 and large dist2 are set to 0, and large dist1 is accumulated to 1 when a pixel having a difference value of more than a preset value (for example, 20) from the same position as the luminance component of the image block 2 appears in the luminance component of the image block 1. When one pixel value difference value at the same position of the luminance component of the image block 1 as that of the luminance component of the image block 3 occurs more than a preset value (for example, 20) of pixels, large dist2 is added up by 1.

When the following 4 conditions are simultaneously satisfied, image block 1 is considered as an alternative block to update a specific background block.

a) Both large dist1 and large dist2 are less than a preset proportion (e.g., 2%) of PixCount;

b) Dist Y and Dist Y' are both less than a preset multiple (e.g., 6 times) of PixCount;

c) Dist U and Dist U' are each smaller than a predetermined multiple (e.g., 0.5 times) of PixCount

d) DistV and Dist V' are each smaller than a predetermined multiple (e.g., 0.5 times) of PixCount

After determining all the image blocks available for updating the specific reference image in the current image to be encoded (specifically image blocks 41, 42, 43 and 44 in fig. 3), the specific reference image is updated with all the image blocks available for updating the specific reference image. For example, as shown in fig. 3, the pixel values in the image blocks 41, 42, 43, and 44 are used to replace the pixel values of the image block at the same position in the specific reference image, respectively.

As shown in fig. 4, fig. 4 is a flow chart of a video encoding method according to another embodiment of the present invention. As shown in fig. 4, the method includes:

401. when the current image to be coded is available for updating the specific reference image, and when the image block is available for updating the specific reference image, updating the specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be coded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be coded.

There are various methods for determining that a current image to be encoded can be used to update a specific reference image.

In some embodiments, when the current picture to be encoded is available for inter prediction, it is determined that the current picture to be encoded is available for updating the particular reference picture. For example, when it is determined that the current to-be-encoded code image is an intra-prediction image or a forward-prediction image, it is determined that the current to-be-encoded image is available for updating the specific reference image. For example, since an I picture and a P picture (or a forward predictive picture) can be reference pictures for inter prediction of other pictures, when it is determined that a current picture to be encoded is an I picture or a P picture (or a forward predictive picture), it is determined that the current picture to be encoded is a reference picture. In some embodiments, a partial B-picture (or bi-predictive picture) may also be used as a reference picture for inter-prediction of other pictures, e.g., in the hierarchical B (Hierarchical B) technique, a B-picture located at a lower level may be used as a reference frame, and thus, when it is determined that the current picture to be encoded is this partial B-picture, it may also be determined that the current picture to be encoded may be used as a reference picture.

In some embodiments, when the current image to be encoded is an intra-prediction image and/or a random access point, all image blocks of the particular reference image are updated according to all image blocks of the current image to be encoded.

In some embodiments, when an image in a video can be used as a reference image (i.e. can be used for inter prediction), whether the image is used as a long-term reference image or a short-term reference image is not distinguished, so long as the current image to be encoded can be used as a reference image for inter prediction of other images, the current image to be encoded can be determined as the reference image, and then an image block in the image to be encoded can be used for updating a specific image block.

In some embodiments, when an image in the video may be used as a reference image, the long-term reference image and the short-term reference image are distinguished. Wherein the short-term reference picture is a concept corresponding to the long-term reference picture. Short-term reference pictures refer to the fact that they can only exist in the reference picture buffer for a period of time, after which the coded reference pictures after the short-term reference pictures are shifted out of the reference picture buffer after several shifting-in and shifting-out operations in the reference picture buffer. Accordingly, a long-term reference picture (or a part of data of a long-term reference picture) is always present in the reference picture buffer, and the long-term reference picture (or a part of data of a long-term reference picture) is not affected by the shifting-in and shifting-out operations of the encoded or decoded reference picture in the reference picture buffer, and is shifted out of the reference picture buffer only when an update instruction operation is issued by the encoding end. It should be noted that the short-term reference pictures and long-term reference pictures may differ in the different standards, as in the h.264/AVC or h.265/HEVC standards, short-term reference pictures are referred to as short-term reference frames (short-term reference), long-term reference pictures are referred to as long-term reference frames (long-term reference), and long-term reference pictures are referred to as background frames in the AVS1 video and AVS2 video standards, as well as long-term reference pictures are referred to as golden frames in the VP8, VP9 standards.

In some embodiments, when a current image to be encoded is unavailable as a short-term reference image and is unavailable as a long-term reference image, it is determined that an image block in the image to be encoded is unavailable for updating a particular image block in the particular reference image.

In some embodiments, when a current image to be encoded may be a short-term reference image and may be a long-term reference image, it is determined that image blocks in the image to be encoded may be used to update the particular reference image.

In some embodiments, when a current picture to be encoded is not available as a short-term reference picture but is available as a long-term reference picture, it is determined that the picture to be encoded is available for updating the particular reference picture.

In some embodiments, when a current picture to be encoded may be a short-term reference picture but may not be a long-term reference picture, it is determined that the picture to be encoded is not available for updating the particular reference picture, i.e., it is determined that a picture block in the picture to be encoded is not used for updating a particular picture block in the particular reference picture.

In some embodiments, when a current picture to be encoded may be a short-term reference picture but may not be a long-term reference picture, it is determined that the picture to be encoded may be used to update the particular reference picture.

In some embodiments, when a current picture to be encoded may be a short-term reference picture and may be a long-term reference picture, it is determined that the picture to be encoded may be used to update the particular reference picture.

In some embodiments, a flag bit is also added in at least one of the following for identifying whether an image that is not available as a short-term reference image is available for updating the particular reference image:

video parameter set, sequence header, picture parameter set, picture header, slice header, reference picture set, reference configuration set.

Optionally, the flag bit is a time domain scalable flag bit. When there is a temporal scalability requirement, the value of the flag bit is used to indicate that an image block in an image that is not a short-term reference image is not available for updating the particular reference image; and/or, when there is no temporal scalability requirement, the value of the flag bit is used to indicate that an image block in an image that may not be a short-term reference image is available to update the particular reference image.

Considering that an image block in an image which cannot be used as a short-term reference image is not used for updating a specific reference image, the updating speed of the specific reference frame is influenced, and therefore the effect of the specific reference frame on improving the coding quality is influenced, the scheme that the image block in a current image to be coded which cannot be used as a short-term reference image but can be used as a long-term reference image is still used for updating the specific reference frame is adopted, and the influence can be reduced.

Moreover, the significance of the scheme that the image block in the current to-be-encoded image which cannot be used as a short-term reference image but can be used as a long-term reference image can still be used for updating a specific reference image is that when the coding system does not care about the parallel coding characteristic of the short-term reference image or does not care about the characteristic of directly discarding the characteristic of accelerating the decoding speed of the short-term reference image when not care about decoding, the scheme can ensure the effect of the specific reference image on improving the coding quality and can also consider the definition of the image which cannot be used as the reference image in the concept of the short-term reference image; when the encoding system either intends not to use the parallel encoding characteristic of the short-term reference picture or directly discards the characteristic of accelerating the encoding speed when it intends to decode, it may still be specified that the image blocks in the pictures that cannot be used as short-term reference pictures cannot be used for updating the specific reference frame.

Further, a flag bit may be added to at least one of the video parameter set, the sequence header, the picture parameter set, the picture header, the slice header, the reference picture set, and the reference configuration set, where the flag bit is used to identify whether or not the picture block information in the pictures that cannot be used as short-term reference pictures is available for updating the specific reference frame. In some embodiments, the flag bit may also be a time domain scalable flag bit. When the coding system has a time domain scalability requirement, setting a value of the flag bit for indicating that an image block in an image which cannot be a short-term reference image is not available for updating of a specific reference frame; and/or setting a value of the flag bit for indicating that an image block in an image that cannot be a short-term reference image is available for updating of a specific reference frame when the encoding system does not have a temporal scalability requirement.

The explanation of the specific reference image and the specific image block may refer to the explanation of the specific reference image and the specific image block in the above description, and will not be repeated herein.

For how the determined image block may be used to update the specific reference image, reference may be made to the explanation of "the determined image block may be used to update the specific reference image" in the above description, which is not repeated herein.

For how to update the specific image block in the specific reference image according to the image block in the current image to be encoded, reference may be made to the explanation of "update the specific image block in the specific reference image according to the image block in the current image to be encoded" in the above description, which is not described herein.

There are various methods for selecting M image blocks from among all the image blocks determined from the current image to be encoded that can be used to update a specific reference image. The explanation of the above description of the method for selecting M image blocks from all the image blocks that can be used to update the specific reference image, which are determined from the current image to be encoded, is specifically referred to and will not be repeated here.

In video technology, the image information of an image which is not used as a reference image is not used in the encoding and decoding process of other images in the encoding and decoding processes, that is, when a part of the image is not used for updating a specific reference image, the image information is also used for updating the specific reference image. In this embodiment, when it is determined that the current image to be encoded is available for updating the specific reference image, the image block in the current image to be encoded is considered to be used for updating the specific reference image, so that a situation that a part of images are not available for updating the specific reference image and are also used for updating the specific reference image is avoided, for example, a situation that the construction process of the specific reference image is violated with the definition of the images which are not used as reference images is avoided. In addition, the image which is not used as the reference image is not used as the decoding process of other images when being decoded, so the image which is not used as the reference image can not be decoded when being decoded, the decoding speed of the video code stream can be increased, and the variable frame rate playing of the video content can be realized. Further, in some embodiments, when determining that the current image to be encoded can be used as the reference image, the image block in the current image to be encoded is taken into consideration to update the specific reference image, so that the situation that when updating the specific reference image by using the image block in the image which cannot be used as the reference image, discarding part of the image which cannot be used as the reference image cannot be realized is avoided.

In some embodiments, when the current image to be encoded is not available for updating a particular reference image, the particular image block in the particular reference image is not updated with the image block in the current image to be encoded. For example, when the current to-be-encoded image is not available for inter prediction, it is determined that the current to-be-encoded image is not available for updating a particular reference image. For another example, when a current picture to be encoded may be a short-term reference picture but may not be a long-term reference picture, it is determined that a picture block in the picture to be encoded is not used to update a particular picture block in the particular reference picture. For another example, when the current image to be encoded is not available as a short-term reference image or as a long-term reference image, it is determined that an image block in the image to be encoded is not used to update a particular image block in the particular reference image.

In some embodiments, after determining that the current picture to be encoded is available for updating the particular reference picture, the encoding side also writes a parameter or flag bit in at least one of a video parameter set (VPS, video parameter set), a sequence parameter set (SPS, sequence parameter set), a sequence header, a picture header, a slice header, a reference picture set (RPS, reference picture set), and a reference picture configuration set (RCS, reference configuration set) to indicate that the current picture to be encoded is available for updating the particular reference picture.

In some embodiments, after determining that the current image to be encoded is available for updating the particular reference image, a flag bit of an image block in the current image to be encoded is also encoded, the flag bit being used to identify whether the image block in the current image to be encoded is used for updating the particular reference image.

The video encoding method according to an embodiment of the present invention is described above from the encoding side with reference to fig. 1, 2, 3 and 4, and the video encoding method according to another embodiment of the present invention will be described below in detail from the decoding side with reference to fig. 5. Fig. 5 illustrates a video decoding method according to another embodiment of the present invention, which may be performed by an image processing apparatus, which may be various types of chips for image processing, an image processor, or the like. As shown in fig. 5, the video decoding method includes:

501. when a current image to be decoded can be used as a reference image, and when an image block can be used for updating a specific reference image, updating a specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be decoded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be decoded.

In some embodiments, determining that the current image to be decoded can be used as a reference image can occur prior to decoding the image to be decoded. In this way, it is possible to judge that each image block satisfies the condition for updating the specific reference image at the time of decoding each image block in the image to be decoded or after decoding, based on the result of the determination. Or determining that the current image to be decoded can be used as a reference image, or can occur when each image block in the image to be decoded is decoded or after decoding, namely, when each image block is decoded or after decoding, firstly determining that the image in which the image block is currently located is an image which can be used as the reference image, and when the image block is determined to be used as the reference image, judging that the image block can be used for updating the specific reference image.

In some embodiments, determining that the current image to be decoded can be used as the reference image can be performed by acquiring a parameter or a flag bit indicating a reference relationship of the current image to be decoded, and determining whether the current image to be decoded is an image that can be used as the reference image according to the parameter or the flag bit. Wherein, parameters or flag bits for indicating the reference relation of the current image to be decoded can be acquired through various approaches. For example, a parameter or flag indicating a reference relationship of a current image to be decoded may be acquired from at least one of a video parameter set (VPS, video parameter set), a sequence parameter set (SPS, sequence parameter set), a sequence header (sequence header), a picture header (slice header), a slice header (slice header), a reference image set (RPS, reference picture set), and a reference image configuration set (RCS, reference configuration set).

In some embodiments, since the I picture and the P picture (or the forward predictive picture) may be reference pictures for inter prediction of other pictures, when it is determined that the current picture to be decoded is the I picture or the P picture (or the forward predictive picture), it is determined that the current picture to be decoded is a picture that can be a reference picture. In some embodiments, a partial B-picture (or bi-predictive picture) may also be used as a reference picture for inter-prediction, e.g., in the hierarchical B (Hierarchical B) technique, a B-picture located at a lower level may be used as a reference frame, and thus, when it is determined that the current picture to be encoded is this partial B-picture, it may also be determined that the current picture to be encoded may be used as a reference picture.

The method for judging that the image block is available for updating the specific reference image may refer to the method for judging that the image block in the current image to be encoded is available for updating the specific reference image in the above description. Or in some embodiments, the decoding end also parses out the flag bit of each image from the code stream, where the flag bit of each image is used to indicate whether each image block in the image is used to update a specific reference image. The decoding end can acquire the zone bit of the image block from the zone bit, and judge whether the image block is used for updating the specific reference image according to the zone bit.

The method for updating the specific image block in the specific reference image according to the image block may refer to the method for updating the specific image block in the specific reference image according to the image block in the current image to be encoded in the above description, which is not described herein. The specific how the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be decoded can refer to how the position of the specific image block in the specific reference image in step 101 is determined by the position of the image block in the current image to be encoded, which is not described herein.

In some embodiments, the number of image blocks in the current image to be decoded, which may be used to update the specific reference image, may be 1 or greater than 1. In some embodiments, the number of image blocks in the current image to be decoded that can be used to update the particular reference image may be unlimited, i.e., all image blocks in the current image to be decoded that satisfy the conditions that can be used to update the particular reference image are used to update the particular reference image.

In some embodiments, the number of image blocks available for updating the specific reference image is larger, and from the standpoint of complexity of implementation of the codec system, the number of image blocks available for updating the specific reference image in the current image to be decoded may be limited to be no greater than M, where M is an integer no less than 1.

Thus, after determining all the image blocks available for updating the specific reference image in the current image to be decoded, if the number of the determined image blocks is not greater than M, updating the specific reference image according to each determined image block. The method for updating the specific reference image according to each image block may refer to the above description, and will not be described herein. If the number of the determined image blocks is greater than M, selecting M image blocks from the determined image blocks, and updating the specific reference image according to the M image blocks.

The value of M corresponding to the current image to be decoded may be based on a variety of determination methods. For example, the value of M corresponding to the current image to be decoded is determined based on the type of the current image to be decoded. In some embodiments, when the image types of the current image to be decoded are different, the values of M corresponding to the current image to be decoded are different. For example, when the current image to be decoded is an I image, the current image to be decoded has at most a first preset number of image blocks for updating a specific reference image; when the current image to be decoded is a P image (or a forward predictive image), the current image to be decoded has at most a second preset numerical image block for updating a specific reference image; when the current image to be decoded is a B image (or a bi-directional predicted image), the current image to be decoded is provided with at most a third preset value image block for updating the specific reference image, wherein the first preset value, the second preset value and the third preset value are different.

In some embodiments, the value of M corresponding to the current image to be decoded is determined based on the total number of images of the type to which the current image to be decoded belongs. For example, the M value corresponding to the I picture is half of the total number of pictures of the I picture in the video; the corresponding M value of the P picture (or forward predictive picture) is one fourth of the total number of pictures of the P picture (or forward predictive picture); the B picture (or bi-predictive picture) that can be used as the reference picture corresponds to an M value that is one eighth of the total number of pictures of the B picture (or bi-predictive picture).

In some embodiments, the number of image blocks in the current image to be decoded that can be used to update the particular reference image is carried in at least one of: the method comprises the steps of an image header of a current image to be decoded, an image parameter set of the current image to be decoded, a sequence header corresponding to the current image to be decoded, a sequence parameter set corresponding to the current image to be decoded and a video parameter set corresponding to the current image to be decoded. The decoding end can analyze the number of image blocks which can be used for updating the specific reference image in the current image to be decoded.

In some embodiments, the value of M corresponding to the current image to be decoded is only used to inform the decoding end how many image blocks of the current image are available for updating the specific reference frame, so that the design of the decoding end can be facilitated, and the complexity of the decoding end can be reduced.

As shown in fig. 6, fig. 6 is a flow chart of a video decoding method according to another embodiment of the present invention. As shown in fig. 6, the method includes:

601. when a current image to be coded is available for updating a specific reference image, and when an image block is available for updating the specific reference image, updating a specific image block in the specific reference image according to the image block, wherein the image block is one image block in the current image to be coded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be coded.

In some embodiments, when the current to-be-decoded picture is available for inter prediction, it is determined that the current to-be-decoded picture is available for updating the particular reference picture. For example, since the I picture and the P picture (or the forward predictive picture) can be reference pictures for inter prediction of other pictures, when it is determined that the current picture to be decoded is the I picture or the P picture (or the forward predictive picture), it is determined that the current picture to be decoded is a picture that can be a reference picture. In some embodiments, a partial B-picture (or bi-predictive picture) may also be used as a reference picture for inter-prediction, e.g., in the hierarchical B (Hierarchical B) technique, a B-picture located at a lower level may be used as a reference frame, and thus, when it is determined that the current picture to be encoded is this partial B-picture, it may also be determined that the current picture to be encoded may be used as a reference picture.

In some embodiments, when the current picture to be encoded is an intra-predicted picture and/or a random access point, all picture blocks of the particular reference picture are updated according to all picture blocks of the current picture to be decoded. For specific explanation, reference is made to the explanation of "updating all image blocks of the specific reference image according to all image blocks of the current image to be decoded" above, and details thereof are not repeated here.

The current image to be decoded can be used as a reference image (namely, can be used for inter prediction), and whether the current image to be decoded is an image which can be used as the reference image can be determined according to the parameter or the flag bit by acquiring the parameter or the flag bit which is used for indicating the reference relation of the current image to be decoded. Wherein, the reference relation of the current to-be-decoded image may refer to that the current band decoded image is a short-term reference image or a long-term reference image. Wherein, parameters or flag bits for indicating the reference relation of the current image to be decoded can be acquired through various approaches. For example, a parameter or flag indicating a reference relationship of a current image to be decoded may be acquired from at least one of a video parameter set (VPS, video parameter set), a sequence parameter set (SPS, sequence parameter set), a sequence header (sequence header), a picture header (slice header), a slice header (slice header), a reference image set (RPS, reference picture set), and a reference image configuration set (RCS, reference configuration set).

In some embodiments, when an image in the video can be used as a reference image, whether the image is used as a long-term reference image or a short-term reference image is not distinguished, so long as the current image to be decoded can be used as a reference image for inter-frame prediction of other images, the current image to be decoded can be determined as the reference image, and the current image to be encoded can be used for updating a specific reference image.

In some embodiments, when an image in the video may be used as a reference image, the long-term reference image and the short-term reference image are distinguished. Wherein the short-term reference picture is a concept corresponding to the long-term reference picture. Short-term reference pictures refer to reference pictures that can only exist in a reference picture buffer for a period of time after which a decoded reference picture is moved out of the reference picture buffer after several move-in and move-out operations in the reference picture buffer. Accordingly, the long-term reference picture (or a part of data in the long-term reference picture) is always present in the reference picture buffer, and the long-term reference picture (or a part of data in the long-term reference picture) is not affected by the move-in and move-out operations of the decoded reference picture in the reference picture buffer, and is moved out of the reference picture buffer only when the update instruction operation is issued by the decoding end. It should be noted that the short-term reference pictures and long-term reference pictures may differ in the different standards, as in the h.264/AVC or h.265/HEVC standards, short-term reference pictures are referred to as short-term reference frames (short-term reference), long-term reference pictures are referred to as long-term reference frames (long-term reference), and long-term reference pictures are referred to as background frames in the AVS1 video and AVS2 video standards, as well as long-term reference pictures are referred to as golden frames in the VP8, VP9 standards.

In some embodiments, when a current picture to be decoded is not available as a short-term reference picture and is not available as a long-term reference picture, it is determined that the picture to be decoded is not available for updating the particular reference picture, i.e., it is determined that a picture block in the picture to be decoded is not used for updating a particular picture block in the particular reference picture.

In some embodiments, when a current image to be decoded may be a short-term reference image and may be a long-term reference image, it is determined that image blocks in the image to be decoded may be used to update the particular reference image.

In some embodiments, when a current picture to be decoded is not available as a short-term reference picture but is available as a long-term reference picture, it is determined that the picture to be decoded is available for updating the particular reference picture.

In some embodiments, when a current picture to be decoded is available as a short-term reference picture but not as a long-term reference picture, it is determined that the picture to be decoded is not available for updating the particular reference picture, i.e., it is determined that a picture block in the picture to be decoded is not used for updating a particular picture block in the particular reference picture.

In some embodiments, when a current picture to be decoded may be a short-term reference picture but may not be a long-term reference picture, it is determined that the picture to be decoded may be used to update the particular reference picture.

In some embodiments, when a current picture to be decoded may be a short-term reference picture and may be a long-term reference picture, it is determined that the picture to be decoded may be used to update the particular reference picture.

There are various methods of determining which of the current band decoded picture is, for example, whether the current picture is a short-term reference picture or a long-term reference picture can be obtained by parsing in a video parameter set, a sequence header, a picture parameter set, a picture header, a slice header, a reference picture set, a reference configuration set.

Further, a flag bit can be resolved from at least one of a video parameter set, a sequence header, a picture parameter set, a picture header, a slice header, a reference picture set, and a reference configuration set, where the flag bit is used to identify whether the picture to be decoded is available for updating the specific reference picture. In some embodiments, the flag bit may also be a time domain scalable flag bit. When the decoding end has the time domain scalability requirement, the value of the flag bit is used for indicating that the image block in the image which cannot be the short-term reference image cannot be used for updating the specific reference frame; and/or when the decoding end does not have the time domain scalability requirement, the value of the flag bit is used for indicating that the image block in the image which cannot be the short-term reference image can be used for updating the specific reference frame.

In some embodiments, a flag bit of an image block of the current image to be decoded is also obtained, where the flag bit is used to identify whether the image block in the current image to be decoded is used to update the specific reference image. And determining the image blocks which can be used for updating the specific reference image according to the zone bits.

For how to update the specific image block in the specific reference image according to the image block in the current image to be decoded, reference may be made to the explanation of "update the specific image block in the specific reference image according to the image block in the current image to be decoded" in the above description, which is not described herein.

In some embodiments, when it is determined that the current to-be-decoded image is not available for inter prediction of an image other than the current to-be-decoded image, it is determined that the image block is not used to update the particular reference image.

In some embodiments, when a current picture to be decoded may be a short-term reference picture but may not be a long-term reference picture, it is determined that a picture block in the picture to be decoded is not used to update the particular reference picture.

In some embodiments, when a current picture to be decoded is unavailable as a short-term reference picture and is unavailable as a long-term reference picture, it is determined that a picture block in the picture to be decoded is unavailable for updating the particular reference picture.

Fig. 7 is a schematic block diagram of a video encoding apparatus according to an embodiment of the present invention. As shown in fig. 7, the video encoding apparatus 70 includes:

at least one memory 701 for storing computer-executable instructions;

at least one processor 702, individually or collectively, is configured to: accessing the at least one memory and executing the computer-executable instructions to perform the operations of:

In some embodiments, the processor is further configured to:

when the current image to be coded is an intra-frame predicted image or a forward predicted image, the current image to be coded is determined to be a reference image.

In some embodiments, the processor is further configured to:

and when the current image to be coded is an intra-frame predicted image and/or a random access point, updating all image blocks of the specific reference image according to all image blocks of the current image to be coded.

In some embodiments, the number of image blocks in the current image to be encoded that can be used to update the particular reference image is no greater than M, M being an integer no less than 1.

In some embodiments, the value of M corresponding to the current image to be encoded is determined based on a reference relationship of the current image to be encoded.

In some embodiments, when the types of the current to-be-encoded images are different, the values of M corresponding to the current to-be-encoded images are different.

In some embodiments, the value of M corresponding to the current image to be encoded is determined based on the total number of images of the type to which the current image to be encoded belongs.

In some embodiments, the image block includes a preset number of color channel pixels,

In the sorting of the cost (cost) of each image block in the current image to be coded from small to large, the cost of the image block is positioned in the first M, and the cost of the image block is not less than a preset value; wherein,

the cost of the image block is the sum of pixel differences of the image block in each color channel;

the pixel difference of the image block in each color channel is the sum of the differences between the pixel values of the color channels of the pixels with the same positions in the encoded block and the pixels of the image block;

the encoded block is an image block located at a specific position in an encoded image preceding the current image to be encoded.

In some embodiments, the processor is further configured to:

and encoding a flag bit of the image block, wherein the flag bit is used for identifying whether the image block is used for updating the specific reference image.

In some embodiments, the position of the particular tile in the particular reference image is determined by the position of the tile in the current image to be encoded, comprising:

the position of the specific image block in the specific reference image is the same as the position of the image block in the current image to be encoded, or,

The position of the specific image block in the specific reference image is obtained by adding a preset offset value to the position of the image block in the current image to be coded.

In some embodiments, the processor is further configured to:

when the current image to be encoded cannot be used as a reference image, determining that the image block is not used for updating the specific reference image.

In some embodiments, the particular reference picture is a reference picture that is inter-prediction.

In some embodiments, the particular reference image is a long-term reference frame, or a background frame, or a golden frame.

In some embodiments, the means for determining that an image block is available for updating a particular reference image comprises:

and judging that the image block can be used for updating a specific reference image according to the pixel information of the image block and the pixel information of an encoded block, wherein the encoded block is the image block positioned at a specific position in the encoded image before the current image to be encoded.

In some embodiments, the determining that an image block is available for updating a particular reference image based on the image block and the encoded block comprises:

the image block is determined to be usable for updating the particular reference image based on at least one of:

A pixel value difference of a luminance component between the image block and the encoded block, a total number of pixels of a luminance component of the image block and/or the encoded block, a pixel value difference of a chrominance component between the image block and the encoded block, a total number of pixels of a chrominance component of the image block and/or the encoded block.

Fig. 8 is another schematic block diagram of a video encoding device according to an embodiment of the present invention. As shown in fig. 8, the video encoding apparatus 80 includes:

at least one memory 801 for storing computer-executable instructions;

at least one processor 802, individually or collectively, for: accessing the at least one memory and executing the computer-executable instructions to perform the operations of:

In some embodiments, the processor is further configured to:

when the current image to be coded is not available for updating the specific reference image, the specific image block in the specific reference image is not updated by the image block in the current image to be coded.

In some embodiments, the processor is further configured to:

when the current to-be-encoded image is not available for inter prediction, it is determined that the current to-be-encoded image is not available for updating a particular reference image.

In some embodiments, the processor is further configured to:

when a current picture to be encoded may be a short-term reference picture but may not be a long-term reference picture, it is determined that a picture block in the picture to be encoded is not used to update a particular picture block in the particular reference picture.

In some embodiments, the processor is further configured to:

when the current image to be encoded cannot be used as a short-term reference image or a long-term reference image, determining that the image block in the image to be encoded is not used for updating the specific image block in the specific reference image.

In some embodiments, the processor is further configured to:

When the current to-be-encoded image is available for inter prediction, it is determined that the current to-be-encoded image is available for updating the particular reference image.

In some embodiments, the processor is further configured to:

when the current to-be-encoded code image is an intra-frame prediction image or a forward prediction image, determining that the current to-be-encoded image is available for updating the specific reference image.

In some embodiments, the processor is further configured to:

when a current picture to be encoded is not available as a short-term reference picture but is available as a long-term reference picture, it is determined that the picture to be encoded is available for updating the particular reference picture.

In some embodiments, the processor is further configured to:

when a current image to be encoded may be a short-term reference image and may be a long-term reference image, it is determined that image blocks in the image to be encoded may be used to update the particular reference image.

In some embodiments, the processor is further configured to:

when a current image to be encoded may be a short-term reference image but may not be a long-term reference image, it is determined that image blocks in the image to be encoded may be used to update the particular reference image.

In some embodiments, a flag bit is added in at least one of the following to identify whether an image that is not available as a short-term reference image is available to update the particular reference image:

In some embodiments, the flag bit is a time domain scalable flag bit.

In some embodiments, when there is a temporal scalability requirement, the value of the flag bit is used to indicate that image blocks in images that may not be short-term reference images are not available to update the particular reference image; and/or the number of the groups of groups,

when there is no temporal scalability requirement, the value of the flag bit is used to indicate that image blocks in images that cannot be short-term reference images are available to update the particular reference image.

In some embodiments, the particular reference picture is a picture that is available for inter prediction.

Fig. 9 is a schematic block diagram of a video decoding apparatus according to an embodiment of the present invention. As shown in fig. 8, the video encoding apparatus 90 includes:

at least one memory 901 for storing computer executable instructions;

at least one processor 902, individually or collectively, for: accessing the at least one memory and executing the computer-executable instructions to perform the operations of:

In some embodiments, the processor is further configured to:

acquiring a zone bit of the image block;

and determining that the image block can be used for updating a specific reference image according to the zone bit.

In some embodiments, the processor is further configured to:

when the current image to be decoded is an intra-frame predicted image or a forward predicted image, the current image to be decoded is determined to be a reference image.

In some embodiments, the processor is further configured to:

acquiring parameters or flag bits for indicating the reference relation of the current image to be decoded;

and determining that the current image to be decoded can be used as a reference image according to the parameter or the flag bit.

In some embodiments, the obtaining a parameter or a flag bit for indicating a reference relationship of the current image to be decoded includes:

acquiring parameters or flag bits for indicating the reference relation of the current image to be decoded from at least one of the following:

In some embodiments, the processor is further configured to:

and when the current image to be decoded is an intra-frame predicted image and/or a random access point, updating all image blocks of the specific reference image according to all image blocks of the current image to be decoded.

In some embodiments, the number of image blocks in the current image to be decoded that can be used to update the particular reference image is M, which is an integer no less than 1.

In some embodiments, the value of M corresponding to the current image to be decoded is determined based on a reference relationship of the current image to be decoded.

In some embodiments, when the types of the current to-be-decoded images are different, the values of M corresponding to the current to-be-decoded images are different.

In some embodiments, the value of M corresponding to the current image to be decoded is determined based on the total number of images of the type to which the current image to be decoded belongs.

In some embodiments, the number of image blocks in the current image to be decoded that can be used to update the particular reference image is included in at least one of:

the method comprises the steps of obtaining an image header of a current image to be decoded, an image parameter set of the current image to be decoded, a sequence header corresponding to the current image to be decoded, a sequence parameter set corresponding to the current image to be decoded and a video parameter set corresponding to the current image to be decoded.

In some embodiments, the processor is further configured to:

when it is determined that the current image to be decoded is not available as a reference image, it is determined that the image block is not used to update a particular image block in the particular reference image.

Fig. 10 is a schematic block diagram of a video decoding apparatus according to an embodiment of the present invention. As shown in fig. 8, the video encoding apparatus 100 includes:

at least one memory 1001 for storing computer-executable instructions;

at least one processor 1002, individually or collectively, for: accessing the at least one memory and executing the computer-executable instructions to perform the operations of:

In some embodiments, the processor is further configured to:

when the current image to be decoded is not available for updating the specific reference image, the specific image block in the specific reference image is not updated by the image block in the current image to be decoded.

In some embodiments, the processor is further configured to:

acquiring a zone bit of the image block;

In some embodiments, the processor is further configured to:

and determining that the current image to be decoded can be used for updating a specific reference image according to the parameter or the flag bit.

In some embodiments, the processor is further configured to:

when it is determined that the current image to be decoded is not available for inter prediction except for an image of the current image to be decoded, it is determined that the image block is not used to update a specific image block in the specific reference image.

In some embodiments, the processor is further configured to:

when a current picture to be decoded can be used as a short-term reference picture but not as a long-term reference picture, it is determined that a picture block in the picture to be decoded is not used to update a particular picture block in the particular reference picture.

In some embodiments, the processor is further configured to:

when the current to-be-decoded picture is available for inter prediction, it is determined that the current to-be-decoded picture is available for updating the particular reference picture.

In some embodiments, the processor is further configured to:

when the current to-be-decoded image is an intra-frame predicted image or a forward predicted image, it is determined that the current to-be-decoded image is available for updating the specific reference image.

In some embodiments, the processor is further configured to:

when a current to-be-decoded picture is not available as a short-term reference picture but is available as a long-term reference picture, it is determined that the to-be-decoded picture is available for updating the particular reference picture.

In some embodiments, the processor is further configured to:

when a current to-be-decoded image can be used as a short-term reference image and can be used as a long-term reference image, it is determined that image blocks in the to-be-decoded image can be used to update the particular reference image.

In some embodiments, the reference relationship of the current image to be decoded is: the current band decoded picture is a short-term reference picture or a long-term reference picture.

In some embodiments, the processor is further configured to:

when a current to-be-decoded picture is unavailable as a short-term reference picture and is unavailable as a long-term reference picture, it is determined that a picture block in the to-be-decoded picture is unavailable for updating the particular reference picture.

In some embodiments, the processor is further configured to: obtaining a flag bit from at least one of the following, the flag bit being used to identify whether the image to be decoded is available for updating the specific reference image:

In some embodiments, the flag bit is a time domain scalable flag bit.

In some embodiments, when the flag bit or scalable flag bit is a first value, the value of the flag bit is used to indicate that an image block in an image that may not be a short-term reference image is not available to update the particular reference image; and/or the number of the groups of groups,

and when the flag bit or the telescopic flag bit is a second numerical value, the value of the flag bit is used for indicating that the image block in the image which cannot be the short-term reference image can be used for updating the specific reference image.

Alternatively, the processor in this embodiment may be a central processing unit (Central Processing Unit, abbreviated as "CPU"), a network processor (Network Processor, abbreviated as "NP"), or a combination of CPU and NP. The processor may further comprise a hardware chip. The hardware chip may be an Application-specific integrated circuit (ASIC), a programmable logic device (Programmable Logic Device, PLD), or a combination thereof. The PLD may be a complex programmable logic device (Complex Programmable Logic Device, abbreviated as "CPLD"), a Field programmable gate array (Field-Programmable Gate Array, abbreviated as "FPGA"), a general-purpose array logic (Generic Array Logic, abbreviated as "GAL"), or any combination thereof.

Optionally, the memory in embodiments of the invention may include read only memory and random access memory, and provide instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store information of the device type.

It should also be understood that the various numbers referred to herein are merely descriptive convenience and are not intended to limit the scope of embodiments of the invention.

It should also be understood that, in some embodiments, the sequence number of each process does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Those of ordinary skill in the art will appreciate that the modules of the examples described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or as a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules may be combined or integrated into another module, or some features may be omitted or not performed.

In addition, each functional module in the embodiment of the apparatus may be integrated in one processing unit, or each functional module may be physically present in a separate processing unit, or two or more functional modules may be integrated in one processing unit.

The foregoing is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope disclosed in the embodiments of the present application, and all changes and substitutions are included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A video encoding method, comprising:

updating a specific image block in a specific reference image according to an image block when a current image to be encoded can be used as a reference image and when the image block can be used for updating the specific reference image, and not updating the specific image block in the specific reference image according to the image block when the current image to be encoded cannot be used as the reference image, wherein the image block is one image block in the current image to be encoded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be encoded;

The method further comprises the steps of:

encoding a first flag bit of the current image to be encoded, and newly adding the first flag bit in a code stream, wherein the first flag bit is a flag bit carried in a sequence header, the first flag bit is used for indicating whether the current image to be encoded can be used for updating the specific reference image, an intra-frame predicted image, a forward predicted image and a bi-directional predicted image can be used for updating the specific reference image, and a second flag bit is newly added in a second flag bit of the image block, wherein the second flag bit is used for indicating whether the image block is used for updating the specific reference image when the first flag bit indicates that the current image to be encoded can be used for updating the specific reference image; and/or

And when the current image to be coded is a random access point, updating all image blocks of the specific reference image according to all image blocks of the current image to be coded.

2. The video coding method of claim 1, wherein the method further comprises:

3. The video coding method of claim 1, wherein the method further comprises:

and when the current image to be coded is an intra-frame prediction image, updating all image blocks of the specific reference image according to all image blocks of the current image to be coded.

4. The video coding method according to claim 1, wherein the number of image blocks in the current image to be coded that can be used to update the specific reference image is not greater than M, M being an integer not less than 1.

5. The method according to claim 4, wherein the value of M corresponding to the current image to be encoded is determined based on a reference relationship of the current image to be encoded.

6. The video coding method according to claim 5, wherein when the types of the current to-be-coded pictures are different, the values of M corresponding to the current to-be-coded pictures are different.

7. The video coding method according to claim 5, wherein the value of M corresponding to the current picture to be coded is determined based on the total number of pictures of the type to which the current picture to be coded belongs.

8. The video coding method of claim 4, wherein the image block comprises a preset number of color channel pixels,

9. The video coding method according to claim 1, wherein the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be coded, comprising:

10. The video encoding method according to claim 1, wherein the specific reference picture is a reference picture that is inter-prediction.

11. The video coding method of claim 10, wherein the particular reference picture is a long-term reference frame, or a background frame, or a golden frame.

12. The video coding method according to claim 1, wherein the method of determining that an image block is available for updating a specific reference image comprises:

13. The video coding method of claim 12, wherein: the determining that the image block is available for updating the particular reference image based on the image block and the encoded block includes:

14. A video decoding method, comprising:

updating a specific image block in a specific reference image according to an image block when a current image to be decoded can be used as a reference image and when the image block can be used for updating the specific reference image, and not updating the specific image block in the specific reference image according to the image block when the current image to be decoded cannot be used as the reference image, wherein the image block is one image block in the current image to be decoded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be decoded;

the method further comprises the steps of:

acquiring a first flag bit of the current image to be decoded, wherein the first flag bit is a flag bit carried in a sequence header, the first flag bit is used for indicating whether the current image to be decoded can be used for updating the specific reference image, an intra-frame predicted image, a forward predicted image and a bi-directional predicted image can be used for updating the specific reference image, and acquiring a second flag bit of the image block, wherein the second flag bit is used for indicating whether the image block is used for updating the specific reference image when the first flag bit indicates that the current image to be decoded can be used for updating the specific reference image; and/or

And when the current image to be decoded is a random access point, updating all image blocks of the specific reference image according to all image blocks of the current image to be decoded.

15. The video decoding method of claim 14, wherein the method further comprises:

16. The video decoding method of claim 14, wherein the method further comprises:

17. The video decoding method according to claim 16, wherein the obtaining a parameter or a flag indicating a reference relation of the current image to be decoded includes:

18. The video decoding method of claim 14, wherein the method further comprises:

and when the current image to be decoded is an intra-frame prediction image, updating all image blocks of the specific reference image according to all image blocks of the current image to be decoded.

19. The video decoding method of claim 14, wherein the number of image blocks in the current picture to be decoded that can be used to update the particular reference picture is M, M being an integer no less than 1.

20. The video decoding method of claim 19, wherein the value of M for the current image to be decoded is determined based on a reference relationship of the current image to be decoded.

21. The method according to claim 20, wherein when the types of the current to-be-decoded images are different, the values of M corresponding to the current to-be-decoded images are different.

22. The method according to claim 20, wherein the value of M corresponding to the current image to be decoded is determined based on the total number of images of the type to which the current image to be decoded belongs.

23. The video decoding method of claim 19, wherein the number of image blocks in the current image to be decoded that can be used to update the particular reference image is included in at least one of:

24. The video decoding method of claim 14, wherein the particular reference picture is a reference picture that is inter-prediction.

25. The video decoding method of claim 24, wherein the particular reference picture is a long-term reference frame, or is a background frame, or is a golden frame.

26. A video encoding apparatus, comprising:

at least one memory for storing computer-executable instructions;

The operations further comprise:

27. The video encoding device of claim 26, wherein the processor is further configured to:

28. The video encoding device of claim 26, wherein the processor is further configured to:

29. The video encoding device of claim 26, wherein a number of image blocks in the current image to be encoded that are available for updating the particular reference image is no greater than M, M being an integer no less than 1.

30. The video coding device of claim 29, wherein the value of M for the current picture to be coded is determined based on a reference relationship of the current picture to be coded.

31. The video coding device of claim 30, wherein when the types of the current to-be-coded pictures are different, the values of M corresponding to the current to-be-coded pictures are different.

32. The video coding device of claim 30, wherein the value of M for the current picture to be coded is determined based on a total number of pictures of a type to which the current picture to be coded belongs.

33. The video encoding device of claim 29, wherein the image block comprises a preset number of color channel pixels,

34. The video encoding device of claim 26, wherein the position of the particular tile in the particular reference picture is determined by the position of the tile in the current picture to be encoded, comprising:

35. The video encoding device of claim 26, wherein the particular reference picture is a reference picture that is inter-prediction.

36. The video coding device of claim 35, wherein the particular reference picture is a long-term reference frame, or is a background frame, or is a golden frame.

37. The video coding device of claim 26, wherein the means for determining that an image block is available for updating a particular reference image comprises:

38. The video coding device of claim 37, wherein: the determining that the image block is available for updating the particular reference image based on the image block and the encoded block includes:

39. A video decoding apparatus, comprising:

at least one memory for storing computer-executable instructions;

the operations further comprise:

40. The video decoding device of claim 39, wherein the processor is further configured to:

41. The video decoding device of claim 39, wherein the processor is further configured to:

42. The video decoding device of claim 41, wherein the obtaining a parameter or flag indicating a reference relationship for the current image to be decoded comprises:

43. The video decoding device of claim 39, wherein the processor is further configured to:

44. The video decoding device of claim 39, wherein the number of image blocks in the current picture to be decoded that are available for updating the particular reference picture is M, M being an integer not less than 1.

45. The video decoding device of claim 44, wherein the value of M for the current image to be decoded is determined based on a reference relationship for the current image to be decoded.

46. The video decoding device of claim 45, wherein when the types of the current to-be-decoded pictures are different, the values of M corresponding to the current to-be-decoded pictures are different.

47. The video decoding device of claim 45, wherein the value of M for the current picture to be decoded is determined based on a total number of pictures of a type to which the current picture to be decoded belongs.

48. The video decoding device of claim 44, wherein the number of image blocks in the current picture to be decoded that are available for updating the particular reference image is included in at least one of:

49. The video decoding device of claim 39, wherein the particular reference picture is a reference picture that is inter-prediction.

50. A video decoding device as defined in claim 49, wherein the particular reference picture is a long-term reference frame, or is a background frame, or is a golden frame.

51. A machine-readable storage medium adapted for use in a terminal, the machine-readable storage medium having stored thereon computer instructions which, when executed, perform the video encoding method of any of claims 1-13.

52. A video decoding apparatus, comprising:

at least one memory for storing computer-executable instructions;

the operations further comprise:

53. The video decoding device of claim 52, wherein the processor is further configured to:

54. The video decoding device of claim 52, wherein the processor is further configured to:

55. The video decoding device of claim 54, wherein the obtaining a parameter or flag indicating a reference relationship for the current image to be decoded comprises:

56. The video decoding device of claim 52, wherein the processor is further configured to:

57. The video decoding device of claim 52, wherein the number of image blocks in the current picture to be decoded that are available for updating the particular reference picture is M, M being an integer not less than 1.

58. The video decoding device of claim 57, wherein the value of M for the current picture to be decoded is determined based on a reference relationship for the current picture to be decoded.

59. The video decoding device of claim 58, wherein when the types of the current to-be-decoded pictures are different, the values of M corresponding to the current to-be-decoded pictures are different.

60. The video decoding device of claim 58, wherein the value of M for the current picture to be decoded is determined based on a total number of pictures of a type to which the current picture to be decoded belongs.

61. The video decoding device of claim 57, wherein the number of image blocks in the current picture to be decoded that are available for updating the particular reference image is included in at least one of:

62. The video decoding device of claim 52, wherein the particular reference picture is a reference picture that is inter-prediction.

63. The video decoding device of claim 62, wherein the particular reference picture is a long-term reference frame, or is a background frame, or is a golden frame.

64. A machine-readable storage medium adapted for use in a terminal, the machine-readable storage medium having stored thereon computer instructions which, when executed, perform the video decoding method of any of claims 14-25.

65. A video encoding method, comprising:

updating a specific image block in a specific reference image according to an image block when a current image to be encoded is available for updating the specific reference image and updating the specific image block in the specific reference image according to the image block when the image block is available for updating the specific reference image, and updating the specific image block in the specific reference image without the image block in the current image to be encoded when the current image to be encoded is not available for updating the specific reference image, wherein the image block is one image block in the current image to be encoded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be encoded;

The method further comprises the steps of:

66. The method of video encoding according to claim 65, further comprising:

when a current image to be encoded is not available for updating a particular reference image, the particular image block in the particular reference image is not updated with the image block in the current image to be encoded.

67. The method of video encoding according to claim 65, further comprising:

68. The method of video encoding according to claim 65, further comprising:

69. The method of video encoding according to claim 65, wherein the number of image blocks in the current image to be encoded that are available for updating the specific reference image is not greater than M, M being an integer not less than 1.

70. The method of claim 69, wherein the value of M corresponding to the current picture to be encoded is determined based on a reference relationship of the current picture to be encoded.

71. The method according to claim 70, wherein when the types of the current to-be-encoded images are different, the values of M corresponding to the current to-be-encoded images are different.

72. The method of claim 70, wherein the value of M for the current picture to be encoded is determined based on a total number of pictures of a type to which the current picture to be encoded belongs.

73. The video encoding method of claim 69, wherein the image block includes a preset number of color channel pixels,

74. The method of video encoding according to claim 65, wherein the position of the particular picture block in the particular reference picture is determined by the position of the picture block in the current picture to be encoded, comprising:

75. The method of video encoding of claim 66, wherein the method further comprises:

when the current to-be-encoded image is not available for inter prediction, determining that the image block in the current to-be-encoded image is not available for updating the particular image block in the particular reference image.

76. The method of video encoding of claim 66, wherein the method further comprises:

when a current picture to be encoded may be a short-term reference picture but may not be a long-term reference picture, it is determined that the picture block in the picture to be encoded is not used to update the particular picture block in the particular reference picture.

77. The method of video encoding of claim 66, wherein the method further comprises:

when a current image to be encoded cannot be used as a short-term reference image or a long-term reference image, determining that the image block in the image to be encoded is not used for updating the specific image block in the specific reference image.

78. The method of video coding according to claim 65 or 66, further comprising:

79. The method of video encoding of claim 78, wherein the method further comprises:

80. The method of video coding according to claim 65 or 66, further comprising:

81. The method of video coding according to claim 65 or 66, further comprising:

82. The method of video coding according to claim 65 or 66, further comprising:

83. The method of video coding according to any one of claims 65, 76, 77, wherein a flag bit is added in at least one of the following to identify whether an image that is not available as a short-term reference image is available for updating the particular reference image:

84. The method of claim 83, wherein the flag bit is a time domain scalable flag bit.

85. The video encoding method of claim 84, wherein,

when there is a temporal scalability requirement, the value of the flag bit is used to indicate that an image block in an image that is not a short-term reference image is not available for updating the particular reference image; and/or the number of the groups of groups,

86. The method of video encoding according to claim 65, wherein the particular reference picture is a picture that is available for inter prediction.

87. The method of claim 86, wherein the particular reference picture is a long-term reference frame, or is a background frame, or is a golden frame.

88. The method of video encoding according to claim 65, wherein the method of determining that an image block is available for updating a particular reference image comprises:

89. The method for video encoding of claim 88, wherein: the determining that the image block is available for updating the particular reference image based on the image block and the encoded block includes:

90. A video decoding method, comprising:

updating a specific image block in a specific reference image according to an image block when a current image to be decoded is available for updating the specific reference image and updating the specific image block in the specific reference image according to the image block when the image block is available for updating the specific reference image, and not updating the specific image block in the specific reference image according to the image block when the current image to be decoded cannot be used as the reference image, wherein the image block is one image block in the current image to be decoded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be decoded;

the method further comprises the steps of:

91. The video decoding method of claim 90, wherein the method further comprises:

when a current image to be decoded is not available for updating a specific reference image, the specific image block in the specific reference image is not updated by the image block in the current image to be decoded.

92. The video decoding method of claim 90, wherein the method further comprises:

93. The method of claim 92, wherein the obtaining a parameter or flag indicating a reference relationship of the current image to be decoded comprises:

94. The video decoding method of claim 90, wherein the method further comprises:

95. The method of claim 90, wherein the number of image blocks in the current picture to be decoded that can be used to update the particular reference image is M, M being an integer not less than 1.

96. The method of claim 95, wherein the value of M for the current picture to be decoded is determined based on a reference relationship of the current picture to be decoded.

97. The method according to claim 96, wherein when the types of the current to-be-decoded pictures are different, the values of M corresponding to the current to-be-decoded pictures are different.

98. The method according to claim 96, wherein the value of M corresponding to the current picture to be decoded is determined based on a total number of pictures of a type to which the current picture to be decoded belongs.

99. The video decoding method of claim 95, wherein the number of image blocks in the current image to be decoded that can be used to update the particular reference image is included in at least one of:

100. The video decoding method of claim 91, further comprising:

when it is determined that the current to-be-decoded image is not available for inter prediction except for an image of the current to-be-decoded image, it is determined that the image block is not used to update the particular image block in the particular reference image.

101. The video decoding method of claim 91, further comprising:

when a current picture to be decoded may be a short-term reference picture but may not be a long-term reference picture, it is determined that a picture block in the picture to be decoded is not used to update the particular picture block in the particular reference picture.

102. The video decoding method of claim 90 or 91, wherein the method further comprises:

103. The video decoding method of claim 102, wherein the method further comprises:

104. The video decoding method of claim 90 or 91, wherein the method further comprises:

105. The video decoding method of claim 90 or 91, wherein the method further comprises:

106. The method of video decoding according to claim 93, wherein the reference relationship of the current picture to be decoded is: the current band decoded picture is a short-term reference picture or a long-term reference picture.

107. The video decoding method of claim 91, further comprising:

when a current picture to be decoded is unavailable as a short-term reference picture and is unavailable as a long-term reference picture, it is determined that a picture block in the picture to be decoded is unavailable for updating the particular picture block in the particular reference picture.

108. The method of video decoding according to any one of claims 90, 101, 106, 107,

obtaining a flag bit from at least one of the following, the flag bit being used to identify whether the image to be decoded is available for updating the specific reference image:

109. The method of claim 108, wherein the flag bit is a time-domain scalable flag bit.

110. The video decoding method of claim 108, wherein,

when the flag bit or the telescopic flag bit is a first numerical value, the value of the flag bit is used for indicating that an image block in an image which cannot be a short-term reference image cannot be used for updating the specific reference image; and/or the number of the groups of groups,

111. The method of video decoding according to claim 90, wherein the particular reference picture is a picture that is available for inter prediction.

112. The video decoding method of claim 111, wherein the particular reference picture is a long-term reference frame, or is a background frame, or is a golden frame.

113. A video encoding apparatus, comprising:

at least one memory for storing computer-executable instructions;

updating a specific image block in a specific reference image according to an image block when a current image to be encoded is available for updating the specific reference image and updating the specific image block in the specific reference image according to the image block when the image block is available for updating the specific reference image, and not updating the specific image block in the specific reference image according to the image block when the current image to be encoded cannot be used as the reference image, wherein the image block is one image block in the current image to be encoded, and the position of the specific image block in the specific reference image is determined by the position of the image block in the current image to be encoded;

The operations further comprise:

114. The video encoding device of claim 113, wherein the processor is further configured to:

115. The video encoding device of claim 113, wherein the processor is further configured to:

116. The video encoding device of claim 113, wherein the processor is further configured to:

117. The video encoding device of claim 113, wherein a number of image blocks in the current picture to be encoded that are available for updating the particular reference picture is not greater than M, M being an integer not less than 1.

118. The video encoding device of claim 117, wherein the value of M for the current picture to be encoded is determined based on a reference relationship for the current picture to be encoded.

119. The apparatus of claim 118, wherein when the types of the current to-be-encoded images are different, the values of M corresponding to the current to-be-encoded images are different.

120. The video encoding device of claim 118, wherein the value of M for the current picture to be encoded is determined based on a total number of pictures of a type to which the current picture to be encoded belongs.

121. The video encoding device of claim 117, wherein the image block comprises a preset number of color channel pixels,

122. The video encoding device of claim 113, wherein the processor is further configured to:

123. The video encoding device of claim 113, wherein the position of the particular picture block in the particular reference picture is determined by the position of the picture block in the current picture to be encoded, comprising:

124. The video encoding device of claim 114, wherein the processor is further configured to:

125. The video encoding device of claim 114, wherein the processor is further configured to:

126. The video encoding device of claim 114, wherein the processor is further configured to:

127. The video encoding device of claim 113 or 114, wherein the processor is further configured to:

128. The video encoding device of claim 127, wherein the processor is further configured to:

129. The video encoding device of claim 113 or 114, wherein the processor is further configured to:

130. The video encoding device of claim 113 or 114, wherein the processor is further configured to:

131. The video encoding device of claim 113 or 114, wherein the processor is further configured to:

132. The video coding device of any of claims 113, 125, 126, wherein a flag bit is added to at least one of the following to identify whether an image that is not available as a short-term reference image is available to update the particular reference image:

133. The video encoding device of claim 132, wherein the flag bit is a temporal scalable flag bit.

134. The video encoding device of claim 133, wherein,

135. The video encoding device of claim 113, wherein the particular reference picture is a picture that is available for inter-prediction.

136. The video encoding device of claim 135, wherein the particular reference picture is a long-term reference frame, or is a background frame, or is a golden frame.

137. The video encoding device of claim 113, wherein the means for determining that an image block is available for updating a particular reference image comprises:

138. The video encoding device of claim 137, wherein: the determining that the image block is available for updating the particular reference image based on the image block and the encoded block includes:

139. A video decoding apparatus, comprising:

at least one memory for storing computer-executable instructions;

The method further comprises the steps of:

140. The video decoding device of claim 139, wherein the processor is further configured to:

when a current image to be decoded is not available for updating a specific reference image, updating a specific image block in the specific reference image without the image block of the current image to be decoded.

141. The video decoding device of claim 139, wherein the processor is further configured to:

142. The video decoding device of claim 141, wherein the obtaining a parameter or flag indicating a reference relationship for the current image to be decoded comprises:

143. The video decoding device of claim 119, wherein the processor is further configured to:

144. The video decoding device of claim 139, wherein a number of image blocks in the current to-be-decoded image that are available for updating the particular reference image is M, M being an integer not less than 1.

145. The video decoding device of claim 139, wherein the value of M for the current picture to be decoded is determined based on a reference relationship for the current picture to be decoded.

146. The video decoding device of claim 145, wherein when the types of the current to-be-decoded pictures are different, the values of M corresponding to the current to-be-decoded pictures are different.

147. The video decoding device of claim 146, wherein the value of M for the current picture to be decoded is determined based on a total number of pictures of a type to which the current picture to be decoded belongs.

148. The video decoding device of claim 145, wherein the number of image blocks in the current image to be decoded that are available for updating the particular reference image is included in at least one of:

149. The video decoding device of claim 140, wherein the processor is further configured to:

When it is determined that the current image to be decoded is not available for inter prediction except for an image of the current image to be decoded, it is determined that the image block in the image to be decoded is not used for updating a specific image block in the specific reference image.

150. The video decoding device of claim 140, wherein the processor is further configured to:

151. The video decoding device of claim 139 or 140, wherein the processor is further configured to:

152. The video decoding device of claim 151, wherein the processor is further configured to:

153. The video decoding device of claim 139 or 140, wherein the processor is further configured to:

154. The video decoding device of claim 139 or 140, wherein the processor is further configured to:

155. The video decoding device of claim 141, wherein the reference relationship for the current picture to be decoded is: the current band decoded picture is a short-term reference picture or a long-term reference picture.

156. The video decoding device of claim 140, wherein the processor is further configured to:

157. The video decoding device of any of claims 139, 140, 155, 156, wherein the processor is further configured to:

158. The video decoding device of claim 157, wherein the flag bit is a temporal scalable flag bit.

159. The video decoding device of claim 157, wherein,

160. The video decoding device of claim 139, wherein the particular reference picture is a picture that is available for inter-prediction.

161. The video decoding device of claim 160, wherein the particular reference picture is a long-term reference frame, or is a background frame, or is a golden frame.

162. A machine-readable storage medium adapted for use in a terminal, said machine-readable storage medium having stored thereon computer instructions which, when executed, perform the video encoding method of any of claims 65-89.

163. A machine-readable storage medium adapted for use in a terminal, said machine-readable storage medium having stored thereon computer instructions which, when executed, perform the video decoding method of any of claims 90-112.