CN114339230B

CN114339230B - Transformation core selection method and device for video coding, storage medium and terminal

Info

Publication number: CN114339230B
Application number: CN202210200487.2A
Authority: CN
Inventors: 张鹏; 郝志坚; 向国庆; 范益波; 黄晓峰; 严伟
Original assignee: Advanced Institute of Information Technology AIIT of Peking University; Hangzhou Weiming Information Technology Co Ltd
Current assignee: Advanced Institute of Information Technology AIIT of Peking University; Hangzhou Weiming Information Technology Co Ltd
Priority date: 2022-03-03
Filing date: 2022-03-03
Publication date: 2022-09-02
Anticipated expiration: 2042-03-03
Also published as: CN114339230A

Abstract

The application relates to a transformation core selection method, a transformation core selection device, a storage medium and a terminal for video coding. Wherein the method comprises the following steps: acquiring input data of a residual block in a video image; acquiring a code rate formula of a transformation type of a residual block according to input data and a basis function of the transformation type in an encoder; weighting and summing the Laplace matrixes to obtain an updated matrix; calculating the code rate of the transform type after the residual block is transformed according to a code rate formula of the transform type, a plurality of Laplace matrixes, an update matrix and a Laplace quadratic formula of the residual block; and selecting the transform type with the minimum code rate as the transform type of the residual block in the video coding. According to the method and the device, the code rate of the transformation type of the transformed residual block can be calculated according to the input data of the residual block, a better transformation type is selected for the residual block in a self-adaptive manner, and the complex rate-distortion optimization RDO process is avoided; under the condition of smaller performance loss, the coding time and the coding complexity are reduced, and the coding efficiency is improved.

Description

Transformation core selection method and device for video coding, storage medium and terminal

Technical Field

The present invention relates to the field of video coding technologies, and in particular, to a transform kernel selection method and apparatus for video coding, a storage medium, and a terminal.

Background

Video coding refers to converting a file in a certain video format into a file in another video format by a specific compression technique.

The multiple transform types introduced in the field of video coding improve the coding performance of video coding, but the problem that the transform type selected during video coding does not have optimality and adaptivity also exists.

Based on the problems, the invention provides a method, a device, a storage medium and a terminal for selecting a transformation core for video coding, which can adaptively select a better transformation type for a residual block according to the characteristics of an input residual, and avoid a complex Rate Distortion Optimization (RDO) process; under the condition of smaller performance loss, the coding time and the coding complexity are reduced, and the coding efficiency is improved.

Disclosure of Invention

The embodiment of the application provides a transformation core selection method, a transformation core selection device, a storage medium and a terminal for video coding. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

In a first aspect, an embodiment of the present application provides a transform core selection method for video coding, where the method includes:

acquiring input data of a residual block in a video image;

acquiring a code rate formula of the transformation type of the residual block according to input data and a basis function of the transformation type in an encoder;

weighting and summing the Laplace matrixes to obtain an updated matrix;

calculating the code rate of the transform type after the residual block is transformed according to a code rate formula of the transform type, a plurality of Laplace matrixes, an update matrix and a Laplace quadratic formula of the residual block;

and selecting the transform type with the minimum code rate as the transform type of the residual block in the video coding.

Optionally, obtaining a code rate formula of a transform type of the residual block according to the input data and a basis function of the transform type in the encoder includes:

performing one-dimensional transformation on input data and a base function of a transformation type in an encoder to obtain a one-dimensional transformation result of a residual block;

and acquiring a code rate formula of the transformation type of the residual block according to the one-dimensional transformation result and the punishment.

Optionally, calculating the code rate of the transform type after the transform of the residual block according to a code rate formula of the transform type, a plurality of laplacian matrices, an update matrix, and a laplacian quadratic formula of the residual block, includes:

substituting the updated matrix into a Laplace quadratic formula of the residual block to obtain a quadratic formula of the updated matrix;

and obtaining a code rate formula of the transformation type of the residual block corresponding to the updated matrix according to the quadratic formula of the updated matrix and the code rate formula of the transformation type.

Optionally, the method further includes:

extracting the weight of the update matrix according to a code rate formula of the transformation type of the residual block corresponding to the update matrix;

and fitting and updating the matrix weight according to the punishment degree and the plurality of Laplace matrixes.

Optionally, the method further includes:

and selecting the fitted updated matrix weight in a line-down adjustment mode, and determining a plurality of residual weights corresponding to the Laplace matrixes.

Optionally, the method further includes:

and acquiring the code rate of the transformed type of the residual block according to the residual weights, the Laplace matrixes and the Laplace quadratic formula of the residual block.

Optionally, the transform types include: discrete cosine type II transform, discrete cosine type VIII transform and/or discrete sine type VII transform.

In a second aspect, an embodiment of the present application provides a transform core selection apparatus for video coding, including:

the residual data acquisition module is used for acquiring input data of a residual block in a video image;

a code rate formula obtaining module, configured to obtain a code rate formula of a transform type of the residual block according to input data and a basis function of the transform type in the encoder;

the updating matrix obtaining module is used for carrying out weighted summation on the plurality of Laplacian matrixes to obtain an updating matrix;

the code rate calculation module is used for calculating the code rate of the transformed type of the residual block after transformation according to a code rate formula of the transformed type, a plurality of Laplace matrixes, an updated matrix and a Laplace quadratic formula of the residual block;

and the transformation type selection module is used for selecting the transformation type with the minimum code rate as the transformation type of the residual block in the video coding.

In a third aspect, embodiments of the present application provide a computer storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor and to perform the above-mentioned method steps.

In a fourth aspect, an embodiment of the present application provides a terminal, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the above-mentioned method steps.

The technical scheme provided by the embodiment of the application can have the following beneficial effects:

in the embodiment of the application, the transform kernel selection method for video coding first obtains input data of a residual block in a video image; then, according to the input data and the base function of the transformation type in the encoder, a code rate formula of the transformation type of the residual block is obtained; then, carrying out weighted summation on the plurality of Laplace matrixes to obtain an updated matrix; secondly, calculating the code rate of the transformed type of the residual block after transformation according to a code rate formula of the transformed type, a plurality of Laplace matrixes, an updating matrix and a Laplace quadratic formula of the residual block; and finally, selecting the transform type with the minimum code rate as the transform type of the residual block in the video coding. According to the method and the device, the code rate of the transform type of the transformed residual block can be calculated according to the input data of the residual block, the transform type with the minimum code rate is selected as the transform type of the residual block in the video coding, a better transform type can be selected for the residual block in a self-adaptive manner, and the complex rate-distortion optimization (RDO) process is avoided; under the condition of smaller performance loss, the coding time and the coding complexity are reduced, and the coding efficiency is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a flowchart illustrating a transform kernel selection method for video coding according to an embodiment of the present application;

fig. 2 is a laplacian matrix of a transform kernel selection method for video coding according to an embodiment of the present application

~

A matrix map of (a);

fig. 3 is a laplacian matrix of a transform kernel selection method for video coding according to an embodiment of the present application

~

A matrix map of (c);

fig. 4 is a flowchart illustrating another transform core selection method for video encoding according to an embodiment of the present application;

fig. 5 is a schematic diagram of an apparatus for selecting a transform core for video coding according to an embodiment of the present application;

fig. 6 is a schematic diagram of a terminal according to an embodiment of the present application.

Detailed Description

The following description and the drawings sufficiently illustrate specific embodiments of the invention to enable those skilled in the art to practice them.

It should be understood that the described embodiments are only some embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of systems and methods consistent with certain aspects of the invention, as detailed in the appended claims.

In the description of the present invention, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The specific meanings of the above terms in the present invention can be understood in a specific case to those of ordinary skill in the art. In addition, in the description of the present invention, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, a and/or B, which may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

Transform coding is an important component in video coding and is also a major tool for improving video compression rate. The type of coding widely used in previous generation encoders is the discrete cosine II transform (DCT-II), such as HEVC and AVS 2; new transform types are introduced in the new generation of video coding protocols, including but not limited to discrete cosine type VIII (DCT-VIII) and discrete sine type VII (DST-VII) transforms, which can have higher energy concentration efficiency for residual blocks with certain characteristics, and which can provide higher compression rates than when discrete cosine type II (DCT-II) transforms are used.

In the first generation Open Media Video Alliance (Alliance for Open Media Video 1, AV 1), four basic transform types were used: DCT, ADST, flip-ADST and IDTX; three transform types used in Versatile Video Coding (VVC) are: DCT-II, DCT-VIII and DST-VII; similarly, three transform types used in the third generation Audio Video coding Standard 3 (AVS 3) are: DCT-II, DCT-VIII and DST-VII; the introduction of these transform types can provide the encoder with a 2% to 3% coding performance gain. The introduction of multiple transform types also introduces computational complexity while improving coding performance. Compared with the previous generation of encoders which only use the DCT-II feature, the new generation of encoders need to choose among multiple transform types. In order to reduce the complexity of calculation, the two-dimensional transform is generally split into two one-dimensional transforms (i.e., a row transform and a column transform) to perform the transform in the encoder, and each one-dimensional transform may select one of a plurality of transform types as a transform type used in actual encoding. For the first generation open media video alliance AV1, each residual block needs to select one transform type from 4x4= 16. For both multipurpose video coding VVC and third generation audio video coding standard AVS3, each residual block needs to select one transform type from 3x3= 9. In a conventional encoder, a transform type is generally selected by using a Rate-Distortion Optimization (RDO) method, and the transform type selected by using the RDO method has high complexity and large computation delay, and is not suitable for a scene requiring instantaneous encoding and decoding.

In order to solve the problem of high complexity in the transformation core selection process, different coding protocols provide different fast algorithms. Under the current situation, in the first generation open media video alliance AV1, a neural network may be used to select a transformation type, and when the neural network is used to select the transformation type, relevant information may be extracted from a residual block, the relevant information is input into the trained neural network, different transformation types can be scored, part of the transformation types are screened, and the remaining transformation types are input into a process of rate-distortion optimization RDO, so as to select an optimal transformation type; in the intra-frame coding of the multi-purpose video coding VVC, the relationship between different angle modes and transformation types can be obtained through statistics, and a plurality of fixed transformation types are preset for each angle mode, so that the rate distortion optimization RDO of all the transformation types is avoided, and the circuit complexity is reduced to a certain extent. Although the number of the transformation types in the process of entering the Rate Distortion Optimization (RDO) can be reduced to a certain extent by the two methods, the obvious disadvantages still exist, although the neural networks used in the first generation open media video alliance AV1 are not large, because the parameters in the neural networks are all floating point numbers, a large amount of resources are consumed in hardware design to ensure the precision of the floating point numbers and the accuracy of the neural networks, and more resource expenses exist in hardware; in the multi-purpose video coding VVC, although the complexity of the calculation is controlled, since only rate-distortion optimized RDO traversal is performed for several fixed transform types for each angle mode, it cannot be guaranteed that the selected transform type has optimality and adaptivity, and the performance of the encoder is reduced.

The invention provides a method, a device, a storage medium and a terminal for selecting a transformation core for video coding, which can quickly predict the code rate of a residual block after all transformation types by realizing a specific algorithm, and adaptively select a better transformation type for the residual block according to the characteristic of an input residual by using a code rate priority criterion, wherein the method is applied to a coding protocol with a multi-core transformation function (such as multi-purpose video coding (VVC), a first-generation open media video Alliance (AV) 1, a third-generation audio and video coding standard (AVS 3) to avoid a complex RDO (rate distortion optimization) process and reduce computing resources; under the condition of smaller performance loss, the coding time and the coding complexity are reduced, and the coding efficiency is improved.

A transform kernel selection method for video coding according to an embodiment of the present application will be described in detail below with reference to fig. 1 to 4.

Referring to fig. 1-3, a flow chart of a transform core selection method for video coding is provided according to an embodiment of the present application. As shown in fig. 1-3, the method of the embodiments of the present application may include the steps of:

s110, input data of the residual block in the video image is obtained.

In the embodiment of the application, the input data of the residual block in the video image is acquired and analyzed, so that a better transformation type is adaptively selected for the residual block.

And S120, acquiring a code rate formula of the transformation type of the residual block according to the input data and the basis function of the transformation type in the encoder. Specifically, the 120 includes:

and S121, performing one-dimensional transformation on the input data and the base function of the transformation type in the encoder to obtain a one-dimensional transformation result of the residual block. In the embodiment of the present application, the one-dimensional transformation result of the residual block obtained by combining the input data of the residual block with the basis function of the transformation type is expressed by a one-dimensional transformation formula as follows:

wherein i represents a subscript; t represents the transpose of the matrix; when in use

Basis functions representing discrete cosine type II transforms

When the above one-dimensional transformation formula represents a one-dimensional transformation formula in which the transformation type is discrete cosine II type, the above one-dimensional transformation formula represents a one-dimensional transformation formula in which the transformation type is discrete cosine II type

Basis functions representing discrete sinusoidal type VII transforms

When the above one-dimensional transformation formula represents a one-dimensional transformation formula in which the transformation type is discrete sine VII type transformation, when

When the basis function of the discrete cosine VIII type conversion is expressed, the one-dimensional conversion formula expresses that the conversion type is the one-dimensional conversion formula of the discrete cosine VIII type conversion; x is the input data and x is the output data,

representing the transform coefficient after the one-dimensional transform of the ith row; taking a 4-point transform as an example, when x represents a column vector consisting of 4 input data

When the temperature of the water is higher than the set temperature,

is shown as follows.

In the embodiment of the present application, it is necessary to obtain a basis function of a transform type in an encoder. When the transform type is a discrete cosine type II transform, the basis functions of the discrete cosine type II transform are:

when the transform type is a discrete sine type VII transform, the basis functions of the discrete sine type VII transform are:

in the basis functions of the discrete cosine type II transform and the discrete sine type VII transform, j represents the jth frequency component, and k represents the kth item in the jth frequency component; n is equivalent to N of the following, each of which indicates a transform size, and indicates 4-point transform when N =4 and indicates 8-point transform when N = 8.

In the integer transform used in the encoder, the transform matrix of the discrete cosine VIII type transform and the transform matrix of the discrete sine VII type transform have a certain duality, the transform matrix of the discrete cosine VIII type transform can be directly obtained from the transform matrix of the discrete sine VII type transform, the basis function of the discrete cosine VIII type transform can be directly obtained from the basis function of the discrete sine VII type transform, and details of the basis function of the discrete cosine VIII type transform are not described herein.

In different coding standards, transformation matrices of the same transformation type may not be the same, and the difference between the base functions of the transformation types between different coding standards is the difference of scaling factors, and the base functions of the transformation types between different coding standards show a multiple relationship.

And S122, acquiring a code rate formula of the transformation type of the residual block according to the one-dimensional transformation result and the punishment. In the embodiment of the present application, the code rate formula of the transform type of the predicted residual block is as follows:

wherein C represents a code rate;

to represent

The corresponding weight is also called penalty degree and penalty weight.

When in use

At 1, C represents the sum of the squares of the transform coefficients, i.e., the energy of the input data. In the embodiment of the application, the method can be realized by setting

Is added with a penalty for the energy of a particular frequency. In will

When the high-frequency coefficient is set from small to large, the high-frequency coefficient is not important, the punishment of the high-frequency coefficient is increased, and if the position i is larger,

if the value is larger, the obtained code rate C is larger, and the transformation type is less prone to being selected; in will

When setting from large to small, if i is at a small position,

with larger values, the larger the code rate C, the less easily the transform type can be selected. In the embodiment of the present application, the significance of the transform of the residual block according to the transform type is energy concentration, that is, the energy of the input residual block can be concentrated in the upper left corner after the transform of the transform type, that is, at the position where i is smaller, and the embodiments of the present application can be arranged in the order from small to large

The value of (c).

In the embodiment of the present application, the multiple laplacian matrices may be used to obtain

By passing

And calculating the code rate of the residual block after different transformation types are transformed.

And S130, carrying out weighted summation on the plurality of Laplace matrixes to obtain an updated matrix. In the embodiment of the present application, for a laplacian matrix L, the eigenvalue and eigenvector of the laplacian matrix L are assumed to be

And

then, according to the laplacian quadratic formula of the residual block, the obtained expansion of the laplacian quadratic formula of the residual block is:

the right side of the equation has the same code rate

In a very similar fashion. Since L is a Laplace matrix, so

And

are identical, however

And with

Are not necessarily identical or similar, provided that

And

identical or similar, then Laplace quadratic form of the residual block

Is the same or similar to code rate C.

In the embodiment of the present application, when the transform type is a discrete cosine II transform, 8 laplacian matrices that are very similar to the laplacian matrix L may be used

~

And carrying out weighted summation to obtain an updated matrix F. Laplace matrix

~

The case when the transform length is 8 is shown in fig. 2, with the first row from left to right being a laplacian matrix

To

The second row is a Laplace matrix from left to right

To

The laplacian matrix of

~

Respectively expressed as:

the Laplace matrix

~

The conditions to be satisfied are: laplace matrix

~

Feature vector of

A basis function that is a transform type; laplace matrix

~

Should be sparse in nature; laplace matrix satisfying these conditions

~

So that the embodiment of the application can pass through the Laplace matrix with very small calculation cost

~

Acquired update matrix

And obtaining an approximate value of the code rate C by adopting a linear combination mode.

The laplacian matrices given for the same transform type have the same feature vector and the feature vector is the same as the basis function for that transform type. When the transform type is discrete cosine type II transform, the Laplace matrix

~

And expression of feature vectorsThe formula can be:

where e denotes the e-th laplace matrix.

In the embodiment of the present application, the formula of Laplace quadratic form for making the residual block

The same or similar to the code rate C, the penalty degree can be realized by adopting a linear fitting mode

And a characteristic value

And

identical or similar, by 8 laplacian matrices

~

8 eigenvalues of the updated matrix

Penalty strength of linear fitting

，

And representing the eigenvalue corresponding to the e-th laplace matrix.

In the embodiment of the present application, when the transform types are discrete cosine type VIII transform and discrete sine type VII transform, weighted summation is performed by a plurality of laplacian matrices,the manner of obtaining the update matrix and calculating the eigenvalue of the update matrix is similar to the manner of obtaining the update matrix and the eigenvalue of the update matrix when the transform type is the discrete cosine II transform, and is not described herein again. For example, when the transform type is discrete sine type VII transform, 7 Laplace matrices may be selected

~

And carrying out weighted summation to obtain an updated matrix of discrete sine V-II type transformation. Laplace matrix

~

The case when the transform length is 8 is shown in FIG. 3, with the first row from left to right being a Laplacian matrix

To

The second row is a Laplace matrix from left to right

To

The laplacian matrix of

~

Expressed as:

laplace matrix

~

The table is a sparse matrix; laplace matrix

~

The expression for the eigenvalues and eigenvectors is:

。

and S140, calculating the code rate of the transform type after the transform of the residual block according to the code rate formula of the transform type, the plurality of Laplace matrixes, the update matrix and the Laplace quadratic formula of the residual block. Specifically, S140 includes:

and S141, substituting the updated matrix into the Laplace quadratic formula of the residual block to obtain the quadratic formula of the updated matrix. In the embodiment of the present application, the Laplace quadratic form of the residual block is represented by

Updating the matrix

Then, the quadratic form formula of the updated matrix is:

。

and S142, acquiring a code rate formula of the transformation type of the residual block corresponding to the updated matrix according to the quadratic formula of the updated matrix and the code rate formula of the transformation type. In the embodiment of the present application, the quadratic form formula of the update matrix F is

The updating matrix F is formed by 8 Laplace matrixes

~

And (4) obtaining. Laplace quadratic form formula in residual block

Code rate formula of sum residual block

On the premise of equality, the quadratic formula of the update matrix is equal to the code rate formula of the transform type of the residual block corresponding to the update matrixAnd expanding a quadratic formula of the update matrix F to obtain the eigenvalue of the update matrix F (the eigenvalue of the update matrix F can be obtained by 8 Laplace matrixes

~

Obtaining update matrix eigenvalue)

Will be

Substituting the code rate formula

，

=

To obtain a code rate formula of the transformation type of the residual block corresponding to the update matrix

. At this time, the process of the present invention,

=

。

and S143, extracting the weight of the updated matrix according to the code rate formula of the transformation type of the residual block corresponding to the updated matrix. In the embodiment of the application, the code rate formula is based on the transformation type of the residual block corresponding to the updated matrix

Extracting updated matrix weights

Can be adjusted by

To fit the updated matrix weights

The value of (c).

And S144, fitting and updating the matrix weight according to the punishment degree and the Laplace matrixes. Since the more basic laplacian matrices are used in the process of performing the linear fitting, the higher the computational complexity. Therefore, in the embodiment of the present application, when the matrix weight is updated by the laplacian matrix fitting, the number of laplacian matrices used may be set to not more than 3.

In the embodiment of the application, when the transformation type is discrete cosine transformation and the transformation size is 4-point transformation, the punishment degree for improving the high-frequency coefficient

The punishment can be sequentially added

Setting as 1, 2, 4, 8 according to punishment degree

Different values of the settings are combined with the Laplace matrix

And

fitting update matrix weights

。

S145, selecting the fitted updated matrix weight through a line-down adjustment mode, and determining a plurality of Laplace matrices corresponding to the Laplace matricesThe residual weights. In this embodiment of the present application, the update matrix weight after fitting is selected by a way of adjusting under a line may be: using two Laplace matrices

And

l2 norm to represent the two laplacian matrices

And

the degree of similarity of (c); updating matrix weights by traversing within the (-100,100) interval with step size 0.5 in matlab

Selecting a group of weights corresponding to the minimum L2 norm as a Laplace matrix

And

corresponding 2 residual weights.

In the embodiment of the present application, the information is obtained by a way of offline adjustment

The residual weight of (a) may be 4.5,

the residual weight of (d) may be 4.

And S146, obtaining the code rate of the transform type after the transform of the residual block according to the residual weights, the Laplace matrixes and the Laplace quadratic formula of the residual block. In the embodiment of the application, the matrix is based on Laplace matrix

And

laplace matrix

Residual weight of 4.5, laplacian matrix

4, and the laplacian quadratic form of the residual block

Calculating the code rate of the transformation type after the residual block transformation:

。

in the embodiment of the present application, the transformation types include: discrete cosine type II transform, discrete cosine type VIII transform and/or discrete sine type VII transform. When calculating the code rate of the residual block after the transform type transform, the code rate of the residual block after the discrete cosine II type transform, the code rate of the residual block after the discrete cosine VIII type transform, and the calculation method similar to the calculation method of the code rate of the residual block after the discrete sine VII type transform can be calculated by the above method, and the detailed part in the process of calculating the code rate of the residual block after the transform of a certain transform type is not described herein again.

S150, selecting the transformation type with the minimum code rate as the transformation type of the residual block in the video coding. In the embodiment of the application, the size of the code rate of the residual block after discrete cosine II type conversion, discrete cosine VIII type conversion and discrete sine VII type conversion is compared, and the conversion type with the minimum code rate is selected as the conversion type of the residual block in the video coding.

In the embodiment of the present application, the transform kernel selection method for video coding first obtains input data of a residual block in a video image; then according to the input data and the base function of the transformation type in the encoder, obtaining a code rate formula of the transformation type of the residual block; then, carrying out weighted summation on the plurality of Laplacian matrixes to obtain an updated matrix; secondly, calculating the code rate of the transformed type of the residual block after transformation according to a code rate formula of the transformed type, a plurality of Laplacian matrixes, an updated matrix and a Laplacian quadratic formula of the residual block; and finally, selecting the transform type with the minimum code rate as the transform type of the residual block in the video coding. According to the method and the device, the code rate of the transform type of the transformed residual block can be calculated according to the input data of the residual block, the transform type with the minimum code rate is selected as the transform type of the residual block in the video coding, a better transform type can be selected for the residual block in a self-adaptive manner, and the complex rate-distortion optimization (RDO) process is avoided; under the condition of smaller performance loss, the coding time and the coding complexity are reduced, and the coding efficiency is improved.

Fig. 4 is a flowchart illustrating a transform kernel selection method for video coding according to an embodiment of the present application. As shown in fig. 4, the method of the embodiment of the present application may include the following steps:

s210, acquiring input data of a residual block in a video image;

s211, carrying out one-dimensional transformation on the input data and the base function of the transformation type in the encoder to obtain a one-dimensional transformation result of the residual block; the transform types include: discrete cosine type II transform, discrete cosine type VIII transform and/or discrete sine type VII transform;

s212, acquiring a code rate formula of the transformation type of the residual block according to the one-dimensional transformation result and the punishment;

s213, carrying out weighted summation on the Laplace matrixes to obtain an updated matrix;

s214, substituting the updated matrix into the Laplace quadratic formula of the residual block to obtain a quadratic formula of the updated matrix;

s215, obtaining a code rate formula of the transformation type of the residual block corresponding to the updated matrix according to the quadratic formula of the updated matrix and the code rate formula of the transformation type;

s216, extracting the weight of the updated matrix according to a code rate formula of the transformation type of the residual block corresponding to the updated matrix;

s217, fitting and updating the matrix weight according to the punishment degree and the Laplace matrixes;

s218, selecting the fitted updated matrix weight in a offline adjustment mode, and determining a plurality of residual weights corresponding to a plurality of Laplace matrices;

s219, obtaining the code rate of the transform type of the residual block after transform according to the residual weights, the Laplace matrixes and the Laplace quadratic formula of the residual block;

s220, selecting the transformation type with the minimum code rate as the transformation type of the residual block in the video coding.

The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention. For details which are not disclosed in the embodiments of the apparatus of the present invention, reference is made to the embodiments of the method of the present invention.

Referring to fig. 5, a schematic structural diagram of a transform core selection apparatus for video coding according to an exemplary embodiment of the present invention is shown. The device 1 comprises: the device comprises a residual data acquisition module 10, a code rate formula acquisition module 20, an update matrix acquisition module 30, a code rate calculation module 40 and a transformation type selection module 50.

A residual data obtaining module 10, configured to obtain input data of a residual block in a video image;

a code rate formula obtaining module 20, configured to obtain a code rate formula of a transform type of the residual block according to the input data and a basis function of the transform type in the encoder;

an update matrix obtaining module 30, configured to perform weighted summation on the multiple laplacian matrices to obtain an update matrix;

a code rate calculation module 40, configured to calculate a code rate of the transform type after the residual block transform according to a code rate formula of the transform type, the multiple laplacian matrices, the update matrix, and a laplacian quadratic formula of the residual block;

and a transform type selection module 50, configured to select a transform type with a smallest bitrate as a transform type of a residual block in video coding.

It should be noted that, when the transform core selection apparatus for video coding provided in the foregoing embodiment executes the transform core selection method for video coding, the division of the above functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the transform core selection apparatus for video coding and the transform core selection method for video coding provided in the foregoing embodiments belong to the same concept, and details of implementation processes thereof are referred to in the method embodiments and are not described herein again.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

In the embodiment of the present application, the transform kernel selection apparatus for video coding first obtains input data of a residual block in a video image; then, according to the input data and the base function of the transformation type in the encoder, a code rate formula of the transformation type of the residual block is obtained; then, carrying out weighted summation on the plurality of Laplace matrixes to obtain an updated matrix; secondly, calculating the code rate of the transformed type of the residual block after transformation according to a code rate formula of the transformed type, a plurality of Laplace matrixes, an updating matrix and a Laplace quadratic formula of the residual block; and finally, selecting the transform type with the minimum code rate as the transform type of the residual block in the video coding. According to the method and the device, the code rate of the transform type of the transformed residual block can be calculated according to the input data of the residual block, the transform type with the minimum code rate is selected as the transform type of the residual block in the video coding, a better transform type can be selected for the residual block in a self-adaptive mode, and the complex Rate Distortion Optimization (RDO) process is avoided; under the condition of smaller performance loss, the coding time and the coding complexity are reduced, and the coding efficiency is improved.

The present invention also provides a computer readable medium, on which program instructions are stored, which program instructions, when executed by a processor, implement the transform core selection method for video coding provided by the above-mentioned various method embodiments.

The present invention also provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the transform core selection method for video coding of the various method embodiments described above.

Please refer to fig. 6, which provides a schematic structural diagram of a terminal according to an embodiment of the present application. As shown in fig. 6, terminal 1000 can include: at least one processor 1001, at least one network interface 1004, a user interface 1003, memory 1005, at least one communication bus 1002.

Wherein a communication bus 1002 is used to enable connective communication between these components.

The user interface 1003 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1003 may further include a standard wired interface and a wireless interface.

The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.

Processor 1001 may include one or more processing cores, among other things. The processor 1001 interfaces various components throughout the electronic device 1000 using various interfaces and lines to perform various functions of the electronic device 1000 and to process data by executing or performing instructions, programs, code sets, or instruction sets stored in the memory 1005 and invoking data stored in the memory 1005. Alternatively, the processor 1001 may be implemented in at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 1001, but may be implemented by a single chip.

The Memory 1005 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable medium. The memory 1005 may be used to store an instruction, a program, code, a set of codes, or a set of instructions. The memory 1005 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 6, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a usability analysis application program of vehicle running trajectory data.

In the terminal 1000 shown in fig. 6, the user interface 1003 is mainly used as an interface for providing input for a user, and acquiring data input by the user; and the processor 1001 may be configured to invoke a transform core selection application for video coding stored in the memory 1005, and specifically perform the following operations:

acquiring input data of a residual block in a video image;

acquiring a code rate formula of a transformation type of a residual block according to input data and a basis function of the transformation type in an encoder; wherein the transform type includes: discrete cosine type II transform, discrete cosine type VIII transform and/or discrete sine type VII transform;

weighting and summing the Laplace matrixes to obtain an updated matrix;

In an embodiment, when the processor 1001 obtains a code rate formula of a transform type of a residual block according to input data and a basis function of the transform type in an encoder, the following operations are specifically performed:

In one embodiment, the processor 1001 specifically performs the following operations when executing the calculation of the code rate of the transform type after the transform block transform according to the code rate formula of the transform type, the plurality of laplacian matrices, the update matrix, and the laplacian quadratic formula of the residual block:

acquiring a code rate formula of a transformation type of a residual block corresponding to the update matrix according to a quadratic formula of the update matrix and the code rate formula of the transformation type;

fitting and updating the matrix weight according to the punishment degree and the Laplace matrixes;

selecting the fitted updated matrix weight in a offline adjustment mode, and determining a plurality of residual weights corresponding to a plurality of Laplace matrices;

In the embodiment of the application, the transform kernel selection method for video coding first obtains input data of a residual block in a video image; then according to the input data and the base function of the transformation type in the encoder, obtaining a code rate formula of the transformation type of the residual block; then, carrying out weighted summation on the plurality of Laplace matrixes to obtain an updated matrix; secondly, calculating the code rate of the transformed type of the residual block after transformation according to a code rate formula of the transformed type, a plurality of Laplacian matrixes, an updated matrix and a Laplacian quadratic formula of the residual block; and finally, selecting the transform type with the minimum code rate as the transform type of the residual block in the video coding. According to the method and the device, the code rate of the transform type of the transformed residual block can be calculated according to the input data of the residual block, the transform type with the minimum code rate is selected as the transform type of the residual block in the video coding, a better transform type can be selected for the residual block in a self-adaptive mode, and the complex Rate Distortion Optimization (RDO) process is avoided; under the condition of smaller performance loss, the coding time and the coding complexity are reduced, and the coding efficiency is improved.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can be executed by hardware related to the instructions of the computer program. The storage medium may be a magnetic disk, an optical disk, a read-only memory or a random access memory.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims

1. A method of transform kernel selection for video coding, comprising the steps of:

acquiring input data of a residual block in a video image;

acquiring a code rate formula of the transformation type of the residual block according to the input data and a basis function of the transformation type in an encoder;

weighting and summing a plurality of Laplace matrixes to obtain an updated matrix, wherein the feature vectors of the Laplace matrixes are basis functions of a transform type, and the Laplace matrixes are sparse;

calculating the code rate of the transformed type of the residual block after transformation according to a code rate formula of the transformed type, the plurality of Laplace matrixes, the update matrix and a Laplace quadratic formula of the residual block;

selecting the transform type with the minimum code rate as the transform type of the residual block in video coding;

the obtaining a code rate formula of the transform type of the residual block according to the input data and a basis function of the transform type in an encoder comprises:

performing one-dimensional transformation on the input data and the base function of the transformation type in the encoder to obtain a one-dimensional transformation result of the residual block;

acquiring a code rate formula of the transformation type of the residual block according to the one-dimensional transformation result and the punishment degree;

the calculating a code rate of the transform type after the transform of the residual block according to a code rate formula of the transform type, the plurality of laplacian matrices, the update matrix, and a laplacian quadratic formula of the residual block includes:

because the quadratic formula of the update matrix and the code rate formula of the transform type of the residual block are equal, the quadratic formula of the update matrix is expanded to obtain the eigenvalue of the update matrix, and the eigenvalue is substituted into the code rate formula to obtain the code rate formula of the transform type of the residual block corresponding to the update matrix;

fitting the updated matrix weights according to the punishment degree and the Laplace matrixes;

selecting the fitted updated matrix weights in a way of offline adjustment, and determining a plurality of residual weights corresponding to the plurality of Laplacian matrices;

and acquiring the code rate of the transformed type of the residual block after transformation according to the residual weights, the Laplace matrixes and a Laplace quadratic formula of the residual block.

2. The method of claim 1, wherein the transform type comprises: discrete cosine type II transform, discrete cosine type VIII transform and/or discrete sine type VII transform.

3. A transform kernel selection apparatus for video coding, comprising:

a code rate formula obtaining module, configured to obtain a code rate formula of a transform type of the residual block according to the input data and a basis function of the transform type in an encoder, where the code rate formula obtaining module is configured to:

acquiring a code rate formula of the transformation type of the residual block according to the one-dimensional transformation result and the punishment;

the updating matrix obtaining module is used for performing weighted summation on a plurality of Laplace matrixes to obtain an updating matrix, wherein the feature vectors of the Laplace matrixes are base functions of a transformation type, and the Laplace matrixes are sparse;

a code rate calculation module, configured to calculate a code rate of the transform type after the residual block is transformed according to a code rate formula of the transform type, the multiple laplacian matrices, the update matrix, and a laplacian quadratic formula of the residual block, including:

selecting the fitted updated matrix weight in a offline adjustment mode, and determining a plurality of residual weights corresponding to the Laplace matrices;

obtaining the code rate of the transformed type of the residual block after transformation according to the residual weights, the Laplace matrixes and a Laplace quadratic formula of the residual block;

4. A computer storage medium, characterized in that it stores a plurality of instructions adapted to be loaded by a processor and to perform the method steps according to any of claims 1-2.

5. A terminal, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of any of claims 1-2.