CN116266865A - Hardware acceleration method for motion compensation operation in merge mode of H265 format - Google Patents

Abstract

The invention provides a hardware acceleration method of motion compensation operation under a merge mode of an H265 format, which comprises the steps of judging whether current mmvx is the same as nmv, wherein a motion vector mv obtained through the merge coding mode is mmv, mmvx is an x mmv, and a motion vector mv obtained through motion estimation me is nmv; if the current mmvx is consistent with the nmv value, not performing mc calculation, but copying nmv the pred pixel result as the pred pixel of the current mmvx, so as to omit the mc operation of the current mmvx; if the current mmvx and nmv values are inconsistent, the mmvx reference frame pixels need to be read, and mc operation is performed to obtain pred pixels. The method utilizes the correlation among mv, and reduces the mc calculation of mv in the merge mode under the condition of not affecting the coding quality. The method not only can reduce the operand, but also can reduce the access times to ddr, thereby achieving the function of hardware acceleration.

Description

Hardware acceleration method for motion compensation operation in merge mode of H265 format

Technical Field

The invention relates to the technical field of hardware acceleration, in particular to a hardware acceleration method for motion compensation operation in a merge mode of an H265 format.

Background

With the wide application of high-definition and ultra-high-definition videos (the resolution reaches 4Kx2K and 8Kx 4K), the original video coding standard cannot meet the current video application requirements, and a new generation video coding and decoding standard H265 is gradually applied. The H265 format still employs a hybrid coding framework including transform, quantization, entropy coding, intra prediction, inter prediction, and loop filtering. But the H265 format introduces new coding techniques in each link, including techniques of block division based on quadtree, supporting more intra-frame prediction modes, more advanced inter-frame prediction techniques (merge, AMVP), deblocking filtering, sample adaptive compensation, etc. in the post-processing stage. The inter-frame prediction technology is a key coding technology in H265, and the module uses an image coded by an adjacent frame as a reference image of a current frame, predicts a current block to be coded by acquiring motion information of a motion block, can remove a large amount of redundant information in a time domain, and can obtain higher compression efficiency compared with other links. In the H265 standard, inter prediction adopts nme mode and merge mode, where nme mode predicts mv through motion estimation process, merge mode can use neighboring coded PU blocks to predict mv.

Briefly, when H265 is encoded, mv needs to be mc to obtain pred pixels. There are two types of mv for inter prediction, one is nmv by me and one is mmv by merge technique. And performing mc operation on mv to obtain pred value corresponding to mv. mmv the number maxNumMergeCand can be configured in the range of 1 to 5.

The basic flow of the H265 inter prediction nme mode is as follows:

1. nmv by motion estimation;

2. and carrying out mc on nmv to obtain pred pixels.

The basic flow of the H265 inter prediction merge mode is as follows:

1. obtaining a plurality of candidates mmv from adjacent PUs through a merge technology, wherein the number maxNumMergeCand of mmv can be configured in a range of 1-5;

2. each mmv is given an mc in turn to get pred pixels.

The flow of mc operation on 1 piece nmv and 5 pieces mmv in hardware design, as shown in fig. 1, is as follows:

1. reading nmv, namely reading ref data from ddr, performing motion compensation, and storing a nmv motion compensation result into ram;

2. reading mmv, reading ref data from ddr for motion compensation, and storing mmv0 motion compensation result to ram;

3. reading mmv, namely reading ref data from ddr to perform motion compensation, and storing a motion compensation result of mmv1 into ram;

4. reading mmv, namely reading ref data from ddr to perform motion compensation, and storing a motion compensation result of mmv2 into ram;

5. reading mmv, namely reading ref data from ddr to perform motion compensation, and storing a motion compensation result of mmv to ram;

6. reading mmv, namely reading ref data from ddr to perform motion compensation, and storing a motion compensation result of mmv to ram;

wherein the hardware performs the flow pseudocode of the mc operation on 1 nmv and 5 mmv as shown in fig. 2.

However, the drawbacks of the prior art include:

performing mc operations on one mv requires reading its corresponding ref data from ddr, which affects the bandwidth of ddr. The greater the number of accesses to ddr ref data, the greater the bandwidth impact on ddr. Since the merge technique has at most 5 mv candidates and at most 1 mv for me, one PU needs to perform mc on at most 6 mvs, which affects the bandwidth of ddr and requires a sufficient amount of computation to perform mc computation.

The following are terms of the prior art, including:

h265: a video encoding format;

mv: abbreviations for motion vectors; an offset in the reference frame relative to the current frame;

me: motion estimation abbreviation for motion estimation; the object motion between two frames is translational motion, and the displacement is not very large, so that motion vectors are distributed by taking blocks as units, a large number of reference frame predictions are adopted in motion estimation to improve precision, and the current block to be coded can search the optimal matching block in all reference frames in a buffer memory for motion compensation, so that the redundancy of a time domain can be removed well. A motion vector MV is sought for each block and motion compensated predictive coding is performed. Each partition area is provided with a corresponding motion vector, and the motion vector and the selection mode of the block are encoded and transmitted;

mc: motion compensation abbreviation for motion compensation; motion compensation is an effective method for predicting and compensating a current local image by referring to the local image in a frame, and reducing redundant information of a frame sequence; motion compensation is a method of describing the difference between adjacent frames (adjacent here means adjacent in coding relation, not necessarily adjacent in playing order), specifically how each best matching block of the previous frame (adjacent here means previous in coding relation, not necessarily previous in playing order to the current frame) moves to the position of the block to be coded in the current frame;

merge: merge can be seen as a coding mode in which the mv of the current PU is directly predicted from spatially or temporally adjacent PUs;

pred pixels: pixels obtained by mc of the reference frame pixels;

nme: nme can be seen as a coding mode in which the mv of the current PU is predicted by motion estimation;

AMVP: advanced Motion Vector Predictor abbreviations;

PU: abbreviation of prediction unit.

Disclosure of Invention

In order to solve the above-mentioned problems in the prior art, an object of the present application is to: by using the correlation between mv, the mc calculation of mv in the merge mode is reduced without affecting the coding quality. The method not only can reduce the operand, but also can reduce the access times to ddr, thereby achieving the function of hardware acceleration.

Specifically, the invention provides a hardware acceleration method of motion compensation operation in a merge mode of an H265 format, which comprises the steps of judging whether current mmvx is the same as nmv, wherein a motion vector mv obtained through the merge coding mode is mmv, mmvx is an x mmv, and a motion vector mv obtained through motion estimation me is nmv; if the current mmvx is consistent with the nmv value, not performing mc calculation, but copying nmv the pred pixel result obtained by mc calculation to serve as the pred pixel of the current mmvx, so that the mc calculation of the current mmvx is omitted; if the current mmvx and nmv values are inconsistent, the mmvx reference frame pixels need to be read, and mc operation is performed to obtain pred pixels.

The method further comprises the steps of: the motion compensation result is stored in the random access memory ram.

In the method, the pipeline stage motion estimation me of the hardware structure is in advance of the merge mode, wherein the pipeline stage refers to the pipeline stage number, and the first-out result of the stage number is the first-in result, that is, the motion vector mv of the me is obtained before the merge mv; meanwhile, the mv of the merge mode has correlation with the mv of the motion estimation me in space and time, so that the mv of the merge is equal to the me mv, and has the characteristic that the pred pixel value is the same if the mv is equal.

According to the method, by utilizing the characteristics, after a candidate mv list is obtained in a Merge mode of an H265 format, mc operation is needed for each mmvx.

The mv list is mmv0-mmv4.

The motion compensation mc operation, that is, the position of the reference block in the reference frame is calculated by using the position of the current block and the motion vector nmv, and the reference block pixel is used as the predicted value of the current block pixel.

The method further comprises the steps of:

s1, reading a reference frame pixel of nmv, performing motion compensation mc operation on nmv, and storing nmv pred pixels into a random access memory ram;

s2, judging whether the current mmvx is the same as nmv:

s2.1, comparing mmv0 with nmv with the 1 st mmv,

when the two values are equal, the mc operation is not performed on mmv0, the pred pixel of nmv is directly copied as the pred pixel of mmv0, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv0 needs to be read, mc operation is performed on mmv0 to obtain pred pixel, and step S3 is performed;

s2.2, comparing the numbers 2 and mmv to mmv1 and nmv,

when the two values are equal, the mc operation is not performed on mmv1, the pred pixel of nmv is directly copied as the pred pixel of mmv1, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv1 needs to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s2.3, comparing the 3 rd mmv to mmv2 and nmv,

when the two values are equal, the mc operation is not performed on mmv, the pred pixel of nmv is directly copied as the pred pixel of mmv2, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv2 needs to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s2.4, comparing the No. 4 mmv with the mmv and nmv,

when the two values are equal, the mc operation is not performed on mmv3, the pred pixel of nmv is directly copied as the pred pixel of mmv3, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv3 needs to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s2.5, comparing the 5 th mmv with the mmv th and nmv,

when the two values are equal, the mc operation is not performed on mmv4, the pred pixel of nmv is directly used as the pred pixel of mmv4, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv is required to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s3, storing pred pixels into ram.

Thus, the present application has the advantages that: mmv and nmv can omit the operation of mc corresponding to mmv. The proposed hardware structure can accelerate hardware, reduce the operation amount of software, and does not affect the coding quality.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate and together with the description serve to explain the invention.

FIG. 1 is a flow diagram of a prior art hardware design with mc operations performed on 1 nmv and 5 mmv.

Fig. 2 is a pseudo code schematic diagram of the operational flow of fig. 1.

FIG. 3 is a flow chart of the mc operation process for mmv0-mmv in the present application.

FIG. 4 is a pseudo code schematic diagram of the mc hardware acceleration operation flow for 1 nmv and 5 mmv in this application.

Fig. 5 is a flow chart of the method according to the invention.

Detailed Description

In order that the technical content and advantages of the present invention may be more clearly understood, a further detailed description of the present invention will now be made with reference to the accompanying drawings.

The terms that will be used herein also include:

nmv: abbreviations for normal mv, herein denote mv obtained through me;

mmv: abbreviations for merge mv, herein means mv obtained via merge technology; ref data: the reference frame data, i.e. selecting some frames from the encoded frames as the reference data of the current frame to be encoded, is mainly used for obtaining the prediction block from the reference frames in the motion compensation process. The number of the optional reference frames in the H265 is 1-15;

ddr: the double-rate synchronous dynamic random access memory stores the whole frame of reference frame data therein and has large capacity;

ram: a random memory, in which pred pixels are stored, with smaller capacity, and a plurality of PU pred pixels can be stored in general;

maxNumMergeCand: number of candidates mv to configure the merge mode;

mmvx: an x mmv;

read_xxx_ref: reading pixel values of a reference frame corresponding to xxx;

do_xxx_mc: performing mc on pixel values of a reference frame corresponding to xxx;

store_xxx_pred: storing pred pixel values of xxx to ram;

copy_ nmv _pred: copy the pred pixel value of xxx to ram.

Since the pipeline level me of the hardware structure is ahead of merge, that is, the mv of the me is obtained ahead of merge mv. At the same time, there is a strong correlation between the merge mv and the me mv in space and time, so that the merge mv is equal to the me mv with a high probability, and the equality of mv means that the pred pixel value is the same.

By utilizing the characteristics, H265 needs to perform mc operation on each mmvx after obtaining a candidate mv list (mmv 0-mmv 4) in the Merge mode, and performs the mc operation on the mmvx (mmv-mmv 4), and the following method is proposed herein, as shown in fig. 3, firstly, judging whether the current mmvx is the same as nmv:

(1) If the current mmvx is consistent with the nmv value, then the mc calculation is not performed, but the pred pixel result obtained by the mc calculation is copied nmv as the pred pixel (copy_ nmv _pred) of the current mmvx, so that the mc calculation of the current mmvx is omitted.

(2) If the current mmvx does not match nmv, then the mmvx reference frame pixel (read nmv ref) needs to be read and mc operation (do mmvx mc) is performed to obtain pred pixels.

And performing mc operation, namely calculating the position of the reference block in the reference frame by using the position of the current block and the motion vector nmv, and taking the pixels of the reference block as the predicted value of the pixels of the current block.

As shown in fig. 5, the present invention relates to a hardware acceleration method for motion compensation operation in a merge mode of H265 format, which includes the following steps:

s1, reading a reference frame pixel of nmv, performing motion compensation mc operation on nmv, and storing nmv pred pixels into a random access memory ram;

s2, judging whether the current mmvx is the same as nmv:

if the current mmvx matches the nmv value, then no mc calculation is performed, but the pred pixel result of nmv is copied as the pred pixel (copy_ nmv _pred) of the current mmvx, omitting the mc operation of the current mmvx.

If the current mmvx does not match nmv, then the mmvx reference frame pixel (read nmv ref) needs to be read and mc operation (do mmvx mc) is performed to obtain pred pixels.

S3, storing pred pixels into ram.

The step S2 further comprises:

s2.1, comparing mmv0 with nmv with the 1 st mmv,

when the two values are equal, the mc operation is not performed on mmv0, the pred pixel of nmv is directly copied as the pred pixel of mmv0, and step S3 is performed;

when the two values are not equal, the reference frame pixel of mmv0 needs to be read, and mc is performed on mmv0

Calculating to obtain pred pixels, and performing step S3;

s2.2, comparing the numbers 2 and mmv to mmv1 and nmv,

when the two values are equal, the mc operation is not performed on mmv1, the pred pixel of nmv is directly copied as the pred pixel of mmv1, and step S3 is performed;

when the two values are not equal, the reference frame pixel of mmv1 needs to be read, and mc is performed on mmv1

Calculating to obtain pred pixels, and performing step S3;

s2.3, comparing the 3 rd mmv to mmv2 and nmv,

when the two values are equal, the mc operation is not performed on mmv, the pred pixel of nmv is directly copied as the pred pixel of mmv2, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv2 needs to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s2.4, comparing the No. 4 mmv with the mmv and nmv,

when the two values are equal, the mc operation is not performed on mmv3, the pred pixel of nmv is directly copied as the pred pixel of mmv3, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv3 needs to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s2.5, comparing the 5 th mmv with the mmv th and nmv,

when the two values are equal, the mc operation is not performed on mmv4, the pred pixel of nmv is directly used as the pred pixel of mmv4, and step S3 is performed;

if the two values are not equal, the reference frame pixel of mmv needs to be read, and the pred pixel is obtained by performing mc operation on mmv4, and step S3 is performed.

The detailed flow is as follows:

s1, reading a reference frame pixel of nmv, performing mc operation on nmv, and storing nmv pred pixels to ram.

S2.1, comparing mmv0 with nmv, if the values are equal, then not carrying out mc operation on mmv0, directly copying nmv pred pixels as mmv0 pred pixels, and storing the result to ram; when the two values are not equal, the reference frame pixel of mmv0 needs to be read, and the mc operation is performed on mmv to obtain pred pixel, and the pred pixel is stored in ram.

S2.2, comparing mmv1 with nmv, if the values are equal, then not carrying out mc operation on mmv1, directly copying nmv pred pixels as mmv1 pred pixels, and storing the pred pixels to ram; when the two values are not equal, the reference frame pixel of mmv1 needs to be read, and the mc operation is performed on mmv to obtain pred pixel, and the pred pixel is stored in ram.

S2.3, comparing mmv with nmv, if the values are equal, then not carrying out mc operation on mmv, directly copying nmv pred pixels as mmv2 pred pixels, and storing the result to ram; when the two values are not equal, the reference frame pixel of mmv2 needs to be read, and the pred pixel is obtained by mc operation on mmv and stored in ram.

S2.4, comparing mmv3 with nmv, if the values are equal, then the mc operation is not performed on mmv3, the pred pixel of nmv is directly copied as the pred pixel of mmv, and the pred pixel is stored in ram. When the two values are not equal, the reference frame pixel of mmv3 needs to be read, and the pred pixel is obtained by mc operation on mmv and stored in ram.

S2.5, comparing mmv with nmv, if the values are equal, then the mc operation is not performed on mmv, and the pred pixel of nmv is directly taken as the pred pixel of mmv4 and is stored into ram. When the two values are not equal, the reference frame pixel of mmv needs to be read, and the pred pixel is obtained by mc operation on mmv4 and stored in ram.

In the process, the mc operation of the merge mv which is equal to the me mv and the reading of the reference frame can be directly omitted, so that hardware resources are saved, and hardware acceleration is realized.

Pseudo code of the flow of the mc hardware acceleration operation for 1 nmv and 5 mmv as presented in this application is shown in fig. 4.

In the present application, when mmv and nmv are equal, the operation of mc corresponding to mmv may be omitted. The proposed hardware structure can accelerate hardware, reduce the operation amount of software, and does not affect the coding quality.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, and various modifications and variations can be made to the embodiments of the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (7)

1. The hardware acceleration method of motion compensation operation in the merge mode of the H265 format is characterized by comprising the steps of judging whether the current mmvx is identical to nmv, wherein the motion vector mv obtained through the merge coding mode is mmv, the mmvx is the x mmv, and the motion vector mv obtained through motion estimation me is nmv; if the current mmvx is consistent with the nmv value, performing motion compensation mc operation, and copying nmv a pred pixel result obtained by the mc operation to serve as a pred pixel of the current mmvx, so that the mc operation of the current mmvx is omitted; if the current mmvx and nmv values are inconsistent, the mmvx reference frame pixels need to be read, and mc operation is performed to obtain pred pixels.

2. The method for hardware acceleration of motion compensation operations in merge mode in H265 format of claim 1, further comprising: the motion compensation result is stored in the random access memory ram.

3. The method for accelerating the hardware of the motion compensation operation in the merge mode with the H265 format according to claim 2, wherein the pipeline level motion estimation me of the hardware structure is the one leading the merge mode, that is to say, the motion vector mv of the me is obtained before the merge mv; at the same time, the mv of the merge mode has correlation with the mv of the motion estimation me in space and time, so that the mv of the merge is equal to the mv of the me, and the same characteristic is that the pred pixel value is the same if the mv is equal.

4. The method for accelerating motion compensation operation in Merge mode with H265 format according to claim 3, wherein said method uses said characteristics, and after obtaining candidate mv list in Merge mode with H265 format, it needs to make mc operation for each mmvx.

5. The hardware acceleration method of motion compensation operation in merge mode of H265 format according to claim 4, wherein the mv list is mmv0-mmv4.

6. The method for hardware acceleration of motion compensation operation in merge mode according to claim 5, wherein the motion compensation mc operation is to calculate the position of the reference block in the reference frame by using the position of the current block and the motion vector nmv, and use the reference block pixel as the predicted value of the current block pixel.

7. The method for hardware acceleration of motion compensation operations in merge mode in H265 format according to claim 6, further comprising the steps of:

s1, reading a reference frame pixel of nmv, performing motion compensation mc operation on nmv, and storing nmv pred pixels into a random access memory ram;

s2, judging whether the current mmvx is the same as nmv:

s2.1, comparing mmv0 with nmv with the 1 st mmv,

when the two values are equal, the mc operation is not performed on mmv0, the pred pixel of nmv is directly copied as the pred pixel of mmv0, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv0 needs to be read, mc operation is performed on mmv0 to obtain pred pixel, and step S3 is performed;

s2.2, comparing the numbers 2 and mmv to mmv1 and nmv,

when the two values are equal, the mc operation is not performed on mmv1, the pred pixel of nmv is directly copied as the pred pixel of mmv1, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv1 needs to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s2.3, comparing the 3 rd mmv to mmv2 and nmv,

when the two values are equal, the mc operation is not performed on mmv, the pred pixel of nmv is directly copied as the pred pixel of mmv2, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv2 needs to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s2.4, comparing the No. 4 mmv with the mmv and nmv,

when the two values are equal, the mc operation is not performed on mmv3, the pred pixel of nmv is directly copied as the pred pixel of mmv3, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv3 needs to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s2.5, comparing the 5 th mmv with the mmv th and nmv,

when the two values are equal, the mc operation is not performed on mmv4, the pred pixel of nmv is directly used as the pred pixel of mmv4, and step S3 is performed;

when the two values are different, the reference frame pixel of mmv is required to be read, mc operation is performed on mmv to obtain pred pixel, and step S3 is performed;

s3, storing pred pixels into ram.

Images