CN111698502A - VVC (variable visual code) -based affine motion estimation acceleration method and device and storage medium - Google Patents

Abstract

The invention discloses an affine motion estimation acceleration method, device and storage medium based on VVC coding, comprising the following steps: if RDcost _AffineMerge >λ*RDcost _Merge is satisfied, or, when the current coding unit constructs an Affine Merge mode candidate list, If the optimal prediction mode of the adjacent coding unit does not have an affine mode, skip the affine motion estimation process of the current coding unit; otherwise, the current coding unit continues to perform affine motion estimation, where RDcost _AffineMerge indicates that the current coding unit executes the Affine Merge mode The rate-distortion cost of ; RDcost _Merge represents the rate-distortion cost of the current coding unit executing the Merge mode, λ is the threshold, and λ≥1. The present invention executes the Affine Merge mode information according to the current coding unit and skips unnecessary affine motion estimation, which can reduce the time complexity of the encoder, effectively improve the efficiency of the encoder, and is conducive to practical application.

Description

Affine motion estimation acceleration method, device and storage medium based on VVC coding

technical field

The present invention relates to the technical field of video coding, and in particular, to an acceleration method, device and storage medium for affine motion estimation based on VVC coding.

Background technique

At present, High Efficiency Video Coding (HEVC for short) has been widely used in commercial use, but it still cannot meet the growing video demand. Therefore, the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG) established the Joint Video Experts Group JVET (Joint Video Exploration Team) to study a new generation of video coding technology. At the meeting held in San Diego, USA on April 10, 2018, JVET released a draft of a new generation of video coding technology, naming the new video standard Versatile Video Coding (VVC for short), as of April 2020, The JVET conference has been in progress for the seventeenth time, and the official test model VTM has been updated to version 8.0. It was designed with two main goals, firstly to specify a video encoding technology with compression capabilities far beyond those of previous generations of such standards, and secondly, the technology being highly versatile and can be used more efficiently than previous generations wider applications covered by the standard. The new generation standard introduces many new coding tools, such as the use of QTMT (Quadtree with Multi-type tree) division structure to replace the quadtree division of HEVC, adaptive motion vector precision and motion compensation technology based on Affine (affine) Although these tools significantly improve the coding efficiency, they also greatly increase the time complexity of the encoder. Such high complexity is not conducive to the use and promotion of the standard in the future, so it is very important to reduce the encoding time of the encoder.

Affine-based motion compensation technology is a displacement transformation model used for fading in/out, rotation, scaling and other irregular motions, which solves the problem of inaccurate motion compensation of translation transformation models. Two affine motion models are provided in the VVC standard: a 4-parameter model and a 6-parameter model. The 4-parameter model needs to obtain the upper-left corner motion vector and the upper-right corner motion vector of the coding unit as its motion control point, and obtain the motion vector of each 4×4 small block in the coding unit according to the parameter model formula. Perform motion compensation. Compared with the 4-parameter model, the 6-parameter model has one more motion vector in the lower left corner as the motion control point, and the subsequent process is the same as that of the 4-parameter model. Affine-based motion compensation technology includes AffineMerge mode (affine merge mode) and affine motion estimation mode (Affine Motion Estimation), both of which are included in the process of inter-frame mode selection, Affine Merge mode is calculated before the ordinary Merge mode Rate-distortion cost RDCost (Comprehensive evaluation of rate and distortion), affine motion estimation and ordinary motion estimation together calculate the rate-distortion cost RDCost. Affine Merge mode is a coding mode introduced by the VVC standard. It uses the motion vector information of adjacent coding units to construct an Affine Merge candidate list, finds the optimal control point motion vector in the candidate list, and then performs affine motion compensation; Merge The mode continues the characteristics of the predecessor standard HEVC, and uses the motion vectors of adjacent coding units to construct a Merge candidate list, and selects the optimal motion vector for motion compensation.

Affine motion estimation is mainly divided into the following steps:

Step 1: Construct an affine AMVP (Advanced Motion Vector Prediction Advanced Motion Vector Prediction Technology) list, fill in two groups of CPMV (Motion Vection Prediction Predictable Control Point Motion Vectors), and take the optimal set as the affine motion estimation starting point.

Step 2: One-way prediction, perform forward prediction and backward prediction respectively, and perform step 4 for each reference frame in the forward reference list List0 and each reference frame in the backward list List1 to obtain the forward optimal control point MV (MotionVection motion vector) and backward optimal control point MV.

Step 3: Bidirectional prediction: Step 4 is performed on the reference lists List0 and List1 to obtain the optimal control point MV combined by the bidirectional prediction, and the two are weighted and averaged.

Step 4: Perform motion estimation and motion compensation according to CPMV. Use the gradient descent search method to obtain a set of optimal control point MVs, obtain the motion vector of each 4×4 small block in the coding unit CU according to the control point MV, perform motion compensation on it, and calculate the total RD Cost.

Step 5: Compare the optimal rate-distortion cost RD cost for forward prediction, the optimal rate-distortion cost RDCost for backward prediction, and the optimal rate-distortion cost RD Cost for bidirectional prediction. Set the mode with the least cost as the optimal mode of the current affine motion estimation, and save the prediction direction, reference frame, control point MV and other information.

In the current VVC standard encoder, since all coding units need to do affine motion estimation once; and affine motion estimation is done for each reference frame in each prediction direction, the coding time is too long. defect.

SUMMARY OF THE INVENTION

The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an acceleration method, device and storage medium for affine motion estimation based on VVC coding.

A first aspect of the present invention provides an acceleration method for affine motion estimation based on VVC coding, comprising the following steps:

If RDcost _AffineMerge >λ*RDcost _Merge is satisfied, or, when the current coding unit constructs the Affine Merge mode candidate list, the optimal prediction mode of the adjacent coding unit does not have an affine mode, then skip the affine motion of the current coding unit Estimation process; otherwise, the current coding unit continues to perform affine motion estimation, wherein RDcost _AffineMerge represents the rate-distortion cost of the current coding unit performing the Affine Merge mode; RDcost _Merge represents the current coding unit performing the rate-distortion cost of the Merge mode , λ is the threshold, and λ≥1.

According to the embodiments of the present invention, at least the following beneficial effects are obtained:

The present invention executes the Affine Merge mode information according to the current coding unit and skips unnecessary affine motion estimation, which can reduce the time complexity of the encoder, effectively improve the efficiency of the encoder, and is conducive to practical application.

According to some embodiments of the present invention, if RDcost _AffineMerge >λ*RDcost _Merge is satisfied and the optimal prediction mode of the adjacent coding unit has no affine mode, the affine motion estimation process of the current coding unit is skipped.

According to some embodiments of the present invention, the current coding unit continues to perform affine motion estimation, including the following steps:

If the optimal prediction mode of the parent CU of the current coding unit is affine motion estimation, the current coding unit only searches for and multiplexes the optimal prediction direction and optimal reference frame of the parent CU.

According to some embodiments of the present invention, if the optimal prediction mode of the parent CU of the current coding unit is not affine motion estimation, the following steps are further included:

If the current coding unit is the lower sub-CU of the horizontal binary division, and the optimal prediction direction and the optimal reference frame of the parent CU are consistent with the optimal prediction direction and the optimal reference frame of the upper sub-CU of the horizontal binary division, Then the current coding unit only searches for and multiplexes the optimal prediction direction and optimal reference frame of the parent CU.

According to some embodiments of the present invention, if the optimal prediction mode of the parent CU of the current coding unit is not affine motion estimation, the following steps are further included:

If the current coding unit is the right sub-CU of the vertical binary partition, and the optimal prediction direction and the optimal reference frame of the parent CU are consistent with the optimal prediction direction and the optimal reference frame of the left sub-CU of the vertical binary partition , then the current coding unit only searches for and multiplexes the optimal prediction direction and the optimal reference frame of the parent CU;

According to some embodiments of the present invention, if the optimal prediction mode of the parent CU of the current coding unit is not affine motion estimation, the following steps are further included:

If the current coding unit is the first sub-CU of the horizontal three-way division, the current coding unit only searches for and multiplexes the optimal prediction direction and the optimal reference frame of the sub-CU on the horizontal binary division;

If the current coding unit is the second sub-CU of the horizontal three-pronged division, the current coding unit simultaneously searches for the optimal prediction direction and the optimal reference frame of the sub-CU on the horizontal binary division, and the optimal prediction direction and the optimal reference frame of the sub-CU under the horizontal binary division The optimal prediction direction and the optimal reference frame, and the optimal prediction direction and the optimal reference frame are selected according to the rate-distortion cost of the sub-CU on the horizontal binary division and the sub-CU under the horizontal binary division;

If the current coding unit is the third sub-CU of the horizontal three-way division, the current coding unit only searches for and multiplexes the optimal prediction direction and the optimal reference frame of the lower sub-CU under the horizontal binary division.

According to some embodiments of the present invention, if the optimal prediction mode of the parent CU of the current coding unit is not affine motion estimation, the following steps are further included:

If the current coding unit is the first sub-CU of the vertical triple division, the current coding unit only searches for and multiplexes the optimal prediction direction and the optimal reference frame of the left sub-CU of the vertical binary division;

If the current coding unit is the second sub-CU of the vertical three-way division, the current coding unit simultaneously searches for the optimal prediction direction and the optimal reference frame of the left sub-CU of the vertical binary division, and the right sub-CU of the vertical binary division The optimal prediction direction and the optimal reference frame are selected, and the optimal prediction direction and the optimal reference frame are selected according to the rate-distortion cost of the vertical binary division of the left sub-CU and the vertical binary division of the right sub-CU;

If the current coding unit is the third sub-CU of the vertical triple division, the current coding unit only searches for and multiplexes the optimal prediction direction and the optimal reference frame of the right sub-CU of the vertical binary division.

According to some embodiments of the present invention, the value of λ is 1.05.

A second aspect of the present invention provides an affine motion estimation acceleration device based on VVC coding, comprising: at least one control processor and a memory for communicating with the at least one control processor; the memory stores Instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform a VVC coding based affine motion estimation as described above acceleration method.

An embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute the above-mentioned VVC encoding-based Affine motion estimation acceleration method.

The VVC coding-based affine motion estimation acceleration device and the readable storage medium provided by the embodiments of the present invention can achieve the same beneficial effects as the above method.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a schematic diagram of a block division method provided by the prior art;

2 is a schematic flowchart of a VVC coding-based acceleration method for affine motion estimation according to an embodiment of the present invention;

3 is a schematic diagram of the position of inheritable affine motion predictors provided by the prior art;

4 is a schematic diagram of control point motion vector inheritance provided by the prior art;

5 is a schematic flowchart of an acceleration method for affine motion estimation based on VVC coding provided by an embodiment of the present invention;

6 is a schematic flowchart of an acceleration method for affine motion estimation based on VVC coding provided by an embodiment of the present invention;

7 is a schematic flowchart of an acceleration method for affine motion estimation based on VVC coding provided by an embodiment of the present invention;

8 is a schematic flowchart of an acceleration method for affine motion estimation based on VVC coding provided by an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of an acceleration device for affine motion estimation based on VVC coding according to an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, only used to explain the present invention, and should not be construed as a limitation of the present invention.

In the prior art, the VVC standard encoder stipulates that in the process of inter-frame mode selection, each coding unit (Coding Unit, CU for short) needs to traverse each mode, and select the mode with the least rate-distortion cost as the optimal prediction mode , however, some video sequences do not have or rarely have affine motion. At this time, the affine motion estimation will not be selected as the optimal prediction mode, so there are a lot of redundant affine motion estimations that limit the efficiency of the encoder. , making the encoding time too long.

Affine motion estimation is divided into unidirectional prediction and bidirectional prediction, and unidirectional prediction is further divided into forward prediction and backward prediction. During forward prediction, the candidate reference frame list List0 is traversed (the List0 list is filled with a certain number of previously encoded frames). frame), select the forward optimal reference frame according to the rate-distortion cost, and traverse each of the candidate reference frame list List1 (the List1 list is filled with a certain number of previously encoded frames) during backward prediction. For the reference frame, the backward optimal reference frame is selected, and the bidirectional prediction will select a set of optimal reference frames from the combination of the forward list and the backward list. It takes a lot of time to traverse all prediction directions and reference frames, resulting in a heavy coding burden.

The new standard adopts a more flexible block division method: a structure based on quadtrees and nested multi-type trees. A coding unit is firstly divided into a quadtree, and then is divided into horizontal binary division, vertical binary division, horizontal trigeminal division, and vertical trigeminal division in turn. As shown in Figure 1, there are horizontal two-pronged division, vertical two-pronged division, horizontal three-pronged division, and vertical three-pronged division. For example: a 64*64 block will first try quadtree division (to get 4 32*32 blocks), then try horizontal binary division (upper sub-CU block 64*32 and lower sub-CU block 64*32), vertical binary division Division (left sub-CU block 32*64 and right sub-CU block 32*64), horizontal three-way division (upper sub-CU block 64*16 and neutron CU block 64*32 and lower sub-CU block 64*16), vertical three-way division The same is true. At this time, the 64*64 block is the parent node of all divided sub-CU blocks. The rate-distortion cost is calculated for each division mode, and the rate-distortion cost of no division and these division modes is compared, and the one with the smallest value is selected as the optimal prediction mode.

First embodiment:

2, a kind of affine motion estimation acceleration method based on VVC coding is provided, comprising the following steps:

S100, obtain the rate-distortion cost RDcost _AffineMerge that current coding unit performs Affine Merge mode and the rate-distortion cost RDcost _Merge that performs Merge mode; Obtain current coding unit when constructing Affine Merge mode candidate list, the optimal prediction mode of adjacent coding unit;

S200. If RDcost _AffineMerge >λ*RDcost _Merge , and the optimal prediction mode of the adjacent coding unit does not have an affine mode, skip the affine motion estimation process of the current coding unit, where λ is a threshold and λ≥1.

In this embodiment, two motion affine estimation skipping methods are included, as follows:

First, because both motion affine estimation and Affine Merge belong to affine motion methods, the rate-distortion cost changes of the two are synchronized to a certain extent. When the rate-distortion cost RDcost _AffineMerge of Affine Merge is greater than the rate-distortion cost RDcost _Merge of ordinary Merge mode, the rate-distortion cost of affine motion estimation is very likely to be greater than the rate-distortion cost RDcost _Merge of ordinary Merge mode. It is estimated that there is a high probability that it will not be selected as the optimal prediction mode of the current coding unit, and the affine motion estimation process of the current coding unit can be skipped to shorten the coding time of the encoder.

Second, before the current coding unit performs the affine motion estimation mode, it will perform the Affine Merge mode and the ordinary merge mode. When constructing the Affine Merge candidate list, it will detect whether the optimal prediction mode of the adjacent coding unit is affine or not. If there is no mode, it can indicate that the current coding unit is unlikely to select affine motion estimation, and the affine motion estimation process of the current coding unit can be skipped to shorten the coding time of the encoder.

The following briefly introduces the Affine Merge candidate list and adjacent coding units;

The following three CPMV candidate types are used to form the Affine Merge candidate list:

(1) Inherited Affine Merge candidates derived from the CPMV of adjacent CUs;

(2) Construct affine merge candidate CPMVs derived using translation MVs of adjacent CUs;

(3) 0 vector MV;

In VVC coding, there are at most two inherited affine candidates, which are derived from the affine motion model of adjacent coding units, one from the left-adjacent CU and the other from the upper phase Received from neighboring CU. The candidate blocks are shown in Figure 3. For the left predictor, the scanning order is A0->A1, and for the upper predictor, the scanning order is B0->B1->B2, where A0, A1, B0, B1, and B2 are all adjacent coding units. Only the first inherited candidate from both sides is selected (one on the left, one on the upper side), no pruning is performed between the two inherited candidates. When an adjacent affine CU is identified, its control point motion vector is used in the current CU's Affine Merge candidate list. As shown in Figure 4, if the adjacent block A in the lower left corner is coded in affine mode, its upper left motion vector v2, upper right motion vector v3 and lower right motion vector v4 are obtained as a set of Affine Merge candidates for the current block list. The object that performs affine motion can be regarded as a whole, and the area it occupies is continuous in one frame. Therefore, when adjacent coding units have inheritable affine motion, the probability of the current coding unit performing affine motion will increase. Conversely, when the adjacent coding unit cannot find a coding unit that performs affine motion, the probability of affine motion of the current coding unit is small.

In step S200 in this embodiment, the above two skipping methods are combined, specifically: if the two conditions of RDcost _AffineMerge >λ*RDcost _Merge and the optimal prediction mode of the adjacent coding unit are not affine mode are satisfied at the same time , you can choose to skip the affine motion estimation of the current coding unit.

In this embodiment, the above two skipping methods are combined and selected to skip the affine motion estimation process, which reduces the coding time of the encoder compared with the prior art. Compared with skipping the affine motion estimation process using only any one of the above methods, the accuracy of skipping affine motion estimation in advance is undoubtedly improved, and the encoding time of the encoder is reduced.

as a preferred embodiment. When the threshold λ is set to 1.05, a balance between better coding quality and coding time saving can be achieved, and the specific experimental results refer to the fifth embodiment.

Second embodiment:

Referring to FIG. 5 , a method for accelerating affine motion estimation based on WC coding is provided, comprising the following steps:

S300, obtain the rate-distortion cost RDcost AffineMerge that the current coding unit executes the Affine Merge mode and the rate-distortion cost _{RDcost Merge} that executes the Merge mode;

S400. If RDcost _AffineMerge >λ*RDcost _Merge , skip the affine motion estimation process of the current coding unit, where λ=1.05.

As introduced in the first embodiment, which will not be repeated here, the solution of this embodiment can reduce the encoding time of the encoder, but it should be noted that although this solution can shorten the encoding time of the encoder, the accuracy is not as good as that of the first implementation example program.

Third embodiment:

6 , a method for accelerating affine motion estimation based on VVC coding is provided, comprising the following steps:

S500, obtain the optimal prediction mode of the adjacent coding unit when the current coding unit constructs the Affine Merge mode candidate list;

S600. If the optimal prediction mode of the adjacent coding unit does not have an affine mode, skip the affine motion estimation process of the current coding unit.

As introduced in the first embodiment, which will not be repeated here, the solution of this embodiment can reduce the encoding time of the encoder, but it should be noted that although this solution can shorten the encoding time of the encoder, the accuracy is not as good as that of the first implementation example program.

Fourth embodiment:

7, a kind of acceleration method of affine motion estimation based on VVC coding is provided, if the skip condition of the above-mentioned embodiment is not satisfied, the current coding unit continues to perform affine motion estimation, then comprises the following steps:

S700. If the optimal prediction mode of the parent CU of the current coding unit is affine motion estimation, the current coding unit only searches for and multiplexes the optimal prediction direction and the optimal reference frame of the parent CU.

Because, in VVC encoding, the image content of the parent CU includes the image content of the child CU, so the parent CU can well represent the motion direction and texture of the child CU to a certain extent, so only the previously encoded parent CU needs to be reused. The optimal prediction direction and the optimal reference frame are not required to search for other reference directions and reference frames.

Therefore, in this embodiment, when neither of the two skipping methods described in the first embodiment is satisfied, and the optimal prediction mode of the parent CU of the current coding unit is affine motion estimation, the current coding unit only has Search and multiplex the optimal prediction direction and optimal reference frame of the parent CU.

Fifth embodiment:

8 , a method for accelerating affine motion estimation based on VVC coding is provided, comprising the following steps:

A100. When the current coding unit executes the Affine Merge mode, save the number of Affine modes of the adjacent coding units that construct the Affine Merge mode candidate list, and denote it as NumNeighborAffine.

A200: Save the rate-distortion cost of the current coding unit executing the Affine Merge mode, denoted as RDcost _AffineMerge , and save the rate-distortion cost of the current coding unit executing the normal Merge mode, denoted as RDcost _Merge .

A300. If RDcost _AffineMerge >1.05*RDcost _Merge and NumNeighborAffine is equal to 0, skip the affine motion estimation process; otherwise, go to step A400.

It is the same as the first embodiment and will not be repeated here.

A400. If the optimal prediction mode of the parent CU of the current coding unit is affine motion estimation, the current coding unit only searches for the optimal prediction direction DIR _par and the optimal reference frame REF _par of the parent CU; otherwise, go to step A500.

It is the same as the fourth embodiment and will not be repeated here.

A500. Judging the division type and location of the current coding unit is divided into the following situations:

Because, the sub-block distribution of the horizontal trigeminal division and the horizontal binary division have a large part of the overlap, for example: the upper sub-CU of the horizontal binary division and the upper sub-CU of the horizontal trigeminal division have half of the area coincident, and their movements The correlation is very strong, and the horizontal binary division is performed before the horizontal three-fork division, so only the prediction direction and reference frame of the horizontal two-fork division sub-block can be saved for the subsequent horizontal three-fork division sub-blocks. Similarly, the vertical three-fork division is also The information of the vertical binary division can be multiplexed, and thus the reference frame search time of the horizontal/vertical triple division can be reduced.

A501. If the current coding unit is the lower sub-CU of the horizontal binary division, and the optimal prediction direction and the optimal reference frame of the parent CU are consistent with the optimal prediction direction and the optimal reference frame of the upper sub-CU of the horizontal binary division, Then the current coding unit only searches for and multiplexes the optimal prediction direction DIR _par and the optimal reference frame REF _par of the parent CU.

It should be noted that, in this step A501, only the affine motion estimation process of the lower sub-CU of the horizontal binary partition can be accelerated, while the upper sub-CU of the horizontal binary partition is still normally subjected to affine motion estimation.

A502. If the current coding unit is the right sub-CU of the vertical binary division, and the optimal prediction direction and the optimal reference frame of the parent CU are consistent with the optimal prediction direction and the optimal reference frame of the left sub-CU of the vertical binary division , the current coding unit only searches for and multiplexes the optimal prediction direction DIR _par and the optimal reference frame REF _par of the parent CU.

It should be noted that, in this step A502, only the affine motion estimation process of the right sub-CU of the vertical binary division can be accelerated, while the left sub-CU of the vertical binary division still performs affine motion estimation normally.

A503. If the current coding unit is the first sub-CU of the horizontal three-way division, the current coding unit only searches for and multiplexes the optimal prediction direction DIR _{BT_UP} and the optimal reference frame REF _{BT_UP} of the sub-CU on the horizontal binary division.

A504. If the current coding unit is the second sub-CU of the horizontal three-pronged division, the current coding unit simultaneously searches for the optimal prediction direction DIR _{BT_UP} and the optimal reference frame REF _{BT_UP} of the sub-CU on the horizontal binary division, and the sub-CU under the horizontal binary division simultaneously. The optimal prediction direction DIR _{BT_DOWN} and the optimal reference frame REF _{BT_DOWN} of the CU are selected, and the optimal prediction direction and the optimal reference frame are selected according to the rate-distortion cost of the sub-CU on the horizontal binary division and the sub-CU under the horizontal binary division.

It should be noted that, in this step A504, the one with the smaller rate-distortion cost is selected. And because the second sub-CU area of the horizontal three-fork division includes a part of the upper sub-CU divided by the horizontal binary division and a part of the lower sub-CU of the horizontal binary division, it is uncertain whether the second sub-CU is consistent with the motion of the upper sub-CU or the lower sub-CU. The CU movements are consistent, so they all search for the best one.

A505. If the current coding unit is the third sub-CU of the horizontal three-way division, the current coding unit only searches for and multiplexes the optimal prediction direction DIR _{BT_DOWN} and the optimal reference frame REF _{BT_DOWN} of the lower sub-CU under the horizontal binary division.

A506. If the current coding unit is the first sub-CU of the vertical three-way division, the current coding unit only searches for and multiplexes the optimal prediction direction DIR _{BT_LEFT} and the optimal reference frame REF _{BT_LEFT} of the left sub-CU of the vertical binary division.

A507. If the current coding unit is the second sub-CU of the vertical three-way division, the current coding unit searches for the optimal prediction direction DIR _{BT_LEFT} and the optimal reference frame REF _{Br_LEFT} of the left sub-CU of the vertical binary division at the same time, and the vertical binary division right The optimal prediction direction DIR _{BT_RIGHT} of the sub-CU and the optimal reference frame REF _{BT_RIGHT} , and the optimal prediction direction and the optimal reference frame are selected according to the rate-distortion cost of the vertical binary division of the left sub-CU and the vertical binary division of the right sub-CU.

It should be noted that, in this step A507, the one with the lower rate distortion cost is selected. And because the second sub-CU area of the vertical three-fork division includes a part of the left sub-CU of the vertical binary division and a part of the right sub-CU of the vertical binary division, it is uncertain whether the second sub-CU is consistent with the movement of the left sub-CU or not. It is consistent with the motion of the right sub-CU, so all searches are performed to select the optimal one.

A508. If the current coding unit is the third sub-CU of the vertical three-way division, the current coding unit only searches for and multiplexes the optimal prediction direction DIR _{BT_RIGHT} and the optimal reference frame REF _{BT_RIGHT} of the right sub-CU of the vertical binary division.

Steps A100 to A300, skip unnecessary affine motion estimation according to the Affine Merge mode information, thereby reducing the time complexity of the video encoder; Steps A400 to A500, reduce the prediction direction and reference according to the current coding unit division characteristics range of frames, thereby reducing the time complexity of the video encoder.

The method of this embodiment firstly skips unnecessary affine motion estimation according to the Affine Merge mode information, and secondly reduces the range of the prediction direction and the reference frame in combination with the coding unit division characteristics, thereby reducing the time complexity of the video encoder, effectively reducing the time complexity of the video encoder. Improve the efficiency of the encoder, and it is beneficial to put into practical application.

It will be implemented based on the VVC official standard reference encoder VTM8.0, using the encoder_randomaccess_vtm.cfg configuration file, the test sequence selects BasketballDrive, Cactus, RaceHorses three relatively violent sequences, of which Cactus has a large number of affine motions, which can be better reflect the effect of the algorithm. The coding performance is evaluated by two indicators: BDBR (Bjotegaard Delta Bit rate) and TS. BDBR represents the bit rate difference between the two coding methods under the same objective quality, which can comprehensively reflect the bit rate and quality of the video. Compared with the original algorithm, the proposed scheme has a higher code rate; TS indicates the reduction degree of the coding time of this scheme based on the original algorithm, and its calculation formula is as follows:

Among them, T _p is the total encoding time after adding the proposed algorithm to the encoder _VTM8.0 , and TO is the total encoding time of the original encoder VTM8.0. The results obtained from the simulation experiments are shown in Table 1 below:

Table 1

It can be seen from the data in Table 1 that the BDBR of the encoder after adding this scheme is only increased by 0.39% on average compared with the original encoder, and the average time of the encoder is reduced by 11.69%, indicating that the coding rate does not increase significantly. The time of the device has been greatly improved. It can be seen that the present invention reduces the coding time and improves the coding efficiency on the premise of ensuring the subjective quality of the video and the compression rate.

The value of the threshold value is described below, and the results obtained by the experimental analysis of the two sets of sequences are shown in Table 2:

Table 2

Among them, ΔT represents the time reduction rate of this scheme compared with the original algorithm. The hit rate represents the correct probability that the coding unit skips the affine motion estimation under the current threshold. It can be found from Table 2 that with the increase of the threshold λ, ΔT decreases and the hit rate increases, indicating that the larger the λ, the less the encoding time decreases, but the higher the accuracy of skipping affine motion estimation. In order to comprehensively consider the balance between encoding quality and encoding time saving, λ is preferably 1.05.

Sixth embodiment:

Referring to FIG. 9 , an acceleration device for affine motion estimation based on VVC coding is provided, and the device can be any type of intelligent terminal, such as a mobile phone, a tablet computer, a personal computer, and the like.

Specifically, the device includes: one or more control processors and memories, and one control processor is taken as an example in FIG. 9 . The control processor and the memory may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 9 .

As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, such as the VVC coding-based affine motion estimation acceleration device in the embodiment of the present invention Corresponding program instructions/modules, the control processor implements a VVC coding-based acceleration method for affine motion estimation in the above method embodiments by running the non-transitory software programs, instructions and modules stored in the memory.

The memory may include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required by at least one function. Additionally, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memories optionally include memories located remotely from the control processor, which remote memories can be connected to the VVC encoding based affine motion estimation acceleration device via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The one or more modules are stored in the memory, and when executed by the one or more control processors, execute the acceleration method for affine motion estimation based on VVC coding in the above method embodiments, for example, execute the above-described method. The method steps S100 to S200 in FIG. 2 .

Also provided is a computer-readable storage medium storing computer-executable instructions that are executed by one or more control processors, for example, by one of the control processors in FIG. 9 , the above-mentioned one or more control processors can be made to execute the acceleration method for affine motion estimation based on VVC coding in the above-mentioned method embodiments, for example, to execute the above-described method steps S100 to S200 in FIG. 2 .

From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform. Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and the program can be executed when the program is executed. , the flow of the above-mentioned method embodiments may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ReadOnly Memory, ROM), or a random access memory (Random Access Memory, RAM) or the like.

In the description of this specification, reference to the terms "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples," or the like, is meant to incorporate the embodiment. A particular feature, structure, material, or characteristic described by an example or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, The scope of the invention is defined by the claims and their equivalents.

Claims (10)

1. An affine motion estimation acceleration method based on VVC coding is characterized by comprising the following steps:

if RDcost is satisfied_AffineMerge>λ*RDcost_MergeOr, when the current coding unit constructs an affinity Merge mode candidate list, if the optimal prediction mode of the adjacent coding unit has no Affine mode, skipping the Affine motion estimation process of the current coding unit; otherwise, the current coding unit continues affine motion estimation, wherein RDcost_AffineMergeRepresenting the rate distortion cost of the current coding unit executing the affinity Merge mode; RDcost_MergeAnd representing the rate distortion cost of the Merge mode executed by the current coding unit, wherein lambda is a threshold value, and lambda is more than or equal to 1.

2. The method of claim 1, wherein the method comprises:

if RDcost is satisfied_AffineMerge>λ*RDcost_MergeAnd if the optimal prediction mode of the adjacent coding unit has no affine mode, skipping the affine motion estimation process of the current coding unit.

3. The method as claimed in claim 1, wherein the current coding unit continues affine motion estimation, and comprises the following steps:

if the optimal prediction mode of the parent CU of the current coding unit is affine motion estimation, the current coding unit only searches and multiplexes the optimal prediction direction and the optimal reference frame of the parent CU.

4. The method of claim 3, wherein if the optimal prediction mode of the parent CU of the current CU is not affine motion estimation, the method further comprises:

and if the current coding unit is the lower child CU divided in the horizontal binary manner, and the optimal prediction direction and the optimal reference frame of the father CU are consistent with the optimal prediction direction and the optimal reference frame of the upper child CU divided in the horizontal binary manner, the current coding unit only searches and multiplexes the optimal prediction direction and the optimal reference frame of the father CU.

5. The method of claim 4, wherein if the optimal prediction mode of the parent CU of the current CU is not affine motion estimation, the method further comprises:

and if the current coding unit is the right child CU in vertical binary division, and the optimal prediction direction of the father CU, the optimal reference frame and the optimal prediction direction and the optimal reference frame of the left child CU in vertical binary division are all consistent, the current coding unit only searches and multiplexes the optimal prediction direction and the optimal reference frame of the father CU.

6. The method of claim 5, wherein if the optimal prediction mode of the parent CU of the current CU is not affine motion estimation, the method further comprises:

if the current coding unit is the first sub-CU of the horizontal three-fork division, the current coding unit only searches and multiplexes the optimal prediction direction and the optimal reference frame of the sub-CU on the horizontal two-fork division;

if the current coding unit is the second sub-CU divided horizontally into three branches, the current coding unit simultaneously searches the optimal prediction direction and the optimal reference frame of the sub-CU divided horizontally into two branches, and selects the optimal prediction direction and the optimal reference frame according to the rate distortion cost of the sub-CU divided horizontally into two branches;

and if the current coding unit is the third sub-CU of the horizontal three-fork division, the current coding unit only searches and multiplexes the optimal prediction direction and the optimal reference frame of the lower sub-CU under the horizontal two-fork division.

7. The method of claim 6, wherein if the optimal prediction mode of the parent CU of the current coding unit is not affine motion estimation, the method further comprises:

if the current coding unit is the first sub-CU divided vertically in three forks, the current coding unit only searches and multiplexes the optimal prediction direction and the optimal reference frame of the left sub-CU divided vertically in two forks;

if the current coding unit is the second sub-CU divided vertically in three forks, the current coding unit simultaneously searches the optimal prediction direction and the optimal reference frame of the left sub-CU divided vertically in two forks, and the optimal prediction direction and the optimal reference frame of the right sub-CU divided vertically in two forks, and selects the optimal prediction direction and the optimal reference frame according to the rate distortion cost of the left sub-CU divided vertically in two forks and the rate distortion cost of the right sub-CU divided vertically in two forks;

and if the current coding unit is the third sub-CU of the vertical trigeminal division, the current coding unit only searches and multiplexes the optimal prediction direction and the optimal reference frame of the right sub-CU of the vertical binary division.

8. The acceleration method for affine motion estimation based on VVC coding of claim 1 or 7, wherein λ is 1.05.

9. An affine motion estimation accelerating device based on VVC coding, characterized in that: comprises at least one control processor and a memory for communicative connection with the at least one control processor; the memory stores instructions executable by the at least one control processor to enable the at least one control processor to perform the VVC code based affine motion estimation acceleration method of any one of claims 1 to 8.

10. A computer-readable storage medium characterized by: the computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method for accelerating affine motion estimation based on VVC coding according to any one of claims 1 to 8.

Images