CN112055207B - Time domain motion vector prediction method, device and storage medium - Google Patents
Time domain motion vector prediction method, device and storage medium Download PDFInfo
- Publication number
- CN112055207B CN112055207B CN202010784588.XA CN202010784588A CN112055207B CN 112055207 B CN112055207 B CN 112055207B CN 202010784588 A CN202010784588 A CN 202010784588A CN 112055207 B CN112055207 B CN 112055207B
- Authority
- CN
- China
- Prior art keywords
- block
- sub
- motion vector
- time domain
- current block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000013598 vector Substances 0.000 title claims abstract description 207
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000002123 temporal effect Effects 0.000 claims abstract description 112
- 230000004044 response Effects 0.000 claims description 15
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000007430 reference method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/109—Selection of coding mode or of prediction mode among a plurality of temporal predictive coding modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/567—Motion estimation based on rate distortion criteria
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application discloses a time domain motion vector prediction method, a device and a storage medium. The time domain motion vector prediction method comprises the following steps: when time domain motion vector prediction is performed based on the sub-blocks, dividing the current block to obtain a plurality of sub-blocks with the same size; and acquiring a temporal motion vector corresponding to a pixel block at any position in each sub-block, and determining the temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block. The time domain motion vector corresponding to the pixel block at any position of each sub-block is obtained, the time domain motion vector corresponding to the pixel block is used as the time domain motion vector of each sub-block, the position of the sub-block for obtaining the time domain motion vector is more flexible, and the positions of the time domain motion vectors obtained by different sub-blocks can be different, so that the accuracy of the time domain motion vector obtained by the sub-block is improved, and the accuracy of the time domain motion vector prediction is further improved.
Description
Technical Field
The application belongs to the technical field of video encoding and decoding, and particularly relates to a time domain motion vector prediction method, time domain motion vector prediction equipment and a storage medium.
Background
Because the data volume of the video image is relatively large, the main function of video coding is to compress video pixel data (RGB, YUV, etc.) into a video code stream, thereby reducing the data volume of the video and realizing the purposes of reducing network bandwidth and memory space in the transmission process.
In general, luminance and chrominance signal values of pixels of adjacent frames are relatively close in time, and have strong correlation. Inter prediction finds the closest matching block to the current block in the reference frame and records motion information between the current block and the matching block, such as motion vector MV (motion vector) and reference frame index. And coding the motion information and transmitting the motion information to a decoding end. At the decoding end, the decoder can find the matching block of the current block as long as the MV of the current block is resolved through the corresponding syntax element. And copying the pixel value of the matched block to the current block, namely the inter prediction value of the current block. However, the existing prediction method of motion information, such as motion vector, between the current block and the matching block is not accurate enough, and a more accurate prediction method is needed to solve the above-mentioned problems.
Disclosure of Invention
The application provides a time domain motion vector prediction method, equipment and a storage medium, which are used for solving the problem that motion vector prediction is not accurate enough.
In order to solve the technical problems, the application adopts a technical scheme that: a method of temporal motion vector prediction, the method comprising: when time domain motion vector prediction is carried out based on the sub-blocks, the current block is segmented, and a plurality of sub-blocks with the same size are obtained; and acquiring a temporal motion vector corresponding to a pixel block at any position in each sub-block, and determining the temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block.
According to an embodiment of the present application, the obtaining a temporal motion vector corresponding to a pixel block at any position in each sub-block, determining a temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block, includes: and acquiring a corresponding time domain motion vector by using a pixel block at the position of the upper left corner, the lower left corner, the upper right corner, the lower right corner or the central point of each sub-block so as to determine the time domain motion vector of each sub-block, wherein the side length of the pixel block is four pixels.
According to an embodiment of the application, the method further comprises: responding to the fact that a current sub-block cannot acquire a valid time domain motion vector; determining a valid temporal motion vector for the current sub-block based on a temporal motion vector for a pixel block at another location in the current sub-block; or acquiring the effective time domain motion vectors of the rest sub-blocks in the current block in a first order, and taking the effective time domain motion vectors of the rest sub-blocks with the previous acquisition order as the time domain motion vector of the current sub-block; or obtaining the effective time domain motion vectors of the first number of the rest sub-blocks in the current block in a second sequence or at will, and taking the weighted average as the effective time domain motion vectors of the current sub-block.
According to an embodiment of the present application, the dividing the current block to obtain a plurality of sub-blocks with the same size includes: dividing the current block according to the size of the current block to obtain a plurality of sub-blocks with the same size and side length larger than a preset value.
According to an embodiment of the present application, the dividing the current block according to the size of the current block includes: comparing lengths of a first side and a second side of the current block, the first side being adjacent to the second side; in response to the length ratio of the first side edge to the second side edge being greater than or equal to a first predetermined multiple, performing average division on the current block in a direction perpendicular to the first side edge, wherein the first predetermined multiple is greater than one; or in response to the length ratio of the second side edge to the first side edge being greater than or equal to a second predetermined multiple, dividing the current block evenly in a direction perpendicular to the second side edge, the second predetermined multiple being greater than one; or in response to the first side being equal to the second side, dividing the current block evenly by a cross division method.
According to an embodiment of the present application, the dividing the current block according to the size of the current block includes: acquiring lengths of a first side and a second side of the current block, wherein the first side is adjacent to the second side; in response to the length of the first side being at least twice the predetermined value, equally dividing the current block in a direction perpendicular to the first side; and/or, in response to the length of the second side being at least twice the predetermined value, dividing the current block equally in a direction perpendicular to the second side.
According to an embodiment of the present application, before the dividing the current block, the method further includes: acquiring two first time domain motion vectors corresponding to a first group of pixel blocks at the diagonal position of the current block and two second time domain motion vectors corresponding to a second group of pixel blocks at the diagonal position of the current block; judging whether the two first time domain motion vectors are identical or not, and/or judging whether the two second time domain motion vectors are identical or not; if yes, the current block is not segmented.
According to an embodiment of the present application, when performing temporal motion vector prediction based on sub-blocks, the method includes: judging whether the current prediction mode is skip/direct inter prediction mode, and whether the lengths of a first side edge and a second side edge of the current block are larger than or equal to the preset value, wherein at least one side edge is larger than or equal to twice of the preset value, and the first side edge is adjacent to the second side edge; if yes, performing time domain motion vector prediction based on the sub-blocks.
According to an embodiment of the application, the method further comprises: and performing motion compensation and distortion cost calculation on the time domain motion vectors of all the sub-blocks of the current block.
In order to solve the technical problems, the application adopts another technical scheme that: a method of temporal motion vector prediction, the method comprising: dividing the current block according to the size of the current block when performing time domain motion vector prediction based on the sub-blocks, and obtaining a plurality of sub-blocks with the same size and side length larger than a preset value; and obtaining the time domain motion vector of each sub-block.
In order to solve the technical problems, the application adopts another technical scheme that: an electronic device comprising a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement any of the time domain motion vector prediction methods described above.
In order to solve the technical problems, the application adopts another technical scheme that: a computer readable storage medium having stored thereon program data which when executed by a processor implements any of the time domain motion vector prediction methods described above.
The beneficial effects of the application are as follows: the method comprises the steps of obtaining a temporal motion vector corresponding to a pixel block at any position of each sub-block, and taking the temporal motion vector corresponding to the pixel block as the temporal motion vector of each sub-block. The sub-blocks can acquire the time domain motion vectors at different positions, so that the accuracy of acquiring the time domain motion vectors by the sub-blocks is improved, and the accuracy of predicting the time domain motion vectors is further improved.
Drawings
For a clearer description of the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the description below are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art, wherein:
FIG. 1 is a flow chart of an embodiment of a temporal motion vector prediction method of the present application;
FIG. 2 is a schematic view of a temporal co-located block range corresponding to a current block with different coordinates when obtaining a temporal motion vector of the current block according to an embodiment of the temporal motion vector prediction method of the present application;
FIG. 3 is a schematic diagram of a frame for determining whether a neighboring block in a spatial location is available when a temporal motion vector of a current block is obtained according to an embodiment of the temporal motion vector prediction method of the present application;
FIG. 4 is a flow chart of adding a termination condition to the start of subTMVP techniques in an embodiment of the temporal motion vector prediction method of the present application;
FIG. 5 is a schematic block diagram of a temporal motion vector of an obtained sub-block according to an embodiment of the temporal motion vector prediction method of the present application;
FIG. 6 is a schematic block diagram of a temporal motion vector of an obtained sub-block according to an embodiment of the temporal motion vector prediction method of the present application;
FIG. 7 is a schematic diagram of a frame in which an obtaining sub-block cannot obtain a valid temporal motion vector according to an embodiment of the temporal motion vector prediction method of the present application;
FIG. 8 is a flow chart of yet another embodiment of the temporal motion vector prediction method of the present application;
FIG. 9 is a schematic block diagram illustrating an embodiment of a temporal motion vector predictor of the present application;
FIG. 10 is a schematic block diagram of a temporal motion vector predictor of another embodiment of the present application;
FIG. 11 is a schematic diagram of a frame of an embodiment of an electronic device of the present application;
FIG. 12 is a schematic diagram of a frame of an embodiment of a computer readable storage medium of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, an embodiment of the present application provides a temporal motion vector prediction method, which includes the following steps:
S11: when the time domain motion vector prediction is performed based on the sub-blocks, the current block is segmented, and a plurality of sub-blocks with the same size are obtained.
If the current prediction mode is skip/direct inter prediction mode, and the first side and the second side of the current block are both greater than or equal to 16 pixels, a preset starting condition of time domain motion vector prediction based on the sub-block is satisfied. At this time, temporal motion vector prediction subTMVP based on the sub-block may be started), and the current block is partitioned to obtain several sub-blocks with the same size.
In some embodiments, the current block is directly subjected to cross segmentation to obtain a plurality of sub-blocks with the same size, and the side length of each sub-block is larger than a predetermined value, specifically, the predetermined value is 4, and in other embodiments, the predetermined value can be other values.
In still other embodiments, partitioning the current block to obtain a number of sub-blocks of the same size includes: dividing the current block according to the size of the current block to obtain a plurality of sub-blocks with the same size and side length larger than a preset value. The segmentation mode is flexibly selected according to the current block size, and the prediction accuracy is effectively improved under the condition of increasing the control complexity.
The method specifically comprises the following schemes:
A. determining a segmentation mode by using the aspect ratio:
The lengths of a first side and a second side of the current block are compared, the first side and the second side being adjacent. In response to the length ratio of the first side edge to the second side edge being greater than or equal to a first predetermined multiple r 0, the first predetermined multiple being greater than one (r 0 > 1), dividing the current block equally in a direction perpendicular to the first side edge into n 0(n0 > 2) equal-sized sub-blocks equally; or in response to the length ratio of the second side edge to the first side edge being greater than or equal to a second predetermined multiple r 1, the second predetermined multiple being greater than a r 1(r1 > 1), dividing the current block equally in a direction perpendicular to the second side edge into n 1(n1 > 2) equal-sized sub-blocks equally; in response to the first side being equal to the second side, the current block is divided equally in a cross division, which may be divided multiple times. The side length of the obtained sub-block size is not less than a predetermined value, otherwise the division is not performed.
The following description is made in connection with several embodiments:
In one embodiment, if the length of the first side is greater than that of the second side, the ratio of the first side to the second side is r 0, and the current block is divided into n 0 sub-blocks along the direction perpendicular to the first side; if the length of the second side is greater than that of the first side, the ratio of the second side to the first side is r 1, and the current block is divided into n 1 sub-blocks along the direction perpendicular to the second side.
In one embodiment, the ratio r 0 of the first side to the second side, or the ratio r 1 of the second side to the first side, is divided into different intervals from small to large, and the values of n 0 or n 1 within each interval are consistent. For example, when r 0 is less than or equal to 2 and less than or equal to 4, n 0 is equal to 2; when r 0 is more than or equal to 8 and less than or equal to 16, n 0 is equal to 8; let n 0 be equal to 4 when 16< r 0. n 1 are set up similarly.
B. determining a segmentation mode by using the side length of the current block:
And acquiring the lengths of a first side edge and a second side edge of the current block, wherein the first side edge and the second side edge are adjacent. In response to the length w 0 of the first side being at least twice the predetermined value, equally dividing the current block in a direction perpendicular to the first side into n 3 equally sized sub-blocks; and/or, in response to the length h 0 of the second side being at least twice the predetermined value, dividing the current block equally in a direction perpendicular to the second side into n 4 equally sized sub-blocks. After division, the side length of the sub-blocks is ensured to be more than or equal to a preset value, otherwise, the division is not carried out.
The following description is made in connection with several embodiments:
In one embodiment, the lengths w 0 of the different first sides correspond to different values of n 3, and the lengths h 0 of the different second sides correspond to different values of n 4. For example, the greater the length w 0 of the first side edge, the greater n 3; the greater the length of the second side h 0, the greater n 4.
In a specific embodiment, the length w 0 of the first side or the length h 0 of the second side is divided into different sections from small to large, and the values of n 3 or n 4 in each section are consistent.
Of course, the split mode may also be determined by combining the split modes of a and B while considering the ratio of the first side and the second side of the current block.
In some embodiments, prior to step S11, it is further required that:
And judging whether the current prediction mode and the current block meet preset starting conditions for carrying out time domain motion vector prediction based on the sub-blocks.
If yes, performing time domain motion vector prediction based on the sub-blocks.
The judging whether the current prediction mode and the current block meet the preset enabling condition for performing time domain motion vector prediction based on the sub-block comprises: judging whether the current prediction mode is skip/direct inter prediction mode, and whether the length of the first side edge and the second side edge of the current block is larger than or equal to a preset value, wherein at least one of the first side edge and the second side edge is larger than or equal to twice of the preset value, and the first side edge and the second side edge are adjacent. In some embodiments, the first side is the width of the current block, and the second side is the height of the current block, specifically, the predetermined value is 4 pixels, in other embodiments, the predetermined value may be other values, and only needs to be matched with the minimum side length required by the sub-block, which is not limited herein. Compared with the prior art, the method has the advantages that the size of the current block can be smaller compared with the method that the first side and the second side of the current block are larger than or equal to 16 pixels, and the side length of the sub-block obtained after segmentation is larger than or equal to 4, so that the application is more flexible.
It should be noted that, before determining whether the preset enabling condition is satisfied, a MVP (temporal prediction motion vector) candidate list under skip/direct needs to be generally constructed. The list includes several types of MVP modes, TMVP (temporal MVP), SMVP (spatial MVP), HMVP (historical MVP) according to the priority order. The temporal motion vector prediction method of the present application relates to TMVP, which is a temporal motion vector corresponding to the upper left corner coordinate of the current block, that is MV (motion vector).
In the present application, when traversing to TMVP in candidate list and satisfying the starting condition of time domain motion vector prediction (subTMVP) based on sub-block, subTMVP is started, and after starting, prediction is performed by subTMVP mode instead of directly performing prediction by TMVP.
The following complements the acquisition method of the current block TMVP:
step1, acquiring the space domain position of the time domain homonymy block corresponding to the current block.
Firstly, determining the airspace position of the current block corresponding to the time domain same-position block, wherein the airspace position of the current same-position block is only related to the current block coordinate and is irrelevant to the current block size.
Let bx, by be the position coordinates of the current block in scu (4*4-sized pixel block) in the whole frame, and determine the spatial position of the co-located block by using a mask: mask= (-1)/(3). "≡" represents bitwise exclusive OR, where after-1 and 3 are bitwise exclusive OR, the last two bits of mask are 0. Next, coordinates (in scu) of the spatial position of the bit block are set as (xpos, ypos):
xpos＝(bx&mask)+2；
ypos＝(by&mask)+2；
The range that xpos may take is: bx-1, bx, bx+1, bx+2; the range that ypos may take is: by-1, by, by+1, by+2. The current block with the same coordinate position corresponds to only one time domain co-located block, the range of the time domain co-located block corresponding to the current block with different coordinates is shown in fig. 2, the positions of all the dots in fig. 2 are the coordinates of all possible co-located blocks, and each small square represents a scu.
Step2, scale the temporal co-located block MV.
If the current frame is a P frame, going to a forward co-located reference frame, namely finding a time domain co-located block in an index0 frame of a forward reference frame list, and taking a forward MV of the block: mv_col_f performs scaling. Scaling according to POC distance t1 between current frame and index0 reference frame in forward reference frame list of current frame and distance t0 between the co-located frame (index 0) of forward co-located block and forward reference frame of co-located block: scaleMV = mv_col_f/t0 t1. The MVP candidate list is filled as TMVP after rounding scaleMV to the signed 16bit inner operation.
If the current frame is a B frame, the time domain co-located blocks MV in two directions need to be acquired. Firstly, going to the backward co-located reference frame, namely finding a time domain co-located block in an index0 frame of a backward reference frame list, and taking a forward MV of the block: and performing twice scaling on the MV_col_f to obtain MV values in two directions of the TMVP. The scaling of the forward MV value is performed according to the POC distance t1 between the current frame and the reference frame of index0 in the forward reference frame list of the current frame, and the distance t0 between the co-located frame (generally also index 0) to which the backward co-located block belongs and the forward reference frame of the co-located block; the value scaling of the backward MV is then scaled according to the POC distance t1 between the current frame and the reference frame of index0 in the backward reference frame list of the current frame, and the distance t0 between the co-located frame (also index0 in general) to which the backward co-located block belongs and the forward reference frame of the co-located block. The scaling formula and subsequent operations are the same as those when P frames.
Step3, when MV of the time domain co-located block is not available.
In addition to the above normal cases, cases may occur where MVs of the temporal co-located block are not available, such as cases where the temporal co-located block is an intra block or IBC block (intra copy block).
The judgment mode is to judge whether the forward reference frame of the time domain homonymy block exists or not. If not, the specific operation is as follows:
If the current frame is a P frame, TMVP is directly set to 0.
If the current frame is a B frame, the first position of the MVP list is filled with TMVP using the weighted MVs of the spatial neighboring blocks as the first position of the MVP candidate list. The rules are as follows:
1) First, it is determined whether or not adjacent blocks of the spatial location are available, as shown in fig. 3:
firstly judging whether the solid line blocks a, b and c are available, if the solid line block c is unavailable, taking the broken line block at the upper left corner as the block c, and judging whether the broken line block c is available.
2) If only 1 spatial block is available in the three spatial blocks, the MVs of the spatial blocks are scaled and then filled in the first position of the MVP candidate list. Scaling is the same as above.
3) If only 2 airspace blocks are available in the three airspace blocks, firstly scaling MVs of the airspace blocks, and then performing the following operations:
i. Subtracting MVs of every two airspace blocks, taking absolute values, respectively representing MVs of three airspace positions by mva, mvb, mvc, wherein unavailable MVs are 0,
mva_ext＝abs(mva-mvb)；
mvb_ext＝abs(mvb-mvc)；
mvc_ext＝abs(mvc-mva)；
Acquiring the minimum value of the three difference values mva _ext, mvb_ext and mvc_ext, and filling (mva +mvb)/2 in the first position of the MVP candidate list if the minimum value is mva _ext; if the minimum value is mvb_ext, filling mvb+mvc)/2; if the minimum value is mvc_ext, (mvc+ mva)/2 is filled in.
4) If MVs of three spatial blocks are available, the filling rule is as follows after scaling the spatial MVs:
a) If there are at least one and two other different directions (one positive two negative or one negative two positive) in the three MVs at the current component (x or y):
i. If (mva <0& mvb >0& mvc > 0) | (mva >0& mvb <0& mvc < 0), then (mvb+mvc)/2 is filled;
ii if (mvb <0& & mva >0& & mvc > 0) | (mvb >0& mva <0 &mvc < 0), filling in (mvc+ mva)/2);
Filling (mva +mvb)/2) if (mvc <0& mva >0& &mvb > 0) | (mvc >0& mva <0 &mvb < 0);
b) If three MVs are all equidirectional:
The procedure is the same as (3).
5) If none are available, 0 is filled.
Since the B frame is used here, the above operation requires a judgment and an arithmetic operation for each of the forward and backward MVs of the spatial block.
In some embodiments, before the current block is segmented, a termination condition may be further added to the start of subTMVP technologies, as shown in fig. 4, including:
S10: acquiring two first time domain motion vectors corresponding to a first group of pixel blocks at the diagonal position of the current block and two second time domain motion vectors corresponding to a second group of pixel blocks at the diagonal position of the current block;
s101: it is determined whether the two first temporal motion vectors are identical and/or whether the two second temporal motion vectors are identical.
S102: if yes, the current block is not segmented.
S103: if not, the current block is segmented.
If not, step S11 is continued.
If yes, the current block should be a smaller block, and the MVs of the whole block are likely to be consistent, so that the temporal motion vector prediction (subTMVP) technology based on the sub-block is not needed, which is equivalent to adding a termination condition for starting the subTMVP technology, and complexity can be reduced under the condition of ensuring the prediction performance.
The following description is made in connection with one embodiment:
as shown in fig. 5, W0< first side length < W1, and H0< second side length < H1 of the current block, the first side may be the width of the current block, and the second side is the height of the current block.
I. when the time domain motion vectors of the upper left corner scu1 and the lower right corner scu4 in the current block are the same, subTMVP is not adopted;
II. When the time domain motion vectors of the upper right corner scu2 and the lower left corner scu3 in the current block are the same, subTMVP is not adopted;
III, when the time domain motion vectors of the upper left corner scu1 and the lower right corner scu4 in the current block are the same, and the time domain motion vectors of the upper right corner scu2 and the lower left corner scu3 are the same, subTMVP is not adopted;
IV, the temporal motion vectors of the upper left corner scu1 and the lower right corner scu4 in the current block are the same, or subTMVP is not used when the temporal motion vectors of the upper right corner scu2 and the lower left corner scu3 are the same.
When the temporal motion vectors of the diagonal directions scu of the current block are identical, it is likely that the motion vectors representing the entire block are identical, and therefore subTMVP technology is not required.
S12: and acquiring a temporal motion vector corresponding to the pixel block at any position in each sub-block, and determining the temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block.
In one embodiment, the corresponding temporal motion vector is obtained using the pixel block in each sub-block at the top left, bottom left, top right, bottom right or center point position to determine the temporal motion vector for each sub-block. The obtained temporal motion vector of the pixel block also needs to be scaled, and the scaled temporal motion vector can be used as the temporal motion vector of the sub-block. It should be noted that, in an embodiment of determining the temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block, the method of scaling the temporal co-located block MV is the same as step2 in step S11. Of course, in other embodiments, other pixel blocks may be selected for each sub-block, and are not limited in this regard. Compared with the prior art, only the pixel blocks with fixed corners of each sub-block can be selected, the method has the advantages that the positions of the sub-blocks for acquiring the time domain motion vectors are more flexible, and the positions of the different sub-blocks for acquiring the time domain motion vectors can be different, so that the accuracy of acquiring the time domain motion vectors by the sub-blocks is improved, and the accuracy of predicting the time domain motion vectors is further improved.
Specifically, as shown in fig. 6, the current block is divided into four sub-blocks by a cross, and the position of the upper left sub-block 1 for selecting and obtaining the time domain motion vector is the lower right corner scu1 in the current sub-block; the position of the upper right sub-block 2 for selecting and acquiring the time domain motion vector is the lower left corner scu2 in the current sub-block; the position of the left lower sub-block 3 for selecting and acquiring the time domain motion vector is the upper right corner scu3 in the current sub-block; the lower right sub-block 4 selects the position where the temporal motion vector is acquired as the upper left corner scu4 in the current sub-block.
In an embodiment, the current sub-block may not obtain a valid temporal motion vector, e.g., a valid temporal motion vector may not be obtained, or the obtained temporal motion vector is not valid outside the image. In the prior art, a time domain motion vector of a current block is directly used as a time domain motion vector of a current sub-block, and after the current sub-block cannot acquire an effective time domain motion vector in the method, the effective time domain motion vector of the current sub-block can be acquired by the following method:
Further comprises:
a: a valid temporal motion vector for the current sub-block is determined based on the temporal motion vector for the pixel block at another location of the current sub-block. Specifically, the other pixel blocks in the current sub-block may be searched according to a certain order, for example, the pixel blocks at four corners of the sub-block are searched first, and then the pixel block at the center point is searched, or the search is directly performed according to the raster scanning order. Until a pixel block is searched, a valid temporal motion vector is obtained to determine the temporal motion vector of the current sub-block to be filled. The time domain motion vectors of the adjacent sub-blocks before the searching sequence are filled with priority, and the searching is not performed after the filling is finished. If the effective temporal motion vector is not obtained, the temporal motion vector of the current block is used as the temporal motion vector of the current sub-block.
B: and acquiring the effective time domain motion vectors of the rest sub-blocks in the current block in a first order, and taking the effective time domain motion vectors of the rest sub-blocks with the previous acquisition order as the time domain motion vectors of the current sub-block. Specifically, the selection of the adjacent sub-blocks may define a first order, e.g., clockwise, counterclockwise, left to right, right to left, top to bottom, bottom to top, etc. And searching the adjacent sub-blocks according to the searching sequence until the effective time domain motion vector is searched to be used as the time domain motion vector of the current sub-block to be filled. The time domain motion vectors of the adjacent sub-blocks before the searching sequence are filled with priority, and the searching is not performed after the filling is finished. If the effective temporal motion vector is not obtained, the temporal motion vector of the current block is used as the temporal motion vector of the current sub-block.
C: the effective time domain motion vectors of the first number of the rest sub-blocks in the current block are obtained in a second sequence or at will and are weighted and averaged to be used as the effective time domain motion vectors of the current sub-block. Wherein the first number n is greater than or equal to 2, specifically, n (n > =2) sub-blocks in the block are selected according to the second order or optionally, and weighted average of the time domain MVs is performed to obtain the MVs of the current sub-block. If the effective temporal motion vector is not obtained, the temporal motion vector of the current block is used as the temporal motion vector of the current sub-block.
The following description is made in connection with one embodiment:
As shown in fig. 7, among the current 4 sub-blocks, sub-block 4 does not acquire a valid MV, and at this time, scheme b is selected, and a clockwise search is performed starting from sub-block 3. The sub-block 3 also has no valid temporal motion vector, and the sub-block 1 has a valid temporal motion vector, so that the temporal motion vector of the sub-block 1 is used as the temporal motion vector of the sub-block 4, and the sub-block 2 is not searched.
The method has the advantages that the time domain motion vectors of the sub-blocks are more flexibly obtained, the time domain motion vectors of the adjacent sub-blocks can be used for filling when the effective motion vectors are not obtained, the correlation is stronger, and the obtained time domain motion vectors of the sub-blocks are more accurate.
S13: and respectively performing motion compensation and distortion cost calculation on the time domain motion vectors of all the sub-blocks of the current block.
In an embodiment, the method further includes performing motion compensation and distortion cost calculation on time domain motion vectors of all sub-blocks of the current block respectively, instead of performing motion compensation and distortion cost calculation by using the original TMVP mode, so as to obtain a more accurate calculation result.
As shown in fig. 8, a further embodiment of the present method provides a temporal motion vector prediction method, which includes the steps of:
S21: when the time domain motion vector prediction is performed based on the sub-blocks, the current block is divided according to the size of the current block, and a plurality of sub-blocks with the same size and side length larger than a preset value are obtained.
In an embodiment, the current block is divided to obtain a plurality of sub-blocks with the same size, including: dividing the current block according to the size of the current block to obtain a plurality of sub-blocks with the same size and side length larger than a preset value. Compared with the prior art, the method has the advantages that the current block is directly subjected to cross segmentation, the segmentation mode is flexibly selected according to the size of the current block, and the prediction accuracy can be effectively improved under the condition that the control complexity is increased.
The rest of the step S21 is substantially the same as the corresponding step S11 in the above embodiment, and will not be repeated here.
S22: a temporal motion vector is obtained for each sub-block.
In an embodiment, obtaining the temporal motion vector for each sub-block includes: and acquiring a temporal motion vector corresponding to the pixel block at any position of each sub-block, and taking the temporal motion vector corresponding to the pixel block as the temporal motion vector of each sub-block. The specific content of step S22 is substantially the same as that of step S22 in the above embodiment, and will not be described here again.
In other embodiments, a pixel block with a fixed corner of each sub-block may also be selected, and the temporal motion vector of the pixel block may be obtained and used as the temporal motion vector of the sub-block.
S23: and respectively performing motion compensation and distortion cost calculation on the time domain motion vectors of all the sub-blocks of the current block.
In an embodiment, the method further includes performing motion compensation and distortion cost calculation on time domain motion vectors of all sub-blocks of the current block respectively, instead of performing motion compensation and distortion cost calculation by using the original TMVP mode, so as to obtain a more accurate calculation result.
Referring to fig. 9, a temporal motion vector predictor 30 according to another embodiment of the present application includes a current block segmentation module 31 and an acquisition module 32 to implement the temporal motion vector predictor method according to the above-mentioned corresponding embodiment. Specifically, when performing temporal motion vector prediction based on the sub-blocks, the current block segmentation module 31 segments the current block to obtain a plurality of sub-blocks with the same size. The obtaining module 32 obtains a temporal motion vector corresponding to a pixel block at any position in each sub-block, and determines the temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block. The position of the time domain motion vector acquired by the sub-block of the device 30 is more flexible, and the positions of the time domain motion vectors acquired by different sub-blocks can be different, so that the accuracy of the time domain motion vector acquired by the sub-block is improved, and the accuracy of the time domain motion vector prediction is further improved.
Referring to fig. 10, a temporal motion vector predictor 40 according to another embodiment of the present application includes a current block segmentation module 41 and an acquisition module 42 to implement the temporal motion vector predictor method according to the above-mentioned corresponding embodiment. Specifically, when performing temporal motion vector prediction based on the sub-blocks, the current block segmentation module 41 divides the current block according to the size of the current block to obtain a plurality of sub-blocks having the same size and a side length greater than a predetermined value. The acquisition module 42 acquires a temporal motion vector for each sub-block. The device 40 flexibly selects the segmentation mode according to the current block size, and can effectively improve the prediction accuracy under the condition of increasing the control complexity.
Referring to fig. 11, a further embodiment of the present application provides an electronic device 50, including a memory 51 and a processor 52 coupled to each other, where the processor 52 is configured to execute program instructions stored in the memory 51 to implement the temporal motion vector prediction method of any one of the above embodiments. In one particular implementation scenario, electronic device 50 may include, but is not limited to: the microcomputer and the server, and the electronic device 50 may also include a mobile device such as a notebook computer and a tablet computer, which is not limited herein.
In particular, the processor 52 is configured to control itself and the memory 51 to implement the frame reference method of any of the embodiments described above. The processor 52 may also be referred to as a CPU (Central Processing Unit ). The processor 52 may be an integrated circuit chip having signal processing capabilities. The Processor 52 may also be a general purpose Processor, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), an Application SPECIFIC INTEGRATED Circuit (ASIC), a Field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 52 may be commonly implemented by an integrated circuit chip.
Referring to fig. 12, a further embodiment of the present application provides a computer readable storage medium 60 having program data 61 stored thereon, the program data 61, when executed by a processor, implementing the temporal motion vector prediction method of any one of the above embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical, or other forms.
The elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over network elements. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium 60. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium 60, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the method of the embodiments of the present application. And the aforementioned storage medium 60 includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing description is only illustrative of the present application and is not intended to limit the scope of the application, and all equivalent structures or equivalent processes or direct or indirect application in other related technical fields are included in the scope of the present application.
Claims (11)
1. A method of temporal motion vector prediction, the method comprising:
when time domain motion vector prediction is carried out based on the sub-blocks, the current block is segmented, and a plurality of sub-blocks with the same size are obtained;
Acquiring a temporal motion vector corresponding to a pixel block at any position in each sub-block, and determining the temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block;
The obtaining the temporal motion vector corresponding to the pixel block at any position in each sub-block, determining the temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block, includes:
And acquiring a corresponding temporal motion vector by using the pixel blocks at the positions of the upper left corner, the lower left corner, the upper right corner, the lower right corner or the central point of each sub-block so as to determine the temporal motion vector of each sub-block.
2. The method of claim 1, wherein all sides of the block of pixels are four pixels in length.
3. The method as recited in claim 1, wherein the method further comprises:
Responding to the fact that a current sub-block cannot acquire a valid time domain motion vector;
Determining a valid temporal motion vector for the current sub-block based on a temporal motion vector for a pixel block at another location in the current sub-block; or alternatively
Acquiring effective time domain motion vectors of the rest sub-blocks in the current block in a first order, and taking the effective time domain motion vectors of the rest sub-blocks with the previous acquisition order as the time domain motion vectors of the current sub-block; or alternatively
And acquiring the effective time domain motion vectors of the first number of the rest sub-blocks in the current block in a second sequence or at will, and taking the weighted average as the effective time domain motion vectors of the current sub-block.
4. The method of claim 1, wherein the partitioning the current block to obtain a plurality of sub-blocks with the same size comprises:
Dividing the current block according to the size of the current block to obtain a plurality of sub-blocks with the same size and side length larger than a preset value.
5. The method according to claim 4, wherein said dividing the current block according to the size of the current block comprises:
Comparing lengths of a first side and a second side of the current block, the first side being adjacent to the second side;
in response to a length ratio of the first side edge to the second side edge being greater than or equal to a first predetermined multiple, equally dividing the current block in a direction perpendicular to the first side edge, the first predetermined multiple being greater than one; or alternatively
In response to the length ratio of the second side edge to the first side edge being greater than or equal to a second predetermined multiple, performing average division on the current block in a direction perpendicular to the second side edge, the second predetermined multiple being greater than one; or alternatively
And in response to the first side edge being equal to the second side edge, carrying out average division on the current block by a cross division method.
6. The method according to claim 4, wherein said dividing the current block according to the size of the current block comprises:
acquiring lengths of a first side and a second side of the current block, wherein the first side is adjacent to the second side;
in response to the length of the first side being at least twice the predetermined value, equally dividing the current block in a direction perpendicular to the first side; and/or the number of the groups of groups,
And in response to the length of the second side being at least twice the predetermined value, equally dividing the current block in a direction perpendicular to the second side.
7. The method of claim 1, further comprising, prior to said partitioning the current block:
acquiring two first time domain motion vectors corresponding to a first group of pixel blocks at the diagonal position of the current block and two second time domain motion vectors corresponding to a second group of pixel blocks at the diagonal position of the current block;
Judging whether the two first time domain motion vectors are identical or not, and/or judging whether the two second time domain motion vectors are identical or not;
If yes, the current block is not segmented.
8. The method according to claim 4, wherein the performing temporal motion vector prediction based on the sub-blocks comprises:
Judging whether the current prediction mode is skip/direct inter prediction mode, and whether the lengths of a first side edge and a second side edge of the current block are larger than or equal to the preset value, wherein at least one side edge is larger than or equal to twice of the preset value, and the first side edge is adjacent to the second side edge;
if yes, performing time domain motion vector prediction based on the sub-blocks.
9. The method as recited in claim 1, wherein the method further comprises:
And performing motion compensation and distortion cost calculation on the time domain motion vectors of all the sub-blocks of the current block.
10. An electronic device comprising a memory and a processor coupled to each other, the processor configured to execute program instructions stored in the memory to implement the prediction method of any one of claims 1 to 9.
11. A computer readable storage medium having stored thereon program data, wherein the program data, when executed by a processor, implements the prediction method of any of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010784588.XA CN112055207B (en) | 2020-08-06 | 2020-08-06 | Time domain motion vector prediction method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010784588.XA CN112055207B (en) | 2020-08-06 | 2020-08-06 | Time domain motion vector prediction method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112055207A CN112055207A (en) | 2020-12-08 |
| CN112055207B true CN112055207B (en) | 2024-05-31 |
Family
ID=73601488
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010784588.XA Active CN112055207B (en) | 2020-08-06 | 2020-08-06 | Time domain motion vector prediction method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112055207B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060059770A (en) * | 2004-11-29 | 2006-06-02 | 엘지전자 주식회사 | Method and apparatus for deriving motion vectors of macro blocks from motion vectors of pictures of base layer when encoding/decoding video signal |
| CN109089119A (en) * | 2017-06-13 | 2018-12-25 | 浙江大学 | A kind of method and apparatus of motion-vector prediction |
| WO2019109955A1 (en) * | 2017-12-08 | 2019-06-13 | 华为技术有限公司 | Interframe prediction method and apparatus, and terminal device |
| CN110545424A (en) * | 2019-08-21 | 2019-12-06 | 浙江大华技术股份有限公司 | Inter-frame prediction method based on MMVD (multimedia MediaVision video) mode, video coding method, related device and equipment |
| CN110876282A (en) * | 2018-07-02 | 2020-03-10 | 华为技术有限公司 | Motion vector prediction method and related device |
| CN111316639A (en) * | 2018-04-09 | 2020-06-19 | 腾讯美国有限责任公司 | Method and apparatus for sub-block motion vector prediction |
| WO2020151274A1 (en) * | 2019-01-23 | 2020-07-30 | 华为技术有限公司 | Image display order determination method and apparatus, and video coding/decoding device |
-
2020
- 2020-08-06 CN CN202010784588.XA patent/CN112055207B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060059770A (en) * | 2004-11-29 | 2006-06-02 | 엘지전자 주식회사 | Method and apparatus for deriving motion vectors of macro blocks from motion vectors of pictures of base layer when encoding/decoding video signal |
| CN109089119A (en) * | 2017-06-13 | 2018-12-25 | 浙江大学 | A kind of method and apparatus of motion-vector prediction |
| WO2019109955A1 (en) * | 2017-12-08 | 2019-06-13 | 华为技术有限公司 | Interframe prediction method and apparatus, and terminal device |
| CN111316639A (en) * | 2018-04-09 | 2020-06-19 | 腾讯美国有限责任公司 | Method and apparatus for sub-block motion vector prediction |
| CN110876282A (en) * | 2018-07-02 | 2020-03-10 | 华为技术有限公司 | Motion vector prediction method and related device |
| WO2020151274A1 (en) * | 2019-01-23 | 2020-07-30 | 华为技术有限公司 | Image display order determination method and apparatus, and video coding/decoding device |
| CN110545424A (en) * | 2019-08-21 | 2019-12-06 | 浙江大华技术股份有限公司 | Inter-frame prediction method based on MMVD (multimedia MediaVision video) mode, video coding method, related device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112055207A (en) | 2020-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11166040B2 (en) | Video signal processing method and apparatus using adaptive motion vector resolution | |
| CN110557640B (en) | Weighted interleaved prediction | |
| US10728571B2 (en) | Inter prediction method and device in video coding system | |
| CN110521205B (en) | Video coding and decoding method and device and related computer readable medium | |
| US12058358B2 (en) | Video signal processing method and apparatus using adaptive motion vector resolution | |
| CN110249628B (en) | Video encoder and decoder for predictive partitioning | |
| US20230239489A1 (en) | Limited memory access window for motion vector refinement | |
| KR20200064153A (en) | Video decoding method and apparatus according to inter prediction in video coding system | |
| JP2019519995A (en) | Video Coding Using Adaptive Motion Information Improvement | |
| CN111567047B (en) | Memory access window for sub-prediction block motion vector derivation | |
| JP2019519998A (en) | Method and apparatus for video coding with automatic refinement of motion information | |
| CN115002456A (en) | Image encoding and decoding method and image decoding apparatus | |
| US10893289B2 (en) | Affine motion prediction-based image decoding method and device using affine merge candidate list in image coding system | |
| CN110740327A (en) | Motion compensation of overlapping blocks | |
| CN111818342B (en) | Inter-frame prediction method and prediction device | |
| US20190191178A1 (en) | Inter prediction method and apparatus in video coding system | |
| CN112292861B (en) | Sub-pixel accurate correction method based on error surface for decoding end motion vector correction | |
| CN112055208A (en) | Video coding method, video coding equipment and storage device | |
| CN112055207B (en) | Time domain motion vector prediction method, device and storage medium | |
| WO2020049447A1 (en) | Fast encoding methods for interweaved prediction | |
| US20210409754A1 (en) | Search region for motion vector refinement | |
| CN110557639B (en) | Application of interleaved prediction | |
| US20240291967A1 (en) | Method, apparatus, and medium for video processing | |
| CN117941356A (en) | Video signal encoding/decoding method and recording medium storing bit stream | |
| CN118044190A (en) | Video signal encoding/decoding method and recording medium storing bit stream |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |