US20220030249A1 - Image encoding/decoding method and device - Google Patents
Image encoding/decoding method and device Download PDFInfo
- Publication number
- US20220030249A1 US20220030249A1 US17/492,779 US202117492779A US2022030249A1 US 20220030249 A1 US20220030249 A1 US 20220030249A1 US 202117492779 A US202117492779 A US 202117492779A US 2022030249 A1 US2022030249 A1 US 2022030249A1
- Authority
- US
- United States
- Prior art keywords
- block
- sub
- prediction
- current block
- inter prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 229
- 230000033001 locomotion Effects 0.000 claims abstract description 265
- 239000013598 vector Substances 0.000 claims abstract description 168
- 238000010586 diagram Methods 0.000 description 54
- 230000008569 process Effects 0.000 description 48
- 238000000638 solvent extraction Methods 0.000 description 43
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 30
- 238000004422 calculation algorithm Methods 0.000 description 18
- 239000000470 constituent Substances 0.000 description 18
- 238000001914 filtration Methods 0.000 description 17
- 238000009795 derivation Methods 0.000 description 16
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 11
- 238000007796 conventional method Methods 0.000 description 11
- 238000013139 quantization Methods 0.000 description 11
- 238000005192 partition Methods 0.000 description 8
- 238000012937 correction Methods 0.000 description 7
- 230000009466 transformation Effects 0.000 description 7
- 241000023320 Luma <angiosperm> Species 0.000 description 5
- 230000000903 blocking effect Effects 0.000 description 5
- 238000006073 displacement reaction Methods 0.000 description 5
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 5
- 230000003044 adaptive effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
- H04N19/45—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder performing compensation of the inverse transform mismatch, e.g. Inverse Discrete Cosine Transform [IDCT] mismatch
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
- H04N19/463—Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
- H04N19/517—Processing of motion vectors by encoding
- H04N19/52—Processing of motion vectors by encoding by predictive encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/537—Motion estimation other than block-based
- H04N19/54—Motion estimation other than block-based using feature points or meshes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/56—Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/567—Motion estimation based on rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/583—Motion compensation with overlapping blocks
Definitions
- the present invention relates to an image signal encoding/decoding method and device. More particularly, the present invention relates to an image encoding/decoding method using inter prediction and an image encoding/decoding device using inter prediction.
- VCEG Video Coding Expert Group
- MPEG Moving Picture Expert Group
- HEVC High Efficiency Video Coding
- HEVC uses a variety of technologies such as intra prediction, inter prediction, transform, quantization, entropy encoding, and in-loop filtering.
- inter prediction of HEVC new technologies such as block merging, advanced motion vector prediction (AMVP) have been applied such that efficient inter prediction is possible.
- AMVP advanced motion vector prediction
- the block is partitioned into small parts, so that rapid increase in overhead may occur and encoding efficiency may be lowered.
- the present invention has been made keeping in mind the above problems, and the present invention is intended to enhance efficiency of inter prediction by providing improved inter prediction.
- the present invention is intended to provide a motion vector derivation method by an image decoding device, where an image encoding device does not need to transmit motion vector information to the image decoding device.
- the present invention is intended to provide a motion vector derivation method of a control point by an image decoding device, wherein in affine inter prediction, an image encoding device does not need to transmit a motion vector of the control point to the image decoding device.
- the present invention is intended to provide inter prediction capable of efficient encoding or decoding when multiple motions are present in one block.
- the present invention is intended to reduce blocking artifacts that may occur when one block is partitioned into multiple regions and encoding or decoding is performed using different types of inter prediction.
- the present invention is intended to enhance efficiency of inter prediction by partitioning a current block to be encoded or decoded, on the basis of a partitioning structure of reconstructed neighboring blocks.
- the present invention is intended to enhance efficiency of inter prediction by partitioning, on the basis of a partitioning structure of reconstructed neighboring blocks, a pre-reconstructed neighboring image region which is used to encode or decode a current block.
- the present invention is intended to enhance efficiency of image encoding or decoding by performing encoding or decoding using a current block or a neighboring image partitioned as described above.
- a reconstructed pixel region within an image to which a current block to be decoded belongs is selected; on the basis of the reconstructed pixel region and a reference image of the current block, a motion vector of the reconstructed pixel region is derived; and the derived motion vector is selected as a motion vector of the current block.
- the reconstructed pixel region may include at least one among a region adjacent to an upper side of the current block and a region adjacent to a left side of the current block.
- the motion vector of the reconstructed pixel region may be derived on the basis of a position of a region corresponding to the reconstructed pixel region, wherein the region corresponding to the reconstructed pixel region is determined within the reference image.
- a reconstructed pixel region within an image to which a current block to be encoded belongs is selected; on the basis of the reconstructed pixel region and a reference image of the current block, a motion vector of the reconstructed pixel region is derived; and the derived motion vector is selected as a motion vector of the current block.
- decoder-side motion vector derivation indication information may be generated and encoded.
- the decoder-side motion vector derivation indication information may indicate whether or not the derived motion vector of the reconstructed pixel region is selected as the motion vector of the current block.
- At least one reconstructed pixel region is selected within an image to which a current block to be decoded using affine inter prediction belongs; on the basis of the at least one reconstructed pixel region and a reference image of the current block, a motion vector of the at least one reconstructed pixel region is derived; and the derived motion vector of the at least one reconstructed pixel region is selected as a motion vector of at least one control point of the current block.
- the at least one reconstructed pixel region may be a region adjacent to the at least one control point of the current block.
- the at least one control point may be positioned at an upper left side, an upper right side, or a lower left side of the current block.
- the motion vector of the control point positioned at a lower right side of the current block may be decoded on the basis of motion information included in a bitstream.
- decoder-side control point motion vector derivation indication information may be decoded.
- the motion vector of the at least one reconstructed pixel region may be derived on the basis of the decoder-side control point motion vector derivation indication information.
- At least one reconstructed pixel region is selected within an image to which a current block to be encoded using affine inter prediction belongs; on the basis of the at least one reconstructed pixel region and a reference image of the current block, a motion vector of the at least one reconstructed pixel region is derived; and the derived motion vector of the at least one reconstructed pixel region is selected as a motion vector of at least one control point of the current block.
- a current block to be decoded is partitioned into multiple regions including a first region and a second region; and a prediction block of the first region and a prediction block of the second region are obtained, wherein the prediction block of the first region and the prediction block of the second region are obtained by different inter prediction methods.
- the first region may be a region adjacent to a reconstructed image region within an image to which the current block belongs
- the second region may be a region that is not in contact with the reconstructed image region within the image to which the current block belongs.
- a motion vector of the first region may be estimated.
- a region positioned at a boundary as a region within the prediction block of the first region or a region positioned at a boundary as a region within the prediction block of the second region may be partitioned into multiple sub-blocks; motion information of a neighboring sub-block of a first sub-block, which is one of the multiple sub-blocks, may be used to generate a prediction block of the first sub-block; and the first sub-block and the prediction block of the first sub-block may be subjected to a weighted sum, so that a prediction block of the first sub-block to which the weighted sum is applied may be obtained.
- a current block to be encoded is partitioned into multiple regions including a first region and a second region; a prediction block of the first region and a prediction block of the second region are obtained, wherein the prediction block of the first region and the prediction block of the second region are obtained by different inter prediction methods.
- the first region may be a region adjacent to a pre-encoded reconstructed image region within an image to which the current block belongs, and the second region may be a region that is not in contact with the pre-encoded reconstructed image region within the image to which the current block belongs.
- a motion vector of the first region may be estimated.
- a region positioned at a boundary as a region within the prediction block of the first region or a region positioned at a boundary as a region within the prediction block of the second region may be partitioned into multiple sub-blocks; motion information of a neighboring sub-block of a first sub-block, which is one of the multiple sub-blocks, may be used to generate a prediction block of the first sub-block; and the first sub-block and the prediction block of the first sub-block may be subjected to a weighted sum, so that a prediction block of the first sub-block to which the weighted sum is applied may be obtained.
- the current block is partitioned into multiple sub-blocks, and the multiple sub-blocks of the current block are decoded.
- the current block on the basis of a partitioning structure of neighboring blocks of the current block, the current block may be partitioned into the multiple sub-blocks.
- the current block on the basis of at least one among the number of the neighboring blocks, a size of the neighboring blocks, a shape of the neighboring blocks, and a boundary between the neighboring blocks, the current block may be partitioned into the multiple sub-blocks.
- a pre-reconstructed pixel region may be partitioned on a per-sub-block basis, and at least one of the multiple sub-blocks of the current block may be decoded using at least one sub-block included in the reconstructed pixel region.
- the reconstructed pixel region may be partitioned on a per-sub-block basis.
- the reconstructed pixel region may be partitioned on a per-sub-block basis.
- the current block on the basis of blocks around a current block to be encoded, the current block may be partitioned into multiple sub-blocks, and the multiple sub-blocks of the current block may be encoded.
- the current block on the basis of a partitioning structure of neighboring blocks of the current block, the current block may be partitioned into the multiple sub-blocks.
- the current block on the basis of the at least one among the number of the neighboring blocks, a size of the neighboring blocks, a shape of the neighboring blocks, and a boundary between the neighboring blocks, the current block may be partitioned into the multiple sub-blocks.
- a pre-reconstructed pixel region may be partitioned on a per-sub-block basis, and at least one of the multiple sub-blocks of the current block may be encoded using at least one sub-block included in the reconstructed pixel region.
- the reconstructed pixel region may be partitioned on a per-sub-block basis.
- the reconstructed pixel region may be partitioned on a per-sub-block basis.
- the amount of encoding information generated as a result of encoding a video may be reduced and thus encoding efficiency may be enhanced. Also, by adaptively decoding an encoded image, reconstruction efficiency of an image may be enhanced and the quality of the reproduced image may be improved.
- the image encoding device does not need to transmit motion vector information to the image decoding device, so that the amount of encoding information may be reduced and thus encoding efficiency may be enhanced.
- blocking artifacts may be reduced that may occur when one block is partitioned into multiple regions and encoding or decoding is performed using different types of inter prediction.
- FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present invention.
- FIG. 2 is a diagram illustrating a method of generating motion information by using motion estimation according to the conventional technology.
- FIG. 3 is a diagram illustrating an example of neighboring blocks that may be used to generate motion information of a current block.
- FIG. 4 is a block diagram illustrating an image decoding device according to an embodiment of the present invention.
- FIGS. 5 a and 5 b are diagrams illustrating inter prediction using a reconstructed pixel region according to a first exemplary embodiment of the present invention.
- FIG. 6 is a flowchart illustrating an inter prediction method according to the first exemplary embodiment of the present invention.
- FIGS. 7 a to 7 c are diagrams illustrating examples of reconstructed pixel regions.
- FIG. 8 is a flowchart illustrating a process of determining an inter prediction method according to an embodiment of the present invention.
- FIG. 9 is a diagram illustrating a process of encoding information that indicates an inter prediction method.
- FIG. 10 is a diagram illustrating a process of decoding DMVD indication information encoded as shown in FIG. 9 .
- FIG. 11 is a diagram illustrating affine inter prediction.
- FIGS. 12 a and 12 b are diagrams illustrating derivation of a motion vector of a control point by using a reconstructed pixel region according to a second exemplary embodiment of the present invention.
- FIG. 13 is a flowchart illustrating an inter prediction method according to the second exemplary embodiment of the present invention.
- FIG. 14 is a flowchart illustrating a process of determining an inter prediction method according to the second exemplary embodiment of the present invention.
- FIG. 15 is a diagram illustrating a process of encoding information that indicates an inter prediction method determined by the process shown in FIG. 14 .
- FIG. 16 is a diagram illustrating a process of decoding DCMVD indication information encoded as shown in FIG. 15 .
- FIG. 17 is a flowchart illustrating an example of an image decoding method in which motion vectors of three control points are derived using a reconstructed pixel region so as to generate a prediction block of a current block.
- FIG. 18 is a diagram illustrating a current block partitioned into multiple regions for inter prediction according to a third exemplary embodiment of the present invention.
- FIG. 19 is a flowchart illustrating an inter prediction method according to the third exemplary embodiment of the present invention.
- FIG. 20 is a diagram illustrating an example of motion estimation and motion compensation using a reconstructed pixel region.
- FIG. 21 is a flowchart illustrating a process of determining an inter prediction method according to an embodiment of the present invention.
- FIG. 22 is a diagram illustrating a process of transmitting, to an image decoding device, information that indicates an inter prediction method.
- FIG. 23 is a diagram illustrating a process of decoding information that indicates which type of inter prediction has been used.
- FIG. 24 is a flowchart illustrating a method of generating an inter prediction block by using information that indicates which type of inter prediction has been used.
- FIG. 25 is a diagram illustrating a method of reducing blocking artifacts according to a fourth exemplary embodiment of the present invention.
- FIG. 26 is a diagram illustrating a method of applying a weighted sum of a sub-block within a prediction block and a sub-block adjacent to the upper side thereof.
- FIG. 27 is a diagram illustrating a method of applying a weighted sum of a sub-block within a prediction block and a sub-block adjacent to the left side thereof.
- FIG. 28 is a flowchart illustrating a process of determining whether or not a weighted sum is applied between sub-blocks.
- FIG. 29 is a flowchart illustrating a process of encoding information that indicates whether or not a weighted sum is applied between sub-blocks.
- FIG. 30 is a flowchart illustrating a process of decoding information that indicates whether or not a weighted sum is applied between sub-blocks.
- FIGS. 31 a and 31 b are diagrams illustrating inter prediction using a reconstructed pixel region according to a fifth exemplary embodiment of the present invention.
- FIG. 32 is a diagram illustrating an example of a case where motion estimation is further performed on a current block on a per-sub-block basis.
- FIG. 33 is a diagram illustrating an example in which a reconstructed pixel region and a current block are partitioned on a per-sub-block basis.
- FIG. 34 is a flowchart illustrating an example of an inter prediction method using a reconstructed pixel region.
- FIG. 35 is a diagram illustrating an example in which reconstructed blocks neighboring a current block are used to partition a reconstructed pixel region into sub-blocks according to the present invention.
- FIG. 36 is a diagram illustrating an example in which reconstructed blocks neighboring a current block are used to partition a current block into multiple sub-blocks according to the present invention.
- FIG. 37 is a flowchart illustrating a method of partitioning a current block into multiple sub-blocks according to an embodiment of the present invention.
- FIG. 38 is a flowchart illustrating a method of partitioning a reconstructed region used to encode or decode a current block into multiple sub-blocks according to an embodiment of the present invention.
- FIG. 39 is a flowchart illustrating an example of an inter prediction method using the sub-blocks of the current block partitioned as shown in FIG. 36 .
- FIG. 40 is a flowchart illustrating a method of encoding information determined according to inter prediction shown in FIG. 39 .
- FIG. 41 is a flowchart illustrating an example of a method of decoding information encoded by the encoding method shown in FIG. 40 .
- FIGS. 42 a and 42 b are diagrams illustrating a sixth exemplary embodiment of the present invention.
- FIG. 43 is a flowchart illustrating an example of a method of determining an inter prediction mode according to the sixth exemplary embodiment of the present invention described with reference to FIGS. 42 a and 42 b.
- FIG. 44 is a diagram illustrating a process of encoding information determined by the method shown in FIG. 43 .
- FIG. 45 is a diagram illustrating a process of decoding information encoded by the method shown in FIG. 44 .
- first”, “second”, etc. can be used to describe various elements, but the elements are not to be construed as being limited to the terms. The terms are only used to differentiate one element from other elements.
- first element may be named the “second” element without departing from the scope of the present invention, and similarly the “second” element may also be named the “first” element.
- the term “and/or” includes a combination of a plurality of items or any one of a plurality of terms.
- FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present invention.
- an image encoding device 100 may include an image partitioning module 101 , an intra prediction module 102 , an inter prediction module 103 , a subtractor 104 , a transform module 105 , a quantization module 106 , an entropy encoding module 107 , a dequantization module 108 , an inverse transform module 109 , an adder 110 , a filter module 111 , and a memory 112 .
- each constituent shown in FIG. 1 are independently shown so as to represent different distinctive functions in the image encoding device, which does not mean that each constituent is constituted as separated hardware or a single software constituent unit.
- each constituent includes each of enumerated constituents for convenience.
- at least two constituents of each constituent may be combined to form one constituent or one constituent may be divided into a plurality of constituents to perform each function.
- the embodiment where each constituent is combined and the embodiment where one constituent is divided are also included in the scope of the present invention, if not departing from the essence of the present invention.
- constituents may not be indispensable constituents performing essential functions of the present invention but be selective constituents improving only performance thereof.
- the present invention may be implemented by including only the indispensable constituents for implementing the essence of the present invention except the constituents used in improving performance.
- the structure including only the indispensable constituents except the selective constituents used in improving only performance is also included in the scope of the present invention.
- the image partitioning module 101 may partition an input image into one or more blocks.
- the input image may have various shapes and sizes, such as a picture, a slice, a tile, a segment, and the like.
- a block may mean a coding unit (CU), a prediction unit (PU), or a transform unit (TU).
- the partitioning may be performed on the basis of at least one among a quadtree and a binary tree.
- Quadtree partitioning is a method of partitioning a parent block into four child blocks of which the width and the height are half of those of the parent block.
- Binary tree partitioning is a method of partitioning a parent block into two child blocks of which either the width or the height is half of that of the parent block.
- the coding unit may mean a unit of performing encoding or a unit of performing decoding.
- the prediction modules 102 and 103 may include the intra prediction module 102 performing intra prediction and the inter prediction module 103 performing inter prediction.
- Whether to perform inter prediction or intra prediction on the prediction unit may be determined, and detailed information (for example, an intra prediction mode, a motion vector, a reference picture, and the like) depending on each prediction method may be determined.
- a processing unit on which prediction is performed may be different from a processing unit in which the prediction method and the detailed content are determined.
- the prediction method, the prediction mode, and the like may be determined on a per-prediction unit basis, and prediction may be performed on a per-transform unit basis.
- a residual value (residual block) between the generated prediction block and an original block may be input to the transform module 105 .
- prediction mode information used for prediction, motion vector information, and the like may be encoded with the residual value by the entropy encoding module 107 and then may be transmitted to a decoder.
- the original block is intactly encoded and transmitted to the decoding module without generating a prediction block by the prediction module 102 , 103 .
- the intra prediction module 102 may generate a prediction block on the basis of information on a reference pixel around the current block, which is information on a pixel within a current picture.
- a prediction mode of the neighboring block of the current block on which intra prediction is to be performed is inter prediction
- a reference pixel included in the neighboring block to which inter prediction has been applied is replaced by a reference pixel within another neighboring block to which intra prediction has been applied.
- Prediction modes in intra prediction may include a directional prediction mode using the information on the reference pixel depending on a prediction direction and a non-directional prediction mode not using directivity information in performing prediction.
- a mode for predicting luma information may be different from a mode for predicting chroma information, and in order to predict the chroma information, intra prediction mode information used to predict the luma information or predicted luma signal information may be utilized.
- the intra prediction module 102 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation module, and a DC filter.
- the AIS filter is a filter performing filtering on a reference pixel of the current block, and may adaptively determine whether to apply the filter depending on a prediction mode of a current prediction unit. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter is not applied.
- the reference pixel interpolation module of the intra prediction module 102 interpolates the reference pixel to generate a reference pixel at a position on a per-fraction basis.
- the prediction mode of the current prediction unit is a prediction mode in which the prediction block is generated without interpolating the reference pixel, the reference pixel is not interpolated.
- the DC filter generates the prediction block through filtering when the prediction mode of the current block is a DC mode.
- the inter prediction module 103 generates the prediction block using a pre-reconstructed reference image stored in the memory 112 and motion information.
- the motion information may contain, for example, a motion vector, a reference picture index, a list 1 prediction flag, a list 0 prediction flag, and the like.
- the first method is a method in which motion information (a motion vector, a reference image index, an inter prediction direction, and the like) is generated using a motion estimation process.
- FIG. 2 is a diagram illustrating a method of generating motion information by using motion estimation according to the conventional technology.
- Motion estimation is a method of generating motion information such as, a motion vector, a reference image index, and an inter prediction direction of a current image region to be encoded using the decoded reference image after being encoded. It is possible that motion estimation is performed in the entire reference image or that in order to reduce the complexity, a search range is set and motion estimation is performed only within the search range.
- the second method of generating the motion information is a method in which motion information of neighboring blocks of the current image block to be encoded is used.
- FIG. 3 is a diagram illustrating an example of neighboring blocks that may be used to generate motion information of a current block.
- FIG. 3 shows an example of spatial candidate blocks A to E and a temporal candidate block COL as neighboring blocks that may be used to generate the motion information of the current block.
- the spatial candidate blocks A to E are present in the same image as the current block, but the temporal candidate block COL is present in an image that is different from the image to which the current block belongs.
- One piece of motion information of the spatial candidate blocks A to E and the temporal candidate block COL, which neighbor the current block, may be selected as the motion information of the present block.
- an index may be defined that indicates which block has the motion information used as the motion information of the current block. This index information also belongs to the motion information.
- the motion information may be generated and the prediction block may be generated through motion compensation.
- a residual block may be generated that includes residual value information which is a difference value between the prediction unit generated by the prediction module 102 , 103 and the original block of the prediction unit.
- the generated residual block may be input to the transform module 105 for transform.
- the inter prediction module 103 may derive the prediction block on the basis of information on at least one picture among the previous picture and the subsequent picture of the current picture. Further, the prediction block of the current block may be derived on the basis of information on a partial region with encoding completed within the current picture.
- the inter prediction module 103 may include a reference picture interpolation module, a motion prediction module, and a motion compensation module.
- the reference picture interpolation module may receive reference picture information from the memory and may generate information on a pixel equal to or smaller than an integer pixel in the reference picture.
- a DCT-based 8-tap interpolation filter having different filter coefficients may be used to generate information on a pixel equal to or smaller than an integer pixel on a per-1 ⁇ 4 pixel basis.
- a DCT-based 4-tap interpolation filter having different filter coefficients may be used to generate information on a pixel equal to or smaller than an integer pixel on a per-1 ⁇ 8 pixel basis.
- the motion prediction module may perform motion prediction on the basis of the reference picture interpolated by the reference picture interpolation module.
- various methods such as a full search-based block matching algorithm (FBMA), a three step search (TSS) algorithm, a new three-step search (NTS) algorithm, and the like, may be used.
- the motion vector may have a motion vector value on a per-1 ⁇ 2 or 1 ⁇ 4 pixel basis on the basis of the interpolated pixel.
- the motion prediction module may predict the prediction block of the current block by using different motion prediction methods.
- various methods such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, and the like, may be used.
- AMVP advanced motion vector prediction
- the subtractor 104 performs subtraction on the block to be currently encoded and on the prediction block generated by the intra prediction module 102 or the inter prediction module 103 so as to generate the residual block of the current block.
- the transform module 105 may transform the residual block containing residual data, using a transform method, such as DCT, DST, Karhunen-Loeve transform (KLT), and the like.
- the transform method may be determined on the basis of the intra prediction mode of the prediction unit that is used to generate the residual block. For example, depending on the intra prediction mode, DCT may be used in the horizontal direction, and DST may be used in the vertical direction.
- the quantization module 106 may quantize values transformed into a frequency domain by the transform module 105 . Quantization coefficients may vary according to a block or importance of an image. The value calculated by the quantization module 106 may be provided to the dequantization module 108 and the entropy encoding module 107 .
- the transform module 105 and/or the quantization module 106 may be selectively included in the image encoding device 100 . That is, the image encoding device 100 may perform at least one of transform and quantization on residual data of the residual block, or may encode the residual block by skipping both transform and quantization. Even though the image encoding device 100 does not perform either transform or quantization or does not perform both transform and quantization, the block that is input to the entropy encoding module 107 is generally referred to as a transform block.
- the entropy encoding module 107 entropy encodes the input data. Entropy encoding may use various encoding methods, for example, exponential Golomb coding, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).
- the entropy encoding module 107 may encode a variety of information, such as residual value coefficient information of a coding unit, block type information, prediction mode information, partitioning unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, and the like, from the prediction module 102 , 103 .
- the coefficient of the transform block may be encoded on a per-partial block basis within the transform block on the basis of various types of flags indicating a non-zero coefficient, a coefficient of which the absolute value is higher than one or two, the sign of the coefficient, and the like may be encoded.
- a coefficient that is not encoded only with the flags may be encoded through the absolute value of the difference between the coefficient encoded through the flag and the actual coefficient of the transform block.
- the dequantization module 108 dequantizes the values quantized by the quantization module 106 , and the inverse transform module 109 inversely transforms the values transformed by the transform module 105 .
- the residual value generated by the dequantization module 108 and the inverse transform module 109 may be combined with the prediction unit predicted through a motion estimation module included in the prediction module 102 , 103 , the motion compensation module, and the intra prediction module 102 such that a reconstructed block is generated.
- the adder 110 adds the prediction block generated by the prediction module 102 , 103 and the residual block generated by the inverse transform module 109 so as to generate a reconstructed block.
- the filter module 111 may include at least one of a deblocking filter, an offset correction module, and an adaptive loop filter (ALF).
- a deblocking filter may include at least one of a deblocking filter, an offset correction module, and an adaptive loop filter (ALF).
- ALF adaptive loop filter
- the deblocking filter may remove block distortion that occurs due to boundaries between the blocks in the reconstructed picture.
- whether to apply the deblocking filter to the current block may be determined on the basis of the pixels included in several rows and columns in the block.
- a strong filter or a weak filter is applied depending on required deblocking filtering strength.
- horizontal direction filtering and vertical direction filtering are configured to be processed in parallel.
- the offset correction module may correct an offset from the original image on a per-pixel basis with respect to the image subjected to deblocking.
- Adaptive loop filtering may be performed on the basis of the value obtained by comparing the filtered reconstructed image and the original image.
- the pixels included in the image may be divided into predetermined groups, a filter to be applied to each of the groups may be determined, and filtering may be individually performed on each group.
- Information on whether to apply ALF of a luma signal may be transmitted for each coding unit (CU), and the shape and the filter coefficient of the ALF filter to be applied may vary depending on each block. Also, regardless of the characteristic of the application target block, the ALF filter in the same form (fixed form) may be applied.
- the memory 112 may store the reconstructed block or picture calculated through the filter module 111 , and the stored reconstructed block or picture may be provided to the prediction module 102 , 103 when performing inter prediction.
- FIG. 4 is a block diagram illustrating an image decoding device 400 according to an embodiment of the present invention.
- the image decoding device 400 may include an entropy decoding module 401 , a dequantization module 402 , an inverse transform module 403 , an adder 404 , a filter module 405 , a memory 406 , and prediction modules 407 and 408 .
- the input bitstream is decoded according to a reverse process of the process performed in the image encoding device 100 .
- the entropy decoding module 401 may perform entropy decoding according to the reverse procedure of the entropy encoding performed by the entropy encoding module 107 of the image encoding device 100 . For example, corresponding to the methods performed by the image encoder, various methods, such as exponential Golomb coding, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC), may be applied.
- various methods such as exponential Golomb coding, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC) may be applied.
- the coefficient of the transform block may be encoded on a per-partial block basis within the transform block on the basis of various types of flags indicating a non-zero coefficient, a coefficient of which the absolute value is higher than one or two, the sign of the coefficient, and the like.
- a coefficient that is not represented only by the flags may be decoded through the sum of a coefficient represented through the flag and a signaled coefficient.
- the entropy decoding module 401 may decode information related to intra prediction and inter prediction performed in the encoder.
- the dequantization module 402 performs dequantization on the quantized transform block to generate the transform block. This operates substantially in the same manner as the dequantization module 108 in FIG. 1 .
- the inverse transform module 403 performs inverse transform on the transform block to generate the residual block.
- the transform method may be determined on the basis of the prediction method (inter or intra prediction), the size and/or the shape of the block, information on the intra prediction mode, and the like. This operates substantially in the same manner as the inverse transform module 109 in FIG. 1 .
- the adder 404 adds the prediction block generated by the intra prediction module 407 or the inter prediction module 408 and the residual block generated by the inverse transform module 403 so as to generate a reconstructed block. This operates substantially in the same manner as the adder 110 in FIG. 1 .
- the filter module 405 reduces various types of noises occurring in the reconstructed blocks.
- the filter module 405 may include a deblocking filter, an offset correction module, and an ALF.
- From the image encoding device 100 it is possible to receive information on whether or not the deblocking filter is applied to the block or picture and information on whether the strong filter is applied or the weak filter is applied when the deblocking filter is applied.
- the deblocking filter of the image decoding device 400 may receive information related to the deblocking filter from the image encoding device 100 , and the image decoding device 400 may perform deblocking filtering on the block.
- the offset correction module may perform offset correction on the reconstructed image on the basis of the type of offset correction, offset value information, and the like applied to the image in performing encoding.
- the ALF may be applied to the coding unit on the basis of information on whether to apply the ALF, ALF coefficient information, and the like received from the image encoding device 100 .
- the ALF information may be provided by being included in a particular parameter set.
- the filter module 405 operates substantially in the same manner as the filter module 111 in FIG. 1 .
- the memory 406 stores the reconstructed block generated by the adder 404 . This operates substantially in the same manner as the memory 112 in FIG. 1 .
- the prediction module 407 , 408 may generate a prediction block on the basis of information related to prediction block generated provided from the entropy decoding module 401 and of information on the previously decoded block or picture provided from the memory 406 .
- the prediction modules 407 and 408 may include the intra prediction module 407 and the inter prediction module 408 . Although not shown, the prediction module 407 , 408 may further include a prediction unit determination module.
- the prediction unit determination module may receive a variety of information input from the entropy decoding module 401 , such as prediction unit information, prediction mode information of an intra prediction method, information related to motion prediction of an inter prediction method, and the like, may separate a prediction unit in a current coding unit, and may determine whether inter prediction is performed on the prediction unit or intra prediction is performed on the prediction unit.
- the inter prediction module 408 may perform inter prediction on the current prediction unit on the basis of information included at least one picture among the previous picture and the subsequent picture of the current picture including the current prediction unit. Alternatively, inter prediction may be performed on the basis of information on some pre-reconstructed regions within the current picture including the current prediction unit.
- the coding unit In order to perform inter prediction, on the basis of the coding unit, it may be determined which mode among a skip mode, a merge mode, and an AMVP Mode is used for the motion prediction method of the prediction unit included in the coding unit.
- the intra prediction module 407 generates the prediction block using the pre-reconstructed pixels positioned near the block to be currently encoded.
- the intra prediction module 407 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation module, and a DC filter.
- the AIS filter is a filter performing filtering on the reference pixel of the current block, and may adaptively determine whether to apply the filter depending on the prediction mode of the current prediction unit.
- the prediction mode of the prediction unit provided from the image encoding device 100 and the AIS filter information may be used to perform AIS filtering on the reference pixel of the current block. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter is not applied.
- the reference pixel interpolation module of the intra prediction module 407 interpolates the reference pixel to generate a reference pixel at a position on a per-fraction basis.
- the generated reference pixel on a per-fraction basis may be used as a prediction pixel of a pixel within the current block.
- the prediction mode of the current prediction unit is a prediction mode in which a prediction block is generated without interpolating the reference pixel, the reference pixel is not interpolated.
- the DC filter may generate a prediction block through filtering when the prediction mode of the current block is a DC mode.
- the intra prediction module 407 operates substantially in the same manner as the intra prediction module 102 in FIG. 1 .
- the inter prediction module 408 generates an inter prediction block using a reference picture stored in the memory 406 and motion information.
- the inter prediction module 408 operates substantially in the same manner as the inter prediction module 103 in FIG. 1 .
- FIGS. 5 a and 5 b are diagrams illustrating inter prediction using a reconstructed pixel region according to a first exemplary embodiment of the present invention.
- the motion vector of the current block may be derived using the reconstructed pixel region.
- FIG. 5 a shows a current block 51 to be encoded or decoded and a pre-reconstructed pixel region C 52 as a region adjacent to the current block 51 .
- the current block 51 and the reconstructed pixel region C 52 are included in the current image 50 .
- the current image 50 may be a picture, a slice, a tile, a coding tree block, a coding block, or other image regions.
- the reconstructed pixel region C 52 may correspond to a reconstructed pixel region after being encoded before encoding of the current block 51 in terms of encoding, and may be a pre-reconstructed region before decoding of the current block 51 in terms of decoding.
- the reconstructed pixel region C 52 neighbors the current block 51 , and thus the image encoding device 100 and the image decoding device 400 may use the same reconstructed pixel region C 52 . Therefore, without encoding the motion information of the current block 51 by the image encoding device 100 , the reconstructed pixel region C 52 is used such that the image encoding device 100 and the image decoding device 400 may generate the motion information of the current block 51 and the prediction block in the same manner.
- FIG. 5 b shows an example of motion estimation and motion compensation using a reconstructed pixel region.
- a reference image 53 shown in FIG. 5 b is searched for a region matched with the reconstructed pixel region C 52 shown in FIG. 5 a .
- a reconstructed pixel region D 54 that is most similar to the reconstructed pixel region C 52 is determined, a displacement between a region 56 , which is at the same position as the reconstructed pixel region C 52 , and the reconstructed pixel region D 54 is determined to be a motion vector 57 of the reconstructed pixel region C 52 .
- the motion vector 57 determined as described above is selected as the motion vector of the current block 51 , and a prediction block 58 of the current block 51 may be derived using the motion vector 57 .
- FIG. 6 is a flowchart illustrating an inter prediction method according to the first exemplary embodiment of the present invention.
- Inter prediction may be performed by the inter prediction module 103 of the image encoding device 100 or by the inter prediction module 408 of the image decoding device 400 .
- Reference images used in inter prediction are stored in the memory 112 of the image encoding device 100 or in the memory 406 of the image decoding device 400 .
- the inter prediction module 103 or the inter prediction module 408 may generate the prediction block of the current block 51 with reference to the reference image stored in the memory 112 or in the memory 406 .
- the reconstructed pixel region 52 is selected to be used in deriving of the motion vector of the current block to be encoded or decoded, at step S 61 .
- the motion vector of the reconstructed pixel region 52 is derived at step S 63 .
- the reference image 53 shown in FIG. 5 b is searched for the region matched with the reconstructed pixel region C 52 .
- the reconstructed pixel region D 54 most similar to the reconstructed pixel region C 52 is determined, the displacement between the region 56 at the same position as the reconstructed pixel region C 52 and the reconstructed pixel region D 54 is determined to be the motion vector 57 of the reconstructed pixel region C 52 .
- the image encoding device 100 or the image decoding device 400 selects the motion vector 57 of the reconstructed pixel region C 52 , determined as described above, as the motion vector of the current block 51 at step S 65 . Using this motion vector 57 , the prediction block 58 of the current block 51 may be generated.
- FIGS. 7 a to 7 c are diagrams illustrating examples of reconstructed pixel regions.
- the letters M, N, O, and P shown in FIGS. 7 a to 7 c denotes pixel intervals, respectively, and it is possible that O and P have negative values assuming that the absolute values of O and P are lower than the horizontal or vertical lengths of the current block, respectively.
- the reconstructed pixel regions at the upper side and the left side of the current block are used as the reconstructed pixel region C 52 or that the two regions are combined into a single piece to be used as the reconstructed pixel region C 52 .
- the reconstructed pixel region C 52 is used by being subjected to subsampling. In this method, only the decoded information around the current block is used to derive the motion information, and thus it is not necessary to transmit the motion information from the encoding device 100 to the decoding device 400 .
- the decoding device 400 also performs motion estimation, so that if motion estimation is performed on the entire reference image, the complexity may extremely increase. Therefore, by transmitting the search range on a per-block basis or in the parent header or by fixing the search region to be the same in the encoding device 100 and in the decoding device 400 , computational complexity of the decoding device 400 may be reduced.
- FIG. 8 is a flowchart illustrating a process of determining an inter prediction method according to an embodiment of the present invention.
- the optimum method may be determined through rate-distortion optimization (RDO).
- RDO rate-distortion optimization
- the process shown in FIG. 8 may be performed by the image encoding device 100 .
- inter prediction according to the conventional method is performed to compute cost_A at step S 81 , and then as described above, inter prediction using the reconstructed pixel region according to the present invention is performed to compute cost_B at step S 82 .
- cost_A is compared with cost_B to determine which method is optimum to use, at step S 83 .
- cost_A is lower, it is set to perform inter prediction using the conventional method at step S 84 . Otherwise, it is set to perform inter prediction using the reconstructed pixel region at step S 85 .
- FIG. 9 is a diagram illustrating a process of encoding information that indicates an inter prediction method determined by the process shown in FIG. 8 .
- the information indicating the inter prediction method determined by the process shown in FIG. 8 is referred to as decoder-side motion vector derivation (DMVD) indication information.
- the DMVD indication information or decoder-side motion vector derivation indication information may be information indicating whether inter prediction using the conventional method is performed or inter prediction using the reconstructed pixel region is performed.
- the DMVD indication information indicating the inter prediction method determined by the process shown in FIG. 8 is encoded at step S 91 .
- the DMVD indication information may be, for example, a 1-bit flag or one of several indexes.
- the motion information is encoded at step S 92 , and the algorithm ends.
- information indicating whether or not inter prediction using the reconstructed pixel region according to the embodiment of the present invention is used may be generated in the parent header first and then may be decoded. That is, when the information indicating whether or not inter prediction using the reconstructed pixel region is used indicates true, the DMVD indication information is encoded. When the information indicating whether or not inter prediction using the reconstructed pixel region is used indicates false, the DMVD indication information is not present within the bitstream and in this case, the current block is predicted using the conventional inter prediction.
- the parent header including the information that indicates whether or not inter prediction using the reconstructed pixel region is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
- FIG. 10 is a diagram illustrating a process of decoding the DMVD indication information encoded as shown in FIG. 9 .
- the decoding device 400 decodes the DMVD indication information at step S 101 , decodes the motion information at step S 102 , and ends the algorithm.
- the DMVD indication information is present in the bitstream.
- the DMVD indication information is not present within the bitstream and in this case, the current block is predicted using the conventional inter prediction.
- the parent header including the information that indicates whether or not inter prediction using the reconstructed pixel region is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
- the inter prediction using the reconstructed pixel region according to the first exemplary embodiment described above is applied to inter prediction using affine transformation.
- a motion vector derivation method using the reconstructed pixel region is applied.
- inter prediction using affine transformation is simply referred to as affine inter prediction.
- FIG. 11 is a diagram illustrating affine inter prediction.
- affine inter prediction motion vectors at four corners of the current block to be encoded or decoded are obtained, and then the motion vectors are used to generate a prediction block.
- the four corners of the current block may correspond to the control points.
- a block identified by motion vectors 11 - 2 , 11 - 3 , 11 - 4 , and 11 - 5 at the four corners (namely, the control points) of the current block (not shown) within the current image may be a prediction block 11 - 6 of the current block.
- This affine inter prediction enables prediction of a block or image region subjected to rotation, zoom-in/zoom-out, translation, reflection, or shear deformation.
- Equation 1 is a general determinant of affine transformation.
- Equation 1 is an equation representing transform of two-dimensional coordinates, wherein (x, y) denotes original coordinates, (x′, y′) denotes destination coordinates, and a, b, c, d, e, and f denote transform parameters.
- Equation 2 represents a method of deriving a motion vector of an arbitrary sub-block within the current block by using two control points at the top left and the top right of the current block.
- Equation 2 (x, y) denotes the position of the arbitrary sub-block within the current block, W denotes the horizontal length of the current block, (MV x , MV y ) denotes the motion vector of the sub-block, (MV 0x , MV 0y ) denotes the motion vector of the top left control point, and (MV 1x , MV 1y ) denotes the motion vector of the top right control point.
- Equation 3 represents a method of deriving a motion vector of an arbitrary sub-block within the current block by using three control points at the top left, the top right, and the bottom left of the current block.
- Equation 3 (x, y) denotes the position of the arbitrary sub-block, W and H denote the horizontal length and the vertical length of the current block, respectively, (MV x , MV y ) denotes the motion vector of the sub-block within the current block, (MV 0x , MV 0y ) denotes the motion vector of the top left control point, (MV 1x , MV 1y ) denotes the motion vector of the top right control point, and (MV 2x , MV 2y ) denotes the motion vector of the bottom left control point.
- the motion vector derivation method using the reconstructed pixel region according to the first exemplary embodiment is applied. Therefore, the image encoding device 100 does not need to transmit motion vector information of multiple control points to the image decoding device 400 .
- FIGS. 12 a and 12 b are diagrams illustrating derivation of the motion vector of the control point by using the reconstructed pixel region according to the second exemplary embodiment of the present invention.
- a current block 12 a - 2 to be encoded or decoded is included in a current image 12 a - 1 .
- Four control points for affine inter prediction are denoted by circles at four corners of the current block 12 a - 2 in FIG. 12 a .
- regions adjacent to three control points at the top left, the top right, and the bottom left pre-reconstructed pixel regions a 12 a - 3 , b 12 a - 4 , and c 12 a - 5 are shown.
- motion vectors of three control points at the top left, the top right, and the bottom left are derived using the reconstructed pixel regions a 12 a - 3 , b 12 a - 4 , and c 12 a - 5 as shown in FIG. 12 b .
- a reconstructed pixel region may be not present nearby.
- the motion vector of the sub-block d 12 a - 6 obtained using the conventional inter prediction method may be set to be the motion vector of the bottom right control point of the current block.
- a reference image 12 b - 1 shown in FIG. 12 b is searched for regions d 12 b - 10 , e 12 b - 11 , and f 12 b - 12 matched with the reconstructed pixel regions a 12 a - 3 , b 12 a - 4 , and c 12 a - 5 , respectively, shown in FIG. 12 a .
- the displacements from the regions 12 b - 6 , 12 b - 7 , and 12 b - 8 which are at the same position as the reconstructed pixel regions a 12 a - 3 , b 12 a - 4 , and c 12 a - 5 , respectively, are determined to be motion vectors 12 b - 2 , 12 b - 3 , and 12 b - 4 of the reconstructed pixel regions a 12 a - 3 , b 12 a - 4 , and c 12 a - 5 , respectively.
- the motion vectors 12 b - 2 , 12 b - 3 , and 12 b - 4 determined as described above are determined to be the motion vectors of three control points at the top left, the top right, and the bottom left of the current block 12 a - 2 .
- the motion vector of the control point at the bottom right the motion vector of the sub-block d 12 a - 6 obtained using the conventional inter prediction method may be used.
- a motion vector of an arbitrary sub-block within the current block may be derived as shown in Equation 4 below.
- Equation 4 (x, y) denotes the position of the arbitrary sub-block within the current block, W and H denote the horizontal length and the vertical length of the current block, respectively, (MV x , MV y ) denotes the motion vector of the sub-block within the current block, (MV 0x , MV 0y ) denotes the motion vector of the top left control point, (MV 1x , MV 1y ) denotes the motion vector of the top right control point, (MV 2x , MV 2y ) denotes the motion vector of the bottom left control point, and (MV 3x , MV 3y ) denotes the motion vector of the bottom right control point.
- the reconstructed pixel regions a 12 a - 3 , b 12 a - 4 , and c 12 a - 5 may be in various sizes and/or shapes as described above with reference to FIGS. 7 a to 7 c .
- the size and/or the shape of the sub-block d 12 a - 6 may be the same as a preset size and/or a preset shape in the encoding device 100 and the decoding device 400 .
- horizontal and/or vertical size information of the sub-block d 12 a - 6 may be transmitted, or size information may be transmitted on a per-exponentiation of two basis.
- motion vectors are derived from four control points, these vectors are used to derive the motion vector of the current block 12 a - 2 or the motion vector of an arbitrary sub-block within the current block 12 a - 2 , and this derived motion vector may be used to derive the prediction block of the current block 12 a - 2 or the prediction block of an arbitrary sub-block within the current block 12 a - 2 .
- the position of the current block 12 a - 2 is coordinates (0, 0), so that the motion vector of the current block 12 a - 2 is the motion vector (MV 0x , MV 0y ) of the top left control point.
- the prediction block of the current block 12 a - 2 may be obtained using the motion vector of the top left control point.
- the motion vector of the sub-block at the position (3,0) within the current block is obtained by substituting a value of three for the variable x in Equation 4 above, a value of zero for the variable y, and a value of eight for both variables W and H.
- FIG. 13 is a flowchart illustrating the inter prediction method according to the second exemplary embodiment of the present invention.
- Inter prediction may be performed by the inter prediction module 103 of the image encoding device 100 or the inter prediction module 408 of the image decoding device 400 .
- Reference images used in inter prediction are stored in the memory 112 of the image encoding device 100 or in the memory 406 of the image decoding device 400 .
- the inter prediction module 103 or the inter prediction module 408 may generate the prediction block of the current block 51 with reference to a reference image stored in the memory 112 or the memory 406 .
- At least one reconstructed pixel region is selected to be used in deriving a motion vector of at least one control point of the current block to be encoded or decoded, at step S 131 .
- a motion vector of at least one control point of the current block to be encoded or decoded at step S 131 .
- three reconstructed pixel regions a 12 a - 3 , b 12 a - 4 , and c 12 a - 5 are selected.
- one or two reconstructed pixel regions may be selected.
- a motion vector of at least one reconstructed pixel region is derived at step S 133 .
- the image encoding device 100 or the image decoding device 400 selects each motion vector of the reconstructed pixel region C 52 , determined as described above, as a motion vector of at least one control point of the current block at step S 135 . At least one motion vector selected as described above may be used to generate the prediction block of the current block.
- FIG. 14 is a flowchart illustrating a process of determining an inter prediction method according to the second exemplary embodiment of the present invention.
- the optimum method may be determined through rate-distortion optimization (RDO).
- RDO rate-distortion optimization
- the process shown in FIG. 14 may be performed by the image encoding device 100 .
- inter prediction is performed using the conventional method to compute cost_A at step S 141 , and as described above, according to the second exemplary embodiment of the present invention, affine inter prediction is performed to compute cost_B at step S 142 .
- cost_A is compared with cost_B to determine which method is optimum to use, at step S 143 .
- cost_A is lower, it is set to perform inter prediction using the conventional method at step S 144 . Otherwise, it is set to perform affine inter prediction at step S 145 according to the second exemplary embodiment of the present invention.
- FIG. 15 is a diagram illustrating a process of encoding information that indicates an inter prediction method determined by the process shown in FIG. 14 .
- the information indicating the inter prediction method determined by the process shown in FIG. 14 is referred to as decoder-side control point motion vector derivation (DCMVD) indication information.
- the DCMVD indication information or decoder-side control point motion vector derivation indication information may be information indicating whether inter prediction using the conventional method is performed or affine inter prediction according to the second exemplary embodiment of the present invention is performed.
- the DCMVD indication information indicating the inter prediction method determined by the process shown in FIG. 14 is encoded at step S 151 .
- the DCMVD indication information may be, for example, a 1-bit flag or one of several indexes.
- the motion information is encoded at step S 152 , and the algorithm ends.
- the information indicating whether or not affine inter prediction is used may be generated in the parent header first and then may be encoded. That is, according to the second exemplary embodiment of the present invention, when the information indicating whether or not affine inter prediction is used indicates true, the DCMVD indication information is encoded. According to the second exemplary embodiment of the present invention, when the information indicating whether or not affine inter prediction is used indicates false, the DCMVD indication information is not present within the bitstream, and in this case, the current block is predicted using the conventional inter prediction.
- the parent header including the information indicating whether or not affine inter prediction according to the present invention is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
- FIG. 16 is a diagram illustrating a process of decoding the DCMVD indication information encoded as shown in FIG. 15 .
- the decoding device 400 decodes the DCMVD indication information at step S 161 , decodes the motion information at step S 162 , and ends the algorithm.
- the DCMVD indication information is present in the bitstream.
- the DCMVD indication information is not present within the bitstream, and in this case, the current block is predicted using the conventional inter prediction.
- the parent header including the information indicating whether or not affine inter prediction is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
- FIG. 17 is a flowchart illustrating an example of an image decoding method in which motion vectors of three control points are derived using a reconstructed pixel region so as to generate a prediction block of a current block.
- the process shown in FIG. 17 relates to the embodiment shown in FIGS. 12 a and 12 b.
- three reconstructed pixel regions a 12 a - 3 , b 12 a - 4 , and c 12 a - 5 are selected.
- one or two reconstructed pixel regions may be selected.
- the image decoding device 400 may determine, on the basis of the DCMVD indication information, which inter prediction is to be performed.
- the DCMVD indication information indicates use of affine inter prediction according to the present invention at step S 171
- the motion vectors of the control points at the top left, the top right, and the bottom left of the current block are estimated and selected using the respective reconstructed pixel regions at step S 172 .
- the motion vector obtained by decoding the transmitted motion information in the bitstream is set to be the motion vector of the control point at the bottom right at step S 173 .
- an inter prediction block of the current block is generated at step S 174 .
- the prediction block of the current block is generated at step S 175 according to the conventional inter prediction in which the motion information is decoded and the decoded motion information is used.
- FIG. 18 is a diagram illustrating a current block partitioned into multiple regions for inter prediction according to a third exemplary embodiment of the present invention.
- FIG. 18 shows a current block 500 to be encoded or decoded and a pre-reconstructed pixel region C 503 as a region adjacent to the current block 500 .
- the current block 500 is partitioned into region A 500 - a and region B 500 - b.
- the pixels within the reconstructed pixel region C 503 is likely to be similar to the pixels included in the region A 500 - a , but is unlikely to be similar to the pixels included in the region B 500 - b . Therefore, in inter prediction on the region A 500 - a , motion estimation and motion compensation using the reconstructed pixel region C 503 are performed to find accurate motion while preventing increase in overhead. In the meantime, as the inter prediction method for the region B 500 - b , the conventional inter prediction may be applied.
- FIG. 19 is a flowchart illustrating an inter prediction method according to a third exemplary embodiment of the present invention.
- Inter prediction may be performed by the inter prediction module 103 of the image encoding device 100 or by the inter prediction module 408 of the image decoding device 400 .
- the reference images used in inter prediction are stored in the memory 112 of the image encoding device 100 or the memory 406 of the image decoding device 400 .
- the inter prediction module 103 or the inter prediction module 408 may generate, with reference to the reference image stored in the memory 112 or the memory 406 , the prediction block of the region A 500 - a and the prediction block of the region B 500 - b within the current block.
- the current block to be encoded or decoded is partitioned into multiple regions including a first region and a second region at step S 51 .
- the first region and the second region may correspond to the region A 500 - a and the region B 500 - b shown in FIG. 5 , respectively.
- the current block 500 shown in FIG. 18 is partitioned into two regions, the region A 500 - a and the region B 500 - b , but may be partitioned into three or more regions and may be partitioned into regions in various sizes and/or shapes.
- the inter prediction method for the region A 500 - a may be, as described above, the method in which motion estimation and motion compensation using the reconstructed pixel region C 503 are performed.
- the conventional inter prediction may be applied.
- a method in which the current block is partitioned into multiple regions and the prediction blocks of the respective regions are derived using different inter prediction methods is referred to as a mixed inter prediction.
- FIG. 20 is a diagram illustrating an example of motion estimation and motion compensation using a reconstructed pixel region.
- a reference image 600 is searched for a region matched with the reconstructed pixel region C 503 shown in FIG. 18 .
- a reconstructed pixel region D 603 that is most similar to the reconstructed pixel region C 503 is determined, a displacement between a region 601 , which is at the same position as the reconstructed pixel region C 503 , and the reconstructed pixel region D 603 is selected as the motion vector 605 of the region A 500 - a.
- the motion vector 605 estimated using the reconstructed pixel region C 503 is selected as the motion vector of the region A 500 - a of the current block.
- the prediction block of the region A 500 - a is generated.
- the reconstructed pixel region C 503 may be in various shapes and/or sizes. Also, it is possible that the upper and left sides of the reconstructed pixel region are separated for use. Also, it is possible that the reconstructed pixel region is used by being subjected to subsampling. In this method, only the decoded information around the current block is used to derive the motion information, and thus it is not necessary to transmit the motion information from the encoding device 100 to the decoding device 400 .
- the decoding device 400 also performs motion estimation, so that if motion estimation is performed on the entire reference image, the complexity may extremely increase. Therefore, by transmitting the search range on a per-block basis or in the parent header or by fixing the search range to be the same in the encoding device 100 and in the decoding device 400 , computational complexity of the decoding device 400 may be reduced.
- the motion information of the decoded block within the reconstructed pixel region C 503 is used to predict the motion vector of the region B 500 - b and the residual vector corresponding to the difference between the motion vector of the region B 500 - b and the prediction motion vector is encoded.
- the motion vector 605 estimated as the motion vector of the region A 500 - a is used to predict the motion vector of the region B 500 - b and the residual vector is encoded.
- the motion vector of the decoded block within the reconstructed pixel region C 503 and the estimated motion vector 605 of the region A 500 - a are used to constitute a motion vector prediction set, the motion vector of the region B 500 - b is predicted, and the residual vector is encoded.
- block merging means that neighboring motion information is intactly applied to a block to be encoded.
- index indicating at which position block merging is performed is used.
- the size of the region B 500 - b is encoded by the encoding device 100 to be transmitted to the decoder on a per-block basis or through the parent header, or uses the same preset value or ratio in the encoding device 100 and the decoding device 400 .
- FIG. 21 is a flowchart illustrating a process of determining an inter prediction method according to an embodiment of the present invention.
- the optimum method may be determined through rate-distortion optimization (RDO).
- RDO rate-distortion optimization
- the process shown in FIG. 21 may be performed by the image encoding device 100 .
- inter prediction is performed using the conventional method to compute cost_A at step S 801 , and as described above, the mixed inter prediction according to the present invention, the current block is partitioned into at least two regions to be subjected to inter prediction individually, is performed to compute cost_B at step S 802 .
- cost_A and cost_B are computed to determine which method is optimum to use, at step S 803 .
- cost_A is lower, it is set to perform the inter prediction using the conventional method at step S 804 . Otherwise, it is set to perform the mixed inter prediction at step S 805 .
- FIG. 22 is a diagram illustrating a process of transmitting, to the decoding device 400 , information that indicates the inter prediction method determined by the process shown in FIG. 21 .
- Information indicating which type of inter prediction has been used for the block to be currently encoded is encoded at step S 901 .
- This information may be, for example, a 1-bit flag or one of several indexes.
- the motion information is encoded at step S 902 , and the algorithm ends.
- the information indicating whether or not the mixed inter prediction according to the embodiment of the present invention is used may be generated in the parent header first and then may be encoded. That is, when in the parent header, the information indicating whether or not the mixed inter prediction is used indicates true, the information indicating which type of inter prediction has been used for the block to be currently encoded is encoded. When the information indicating whether or not the mixed inter prediction is used indicates false, the information indicating which type of inter prediction has been used is not present within the bitstream, and in this case, the current block is not partitioned into multiple regions and the current block is predicted using the conventional inter prediction.
- the parent header including the information indicating whether or not the mixed inter prediction is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
- FIG. 23 is a diagram illustrating a process of decoding information encoded in the manner shown in FIG. 21 , namely, the information indicating which type of inter prediction has been used for the block to be currently encoded.
- the decoding device 400 decodes the information indicating which type of inter prediction has been used for the block to be currently encoded, at step S 1001 , decodes the motion information at step S 1002 , and ends the algorithm.
- the information indicating whether or not the mixed inter prediction is used is present in the parent header of the bitstream
- the information indicating whether or not the mixed inter prediction is used indicates true
- the information indicating which type of inter prediction has been used for the block to be currently encoded is present in the bitstream.
- the information indicating whether or not the mixed inter prediction is used indicates false
- the information indicating which type of inter prediction has been used is not present within the bitstream, and in this case, the current block is not partitioned into multiple regions and the current block is predicted using the conventional inter prediction.
- the parent header including the information indicating whether or not the mixed inter prediction is used may be a block header, a slice header, a tile header, a picture header, or a sequence header.
- the information indicating whether or not the mixed inter prediction is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
- FIG. 24 is a flowchart illustrating a method of generating an inter prediction block by using the information indicating which type of inter prediction has been used. The method shown in FIG. 24 may be performed by the decoding device 400 .
- the current block is partitioned into multiple regions at step S 1102 .
- the current block may be partitioned into the region A 500 - a and the region B 500 - b as shown in FIG. 5 .
- the size of each region resulting from the partitioning is signaled from the encoding device 100 to the decoding device 400 on a per-block basis or through the parent header or is set to a preset value.
- a motion vector of a first region for example, the region A 500 - a
- a prediction block is generated at step S 1103 .
- the decoded motion vector is used to generate a prediction block at step S 1104 , and the algorithm ends.
- the conventional inter prediction is applied as the prediction method of the current block 500 . That is, the decoded motion information is used to generate the prediction block of the current block 500 at step S 1105 , and the algorithm ends.
- the size of the prediction block is the same as the size of the current block 500 to be decoded.
- the fourth exemplary embodiment relates to a method to reduce blocking artifacts that may occur at the boundary of the block when the mixed inter prediction according to the third exemplary embodiment is performed.
- FIG. 25 is a diagram illustrating a method of reducing blocking artifacts that may occur when the mixed inter prediction according to the present invention is performed.
- Prediction block 1 and prediction block 2 shown in FIG. 25 may correspond to the prediction block of the region A 500 - a and the prediction block of the region B 500 - b shown in FIG. 18 , respectively.
- the regions positioned at the boundaries of the prediction block are partitioned into sub-blocks in a predetermined size.
- the motion information of the sub-block around the sub-block of the prediction block is applied to the sub-block of the prediction block so that a new prediction block is generated.
- a weighted sum of the sub-block of the prediction block and the new prediction block is obtained so that the final sub-block of the prediction block is generated. This is referred to as overlapped block motion compensation (OBMC).
- OBMC overlapped block motion compensation
- the motion information of the neighboring sub-block A 2 is applied to the sub-block P 2 to generate a new prediction block of the sub-block P 2 , and then the weighted sum is applied as shown in FIG. 26 to generate the final sub prediction block.
- each sub-block shown in FIG. 25 is 4 ⁇ 4; that there are the prediction block 1 , eight sub-blocks A 1 to A 8 adjacent to the upper side thereof, and eight sub-blocks B 1 to B 8 adjacent to the left side thereof; and that there are the prediction block 2 , four sub-blocks C 1 to C 4 adjacent to the upper side thereof, and four sub-blocks D 1 to D 8 adjacent to the left side thereof.
- each sub-block may be four, other various values may be encoded on a per-block basis or through the parent header and may be then signaled to the decoding device 400 . Accordingly, the encoding device 100 and the decoding device 400 may set the size of the sub-block to be the same. Alternatively, it is possible that the encoding device 100 and the decoding device 400 use sub-blocks in a preset same size.
- FIG. 26 is a diagram illustrating a method of applying a weighted sum of the sub-block within the prediction block and the sub-block adjacent to the upper side thereof.
- the final prediction pixel c is generated using Equation below.
- the remaining 15 pixels may be computed in a manner similar to the above.
- P 2 to P 8 in FIG. 13 are replaced by new prediction pixels to which the weighted sum is applied through the process shown in FIG. 26 .
- FIG. 27 is a diagram illustrating a method of applying a weighted sum of a sub-block within a prediction block and a sub-block adjacent to the left side thereof.
- the sub-blocks P 9 to P 15 are replaced by new prediction pixel values to which the weighted sum is applied as shown in FIG. 27 .
- the same weighting factors are applied to the pixels on a per-row basis, and referring to FIG. 27 , the same weighting factors are applied to the pixels on a per-column basis.
- the weighted sum with the pixels within the neighboring sub-block A 1 is performed as shown in FIG. 26
- the weighted sum with the pixels within the sub-block B 1 is performed as shown in FIG. 27 , thereby obtaining final prediction values.
- the weighted sum calculation method shown in FIG. 26 or FIG. 27 is used to obtain the final prediction values.
- the neighboring sub-blocks used for the weighted sum are C 1 to C 4 or D 1 to D 4 .
- the motion information of the sub-block P 17 is applied to the sub-block C 2 to generate a prediction sub-block, and then the pixel values within the prediction sub-block and the pixel values of the sub-block C 2 are subjected to the weighted sum so that the pixel values of the sub-block C 2 to which the weighted sum is applied is generated.
- FIG. 28 is a flowchart illustrating a process of determining whether or not the weighted sum is applied between sub-blocks at the boundary of the block, when the mixed inter prediction according to the present invention is performed.
- variable BEST_COST storing the optimum cost is initialized to the maximum value
- COMBINE_MODE storing whether or not the mixed inter prediction is used is initialized to false
- WEIGHTED_SUM storing whether or not the weighted sum is used between sub-blocks is initialized to false at step S 1501 .
- inter prediction using the conventional method is performed, and then cost_A is computed at step S 1502 .
- the mixed inter prediction is performed, and then cost_B is computed at step S 1503 .
- COMBINE_MODE is set to false to indicate that the mixed inter prediction is not used and BEST_COST stores cost_A at step S 1505 .
- COMBINE_MODE When the value of cost_B is lower, COMBINE_MODE is set to true to indicate that the mixed inter prediction is used and BEST_COST stores cost_B at step S 1506 . Afterward, the weighted sum is applied between the sub-blocks and cost_C is computed at step S 1507 .
- the variable WEIGHTED_SUM After comparing BEST_COST with cost_C at step S 1508 , when BEST_COST is lower than cost_C, the variable WEIGHTED_SUM is set to false to indicate that the weighted sum is not applied between the sub-blocks at step S 1509 . Otherwise, the variable WEIGHTED_SUM is set to true to indicate that the weighted sum is applied between the sub-blocks at step S 1510 and the algorithm ends.
- FIG. 29 is a flowchart illustrating a process of encoding information determined by the method in FIG. 27 , namely, information indicating whether or not a weighted sum is applied between sub-blocks.
- the process shown in FIG. 29 may be performed by the image encoding device 100 .
- the encoding device 100 encodes the information indicating which type of inter prediction has been used at step S 1601 , and encodes the motion information at step S 1602 .
- the information indicating whether or not the weighted sum is applied between the sub-blocks is encoded at step S 1603 .
- the information indicating whether or not the mixed inter prediction is used is present in the parent header of the bitstream, and when the information indicating whether or not the mixed inter prediction is used indicates true, the information indicating whether or not the weighted sum is applied between the sub-blocks is encoded and then included in the bitstream. However, when the information, included in the parent header, indicating whether or not the mixed inter prediction is used indicates false, the information indicating whether or not the weighted sum is applied between the sub-blocks is not present within the bitstream.
- FIG. 30 is a flowchart illustrating a process of decoding information indicating whether or not a weighted sum is applied between sub-blocks.
- the process shown in FIG. 30 may be performed by the image decoding device 400 .
- the decoding device 400 decodes the information indicating which type of inter prediction has been used at step S 1701 , and decodes the motion information at step S 1702 .
- the information indicating whether or not the weighted sum is applied between the sub-blocks is decoded at step S 1703 .
- the information indicating whether or not the mixed inter prediction is used is present in the parent header of the bitstream, and when the information indicating whether or not the mixed inter prediction is used indicates true, the information indicating whether or not the weighted sum is applied between the sub-blocks is encoded and then included in the bitstream.
- the information indicating whether or not the mixed inter prediction is used indicates false
- the information indicating whether or not the weighted sum is applied between the sub-blocks is not present within the bitstream. In this case, it may be inferred that the information indicating whether or not the weighted sum is applied between the sub-blocks indicates that the weighted sum is not applied between the sub-blocks.
- FIGS. 31 a and 31 b are diagrams illustrating inter prediction using a reconstructed pixel region according to the fifth exemplary embodiment of the present invention.
- the reconstructed pixel region may be used to derive the motion vector of the current block.
- FIG. 31 a shows a current block 252 to be encoded or decode and a pre-reconstructed pixel region C 251 as a region adjacent to the current block 252 .
- the reconstructed pixel region C 251 includes two regions, regions at the left side and the upper side of the current block 252 .
- the current block 252 and the reconstructed pixel region C 251 are included within the current image 250 .
- the current image 250 may be a picture, a slice, a tile, a coding tree block, a coding block, or other image regions.
- the reconstructed pixel region C 251 may correspond to a reconstructed pixel region after being encoded before encoding of the current block 252 in terms of encoding, and may correspond to a pre-reconstructed pixel region before decoding of the current block 252 in terms of decoding.
- the reconstructed pixel region C 251 neighbors the current block 252 , and thus the image encoding device 100 and the image decoding device 400 may use the same reconstructed pixel region C 251 . Therefore, without encoding the motion information of the current block 252 by the image encoding device 100 , the reconstructed pixel region C 251 is used such that the image encoding device 100 and the image decoding device 400 may generate the motion information of the current block 252 and the prediction block in the same manner.
- FIG. 31 b shows an example of motion estimation and motion compensation using a reconstructed pixel region.
- a reference image 253 shown in FIG. 31 b is searched for a region matched with the reconstructed pixel region C 251 shown in FIG. 31 a .
- a reconstructed pixel region D 256 that is most similar to the reconstructed pixel region C 251 is determined, a displacement between a region 254 , which is at the same position as the reconstructed pixel region C 251 , and the reconstructed pixel region D 256 is determined to be a motion vector 257 of the reconstructed pixel region C 251 .
- the motion vector 257 determined as described above is selected as the motion vector of the current block 252 , and a prediction block of the current block 252 may be derived using the motion vector 257 .
- FIG. 32 is a diagram illustrating an example of a case where the motion vector 257 estimated as shown in FIG. 31 b is set as an initial motion vector, the current block 252 is partitioned into multiple sub-blocks A to D, and then motion estimation is further performed on a per-sub-block basis.
- the sub-blocks A to D may be in an arbitrary size.
- MV_A to MV_D shown in FIG. 32 are the initial motion vectors of the sub-blocks A to D, respectively, and are the same as the motion vector 257 shown in FIG. 31 b.
- each sub-block may be encoded on a per-block basis or through the parent header and may be transmitted to the decoding device 400 .
- the encoding device 100 and the decoding device 400 use the same preset size value of the sub-block.
- the reconstructed pixel region C 251 may be in various shapes and/or sizes. Also, it is possible that the reconstructed pixel regions at the upper side and the left side of the current block are used as the reconstructed pixel region C or that as shown in FIGS. 31 a to 31 b , the two regions are combined into a single piece to be used as the reconstructed pixel region C 251 . Also, it is possible that the reconstructed pixel region C 251 is used by being subjected to subsampling.
- the description is given assuming that the reconstructed pixel region C 251 as shown in FIGS. 31 a and 31 b is used as a reconstructed pixel region.
- FIG. 33 is a diagram illustrating an example in which the reconstructed pixel region C 251 and the current block are partitioned on a per-sub-block basis.
- the reconstructed pixel region C 251 is partitioned into sub-blocks a 285 , b 286 , c 287 , and d 288 , and the current block is partitioned in to sub-blocks A 281 , B 282 , C 283 , and D 284 .
- the sub-blocks a 285 and c 287 may be used.
- the sub-blocks b 286 and c 287 may be used.
- the sub-blocks a 285 and d 288 may be used.
- the sub-blocks b 286 and d 288 may be used.
- FIG. 34 is a flowchart illustrating an example of an inter prediction method using a reconstructed pixel region.
- the reconstructed pixel region 251 of the current block 252 is set at step S 291 , and then the reconstructed pixel region 251 is used to perform motion estimation on the reference image 253 at step S 292 .
- the motion vector 257 of the reconstructed pixel region 251 is obtained. Afterward, as shown in FIG.
- the reconstructed pixel region is set on a per-sub-block basis at step S 293 , the motion vector 257 estimated at step S 292 is set as a start point, and then motion is estimated on a per-sub-block basis of the current block at step S 294 .
- FIG. 35 is a diagram illustrating an example of partitioning a reconstructed pixel region into sub-blocks by using reconstructed blocks neighboring a current block according to the present invention.
- the reconstructed neighboring pixel region used for prediction of the current block may be partitioned on the basis of a partitioning structure of reconstructed neighboring blocks.
- the reconstructed pixel region may be partitioned.
- FIG. 35 there are reconstructed block 1 2101 to reconstructed block 5 2105 around the current block 2100 to be encoded or decoded.
- efficiency in motion estimation may decrease due to the dramatic difference in pixel values, which may be present at each of the boundaries of the reconstructed block 1 2101 to the reconstructed block 5 2105 . Therefore, as shown in FIG. 35 , it may be efficient to partition the reconstructed pixel region into sub-blocks a to e for use.
- the reconstructed pixel region shown in FIG. 35 may be partitioned.
- the number of reconstructed neighboring blocks may be considered in partitioning of the reconstructed pixel region.
- two reconstructed blocks the reconstructed block 1 2101 and the reconstructed block 2 2102 , are present at the upper side of the current block 2100 .
- the reconstructed pixel region at the upper side of the current block 2100 is partitioned into two sub-blocks, the sub-blocks a and b.
- the reconstructed pixel region at the left side of the current block 2100 is partitioned into three sub-blocks, the sub-blocks c to e.
- the sizes of the reconstructed neighboring blocks may be considered in partitioning of the reconstructed pixel region.
- the height of the sub-block c of the reconstructed pixel region at the left side of the current block 2100 is the same as that of the reconstructed block 3 2103 .
- the height of the sub-block d is the same as that of the reconstructed block 4 2104 .
- the height of the sub-block e corresponds to a value obtained by subtracting the height of the sub-block c and the height of the sub-block d from the height of the current block 2100 .
- the boundaries between the reconstructed neighboring blocks may be considered in partitioning of the reconstructed pixel region.
- the reconstructed pixel region at the upper side of the current block 2100 is partitioned into two sub-blocks, the sub-blocks a and b.
- the reconstructed pixel region at the left side of the current block 2100 is partitioned into three sub-blocks, the sub-blocks c to e.
- region of the sub-blocks a to e may be various conditions with respect to which region of the sub-blocks a to e is used to perform motion estimation. For example, it is possible that motion estimation is performed using only one reconstructed pixel region having the largest area, or it is possible that m reconstructed pixel regions from the top and n reconstructed pixel regions from the left side are selected according to the priority and used for motion estimation. Alternatively, it is also possible that a filter such as a low-pass filter is applied between the sub-blocks a to e to relieve the dramatic difference in pixel values and then one reconstructed pixel region 251 as shown in FIG. 5 a is used.
- a filter such as a low-pass filter
- FIG. 36 is a diagram illustrating an example of partitioning a current block into multiple sub-blocks by using reconstructed blocks neighboring the current block according to the present invention.
- the method of partitioning the current block shown in FIG. 36 into multiple sub-blocks is similar to the method of partitioning the reconstructed pixel region shown in FIG. 35 . That is, the current block to be encoded or decoded may be partitioned on the basis of a partitioning structure of reconstructed neighboring blocks. In other words, on the basis of at least one among the number of the reconstructed neighboring blocks, the sizes of the reconstructed neighboring blocks, the shapes of the reconstructed neighboring blocks, and the boundaries between the reconstructed neighboring blocks, the current block may be partitioned.
- the current block shown in FIG. 36 is partitioned into multiple sub-blocks A to F.
- Inter prediction may be performed on a per-sub-block basis, wherein the sub-blocks result from the partitioning.
- inter prediction may be performed using reconstructed regions a and c in FIG. 10 for the sub-block A, using reconstructed regions b and c for the sub-block B, using reconstructed regions a and d for the sub-block C, using reconstructed regions b and d for the sub-block D, using reconstructed regions a and e for the sub-block E, and using reconstructed regions b and e for the sub-block F.
- priority is set depending on the sizes of the sub-blocks and the reconstructed pixel regions. For example, in the case of the sub-block A shown in FIG. 36 , because the length is longer than the height, it is possible that the reconstructed region a has priority over the reconstructed region c and inter prediction is performed only using the reconstructed region a. Alternatively, conversely, it is possible that the reconstructed region c has priority depending on the situation, such as image characteristics, and the like.
- FIG. 37 is a flowchart illustrating a method of partitioning a current block into multiple sub-blocks according to an embodiment of the present invention.
- the current block is partitioned into multiple sub-blocks at step S 2201 .
- the neighboring blocks of the current block are pre-reconstructed blocks as shown in FIG. 36 .
- the current block to be encoded or decoded may be partitioned on the basis of a partitioning structure of reconstructed neighboring blocks. That is, on the basis of at least one among the number of the reconstructed neighboring blocks, the sizes of the reconstructed neighboring blocks, the shapes of the reconstructed neighboring blocks, and the boundaries between the reconstructed neighboring blocks, the current block may be partitioned.
- each of the sub-blocks A to F of the current block shown in FIG. 36 may be encoded or decoded using inter prediction.
- inter prediction may be performed using reconstructed regions a and c in FIG. 35 for the sub-block A, using reconstructed regions b and c for the sub-block B, using reconstructed regions a and d for the sub-block C, using reconstructed regions b and d for the sub-block D, using reconstructed regions a and e for the sub-block E, and using reconstructed regions b and e for the sub-block F.
- Information related to inter prediction such as sub block information indicating whether or not partitioning into sub-blocks is performed, which is obtained by performing inter prediction on each of the sub-blocks A to F, motion information, or the like, may be encoded or decoded.
- the method shown in FIG. 37 may be performed by the inter prediction module 103 of the image encoding device 100 or by the inter prediction module 408 of the image decoding device 400 .
- the reference images used in inter prediction are stored in the memory 112 of the image encoding device 100 or in the memory 406 of the image decoding device 400 .
- the inter prediction module 103 or the inter prediction module 408 may generate, with reference to the reference image stored in the memory 112 or the memory 406 , the prediction block of the current block 51 .
- FIG. 38 is a flowchart illustrating a method of partitioning a reconstructed region used in encoding or decoding of a current block into multiple sub-blocks according to an embodiment of the present invention.
- the pre-reconstructed pixel region is partitioned into multiple sub-blocks at step S 2211 .
- the reconstructed neighboring pixel region used for prediction of the current block may be partitioned on the basis of a partitioning structure of reconstructed neighboring blocks. In other words, on the basis of at least one among the number of the reconstructed neighboring blocks, the sizes of the reconstructed neighboring blocks, the shapes of the reconstructed neighboring blocks, and the boundaries between the reconstructed neighboring blocks, the reconstructed pixel region may be partitioned.
- inter prediction may be performed using reconstructed regions a and c in FIG. 35 for the sub-block A, using reconstructed regions b and c for the sub-block B, using reconstructed regions a and d for the sub-block C, using reconstructed regions b and d for the sub-block D, using reconstructed regions a and e for the sub-block E, and using reconstructed regions b and e for the sub-block F.
- Information related to inter prediction such as sub block information indicating whether or not partitioning into sub-blocks is performed, which is obtained by performing inter prediction on each of the sub-blocks A to F, motion information, or the like, may be encoded or decoded.
- the method shown in FIG. 38 may be performed by the inter prediction module 103 of the image encoding device 100 or by the inter prediction module 408 of the image decoding device 400 .
- the reference images used in inter prediction is stored in the memory 112 of the image encoding device 100 or in the memory 406 of the image decoding device 400 .
- the inter prediction module 103 or the inter prediction module 408 may generate, with reference to the reference image stored in the memory 112 or in the memory 406 , the prediction block of the current block 51 .
- FIG. 39 is a flowchart illustrating an example of an inter prediction method using the sub-blocks of the partitioned current block as shown in FIG. 36 .
- the method shown in FIG. 39 may be performed by the inter prediction module 103 of the image encoding device 100 .
- the decoder-side motion vector derivation (DMVD) indication information or decoder-side motion vector derivation indication information is information indicating whether the inter prediction using the conventional method is performed or the above-described inter prediction using the reconstructed pixel region according to the present invention is performed. When the DMVD indication information indicates false, it indicates that the inter prediction using the conventional method is performed. When the DMVD indication information indicates true, it indicates that the inter prediction using the reconstructed pixel region according to the present invention is performed.
- the variable SUB_BLOCK indicates whether or not the current block is partitioned into sub-blocks. When the value of SUB_BLOCK indicates false, it indicates that the current block is not partitioned into sub-blocks. Conversely, when the value of SUB_BLOCK indicates true, it indicates that the current block is partitioned into sub-blocks.
- variable DMVD indication information which indicates whether or not the inter prediction using the reconstructed pixel region is performed, is set to false and the variable SUB_BLOCK, which indicates whether or not partitioning into sub-blocks is performed, is set to false, and then inter prediction is performed on the current block and cost_1 is computed at step S 2301 .
- SUB_BLOCK is set to true and inter prediction is performed, and then cost_2 is computed at step S 2302 .
- the DMVD indication information is set to true and SUB_BLOCK is set to false, and then inter prediction is performed and cost_3 is computed at step S 2303 .
- the DMVD indication information and SUB_BLOCK are set to true, and then inter prediction is performed and cost_4 is computed at step S 2304 .
- the calculated cost_1 to cost_4 are compared with each other, and then the optimum inter prediction method is determined.
- the DMVD indication information and the SUB_BLOCK information related to the determined optimum inter prediction method are stored, and then the algorithm ends.
- FIG. 40 is a flowchart illustrating a method of encoding information determined according to inter prediction shown in FIG. 39 .
- the encoding method in FIG. 40 may be performed by the image encoding device 100 .
- the total number of sub-blocks of the current block is set to six, so that the variable BLOCK NUM, which indicates the total number of sub-blocks to be encoded, is initialized to six and the variable BLOCK INDEX, which indicates the index of the sub-block to be encoded, is initialized to zero at step S 2401 .
- the current block is partitioned into the sub-blocks on the basis of the reconstructed blocks around the current block, so that it is not necessary to encode the number of the sub-blocks.
- the image decoding device 400 partitions the current block into the sub-blocks in the same manner as the image encoding device 100 , so that the image decoding device 400 is capable of determining the number of the sub-blocks that may be present in the current block.
- step S 2401 SUB_BLOCK, the information indicating whether or not the current block is partitioned into sub-blocks, is encoded at step S 2402 . Whether or not the current block is partitioned into the sub-blocks is determined at step S 2403 , and when the partitioning into the sub-blocks is not performed, the value of the variable BLOCK NUM is changed into one at step S 2404 .
- the DMVD indication information indicating whether or not the inter prediction using the reconstructed pixel region has been used is encoded at step S 2405 . Whether or not the inter prediction using the reconstructed pixel region has been used is determined at step S 2406 , and when the inter prediction using the reconstructed pixel region has not been used, the motion information is encoded at step S 2407 . Conversely, when the inter prediction using the reconstructed pixel region has been used, the value of BLOCK INDEX is increased at step S 2408 and is compared with the variable BLOCK NUM at step S 2409 .
- FIG. 41 is a flowchart illustrating an example of a method of decoding information encoded by the encoding method shown in FIG. 40 .
- the total number of sub-blocks of the current block is set to six, so that the variable BLOCK NUM, which indicates the total number of sub-blocks to be decoded, is initialized to six and the variable BLOCK INDEX, which indicates the index of the sub-block to be decoded, is initialized to zero at step S 2501 .
- the image decoding device 400 and the image encoding device 100 partition the current block into the sub-blocks in the same manner, so that the information indicating the number of the sub-blocks does not need to be transmitted to the image decoding device 400 .
- the image decoding device 400 may determine by itself, on the basis of the reconstructed blocks around the current block, the number of the sub-blocks that may be present in the current block.
- step S 2501 SUB BLOCK, the information indicating whether or not the current block is partitioned into sub-block, is decoded at step S 2502 . Whether or not the current block is partitioned into the sub-blocks is determined at step S 2403 , and when the partitioning into the sub-blocks is not performed, the value of the variable BLOCK NUM is changed into one at step S 2404 .
- the DMVD indication information indicating whether or not the inter prediction using the reconstructed pixel region has been used is decoded at step S 2505 . Whether or not the inter prediction using the reconstructed pixel region has been used is determined at step S 2506 , and when the inter prediction using the reconstructed pixel region has not been used, the motion information is decoded at step S 2507 . Conversely, when the inter prediction using the reconstructed pixel region has been used, the value of BLOCK INDEX is increased at step S 2508 and is compared with the variable BLOCK NUM at step S 2509 .
- FIGS. 42 a and 42 b are diagrams illustrating the sixth exemplary embodiment of the present invention.
- the reconstructed pixel region may be partitioned into sub-blocks a to f according to the method shown in FIG. 36 .
- the current block 2600 may be partitioned into sub-blocks A to I.
- the sub-blocks F, G, H, and I are spaced apart from the reconstructed pixel region rather than being in contact therewith, so that inter prediction using the reconstructed pixel region may be inaccurate. Therefore, in the case of the sub-blocks F, G, H, and I, the conventional inter prediction is performed, and only in the case of the sub-blocks A to E, inter prediction using the reconstructed pixel region may be used.
- inter prediction is performed using the reconstructed pixel region adjacent to each sub-block.
- inter prediction may be performed using reconstructed pixel region b for the sub-block B, using reconstructed pixel region c for the sub-block C, using reconstructed pixel region e for the sub-block D, and using reconstructed pixel region f for the sub-block E.
- inter prediction may be performed using either the reconstructed pixel region a or d, or using the reconstructed pixel regions a and d.
- an index indicating which reconstructed pixel region is used for each sub-block when inter prediction using the reconstructed pixel region is performed on the sub-blocks A to E is encoded.
- the reconstructed pixel region b may be used to perform inter prediction on the sub-block A.
- inter prediction may be performed using the reconstructed pixel region c.
- the priority is determined and indexes are assigned.
- sub-blocks F to I encoding or decoding may be possible by performing the conventional inter prediction.
- the sub-blocks F to I may be integrated into one and may be encoded or decoded using the conventional inter prediction.
- FIG. 43 is a flowchart illustrating an example of a method of determining an inter prediction mode according to the sixth exemplary embodiment of the present invention described with reference to FIGS. 42 a and 42 b .
- the reconstructed block 1 2601 to the reconstructed block 6 2606 are present, and that the reconstructed pixel region is partitioned into sub-reconstructed pixel regions a to f.
- the current block 2600 is partitioned into the sub-blocks A to F.
- the sub-block F is the one into which the sub-blocks F to I in FIG. 42 a are integrated.
- the index indicating which sub-reconstructed region is used when performing inter prediction using the reconstructed pixel region is encoded, will be described as an example.
- the sub-block F is encoded or decoded by performing the conventional inter prediction. The description is given assuming that among the sub-blocks within the current block, the sub-block F is encoded or decoded last.
- the current block is subjected to inter prediction without being partitioned into sub-blocks, and then cost_1 is computed at step S 2701 .
- the sub-blocks A to F are subjected to inter prediction individually, and cost_A to cost_F are computed and then added up to compute cost_2 at step S 2702 .
- the computed cost_1 is compared with the computed cost_2 at step S 2703 . When cost_1 is lower, it is determined at step S 2704 that partitioning into sub-blocks is not performed. Otherwise, it is determined at step S 2705 that partitioning into sub-blocks is performed and inter prediction is performed, and the algorithm ends.
- FIG. 44 is a diagram illustrating a process of encoding information determined by the method shown in FIG. 43 .
- the total number of sub-blocks of the current block is set to six, so that the variable BLOCK_NUM, which indicates the total number of sub-blocks to be encoded, is initialized to six and the variable BLOCK INDEX, which indicates the index of the sub-block to be encoded, is initialized to zero at step S 2801 .
- the current block is partitioned into the sub-blocks on the basis of the reconstructed blocks around the current block, so that it is not necessary to encode the number of the sub-blocks.
- the image decoding device 400 partitions the current block into the sub-blocks in the same manner as the image encoding device 100 , so that the image decoding device 400 is capable of determining the number of the sub-blocks that may be present in the current block.
- step S 2801 SUB BLOCK, the information indicating whether or not the current block is partitioned into sub-blocks, is encoded at step S 2802 . Whether or not the current block is partitioned into the sub-blocks is determined at step S 2803 , and when the partitioning into the sub-blocks is not performed, the value of the variable BLOCK NUM is changed into one at step S 2804 .
- Step S 2805 at which the value of BLOCK INDEX is compared with the value of BLOCK_NUM ⁇ 1, is the step of determining whether the conventional inter prediction is used for the block or the inter prediction using the reconstructed pixel region is used for the block.
- it is the last block, namely, the sub-block subjected to the conventional inter prediction, so that the motion information is encoded at step S 2806 .
- it is the sub-block subjected to the inter prediction using the reconstructed pixel region, so that the index indicating which sub-reconstructed region is used is encoded at step S 2807 .
- this step is skipped and the same reconstructed region determined in the encoding device and the decoding device is used.
- step S 2808 the index of the sub-block is increased at step S 2808 , and BLOCK_NUM is compared with BLOCK_INDEX to determine whether or not encoding of all the sub-blocks present in the current block is completed, at step S 2809 . If not, proceeding to step S 2805 takes place and the algorithm continues.
- FIG. 45 is a diagram illustrating a process of decoding information encoded by the method shown in FIG. 44 .
- the total number of sub-blocks of the current block is set to six, so that the variable BLOCK_NUM, which indicates the total number of sub-blocks to be encoded, is initialized to six and the variable BLOCK_INDEX, which indicates the index of the sub-block to be encoded, is initialized to zero at step S 2901 .
- the current block is partitioned into the sub-blocks on the basis of the reconstructed blocks around the current block, so that it is not necessary to encode the number of the sub-blocks.
- the image decoding device 400 partitions the current block into the sub-blocks in the same manner as the image encoding device 100 , so that the image decoding device 400 is capable of determining the number of the sub-blocks that may be present in the current block.
- step S 2901 SUB BLOCK, the information indicating whether or not the current block is partitioned into sub-blocks, is decoded at step S 2902 . Whether or not the current block is partitioned into the sub-blocks is determined at step S 2903 , and when the partitioning into the sub-blocks is not performed, the value of the variable BLOCK_NUM is changed into one at step S 2904 .
- Step S 2905 at which the value of BLOCK_INDEX is compared with the value of BLOCK_NUM ⁇ 1, is the step of determining whether the conventional inter prediction is used for the block or the inter prediction using the reconstructed pixel region is used for the block.
- it is the last block, namely, the sub-block subjected to the conventional inter prediction, so that the motion information is decoded at step S 2906 .
- it is the sub-block subjected to the inter prediction using the reconstructed pixel region, so that the index indicating which sub-reconstructed region is used is decoded at step S 2907 .
- step S 2908 the index of the sub-block is increased at step S 2908 , and BLOCK_NUM is compared with BLOCK_INDEX to determine whether or not decoding of all the sub-blocks present in the current block is completed, at step S 2909 . If not, proceeding to step S 2905 takes place and the algorithm continues.
- various embodiments of the present invention may be implemented by hardware, firmware, software, or combinations thereof.
- implementation is possible by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general processors, controllers, micro controllers, microprocessors, or the like.
- ASICs application specific integrated circuits
- DSPs digital signal processors
- DSPDs digital signal processing devices
- PLDs programmable logic devices
- FPGAs field programmable gate arrays
- controllers controllers, micro controllers, microprocessors, or the like.
- the scope of the present invention includes software or machine-executable instructions (for example, an operating system, an application, firmware, a program, or the like) that cause operation according to the methods of the various embodiments to be performed on a device or a computer, and includes a non-transitory computer-readable medium storing such software or instructions to execute on a device or a computer.
- software or machine-executable instructions for example, an operating system, an application, firmware, a program, or the like
- the present invention is applicable to a field of encoding or decoding an image signal.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
- This application is a Continuation Application of U.S. patent application Ser. No. 16/478,259, filed on Jul. 16, 2019, which is a U.S. National Stage Application of International Application No. PCT/KR2018/000750, filed on Jan. 16, 2018, which claims the benefit under 35 USC 119(a) and 365(b) of Korean Patent Application No. 10-2017-0007346, filed on Jan. 16, 2017, Korean Patent Application No. 10-2017-0007347, filed on Jan. 16, 2017, and Korean Patent Application No. 10-2017-0007348, filed on Jan. 16, 2017 in the Korean Intellectual Property Office, the entire disclosures of which are incorporated herein by reference for all purposes.
- The present invention relates to an image signal encoding/decoding method and device. More particularly, the present invention relates to an image encoding/decoding method using inter prediction and an image encoding/decoding device using inter prediction.
- Recently, demand for multimedia data such as video has rapidly increased on the Internet. However, the rate at which a bandwidth of a channel has developed is insufficient to keep up with the amount of multimedia data that has rapidly increased. Considering this situation, the Video Coding Expert Group (VCEG) of ITU-T and the Moving Picture Expert Group (MPEG) of ISO/IEC, which are the International Organizations for Standardization, established the High Efficiency Video Coding (HEVC)
version 1, a video compression standard, in February 2014. - HEVC uses a variety of technologies such as intra prediction, inter prediction, transform, quantization, entropy encoding, and in-loop filtering. In inter prediction of HEVC, new technologies such as block merging, advanced motion vector prediction (AMVP) have been applied such that efficient inter prediction is possible. However, when multiple motions are present in a block, the block is partitioned into small parts, so that rapid increase in overhead may occur and encoding efficiency may be lowered.
- Accordingly, the present invention has been made keeping in mind the above problems, and the present invention is intended to enhance efficiency of inter prediction by providing improved inter prediction.
- Also, the present invention is intended to provide a motion vector derivation method by an image decoding device, where an image encoding device does not need to transmit motion vector information to the image decoding device.
- Also, the present invention is intended to provide a motion vector derivation method of a control point by an image decoding device, wherein in affine inter prediction, an image encoding device does not need to transmit a motion vector of the control point to the image decoding device.
- Also, the present invention is intended to provide inter prediction capable of efficient encoding or decoding when multiple motions are present in one block.
- Also, the present invention is intended to reduce blocking artifacts that may occur when one block is partitioned into multiple regions and encoding or decoding is performed using different types of inter prediction.
- Also, the present invention is intended to enhance efficiency of inter prediction by partitioning a current block to be encoded or decoded, on the basis of a partitioning structure of reconstructed neighboring blocks.
- Also, the present invention is intended to enhance efficiency of inter prediction by partitioning, on the basis of a partitioning structure of reconstructed neighboring blocks, a pre-reconstructed neighboring image region which is used to encode or decode a current block.
- Also, the present invention is intended to enhance efficiency of image encoding or decoding by performing encoding or decoding using a current block or a neighboring image partitioned as described above.
- In an image decoding method and device according to an embodiment of the present invention, a reconstructed pixel region within an image to which a current block to be decoded belongs is selected; on the basis of the reconstructed pixel region and a reference image of the current block, a motion vector of the reconstructed pixel region is derived; and the derived motion vector is selected as a motion vector of the current block.
- The reconstructed pixel region may include at least one among a region adjacent to an upper side of the current block and a region adjacent to a left side of the current block.
- The motion vector of the reconstructed pixel region may be derived on the basis of a position of a region corresponding to the reconstructed pixel region, wherein the region corresponding to the reconstructed pixel region is determined within the reference image.
- In an image encoding method and device according to an embodiment of the present invention, a reconstructed pixel region within an image to which a current block to be encoded belongs is selected; on the basis of the reconstructed pixel region and a reference image of the current block, a motion vector of the reconstructed pixel region is derived; and the derived motion vector is selected as a motion vector of the current block.
- Also, in the image encoding method and device according to the embodiment of the present invention, decoder-side motion vector derivation indication information may be generated and encoded.
- The decoder-side motion vector derivation indication information may indicate whether or not the derived motion vector of the reconstructed pixel region is selected as the motion vector of the current block.
- In an image decoding method and device according to another embodiment of the present invention, at least one reconstructed pixel region is selected within an image to which a current block to be decoded using affine inter prediction belongs; on the basis of the at least one reconstructed pixel region and a reference image of the current block, a motion vector of the at least one reconstructed pixel region is derived; and the derived motion vector of the at least one reconstructed pixel region is selected as a motion vector of at least one control point of the current block.
- The at least one reconstructed pixel region may be a region adjacent to the at least one control point of the current block.
- Further, the at least one control point may be positioned at an upper left side, an upper right side, or a lower left side of the current block.
- Further, the motion vector of the control point positioned at a lower right side of the current block may be decoded on the basis of motion information included in a bitstream.
- Further, in the image decoding method and device according to the embodiment of the present invention, decoder-side control point motion vector derivation indication information may be decoded.
- In the image decoding method and device according to the embodiment of the present invention, the motion vector of the at least one reconstructed pixel region may be derived on the basis of the decoder-side control point motion vector derivation indication information.
- In an image encoding method and device according to still another embodiment of the present invention, at least one reconstructed pixel region is selected within an image to which a current block to be encoded using affine inter prediction belongs; on the basis of the at least one reconstructed pixel region and a reference image of the current block, a motion vector of the at least one reconstructed pixel region is derived; and the derived motion vector of the at least one reconstructed pixel region is selected as a motion vector of at least one control point of the current block.
- In an image decoding method and device according to yet still another embodiment of the present invention, a current block to be decoded is partitioned into multiple regions including a first region and a second region; and a prediction block of the first region and a prediction block of the second region are obtained, wherein the prediction block of the first region and the prediction block of the second region are obtained by different inter prediction methods.
- The first region may be a region adjacent to a reconstructed image region within an image to which the current block belongs, and the second region may be a region that is not in contact with the reconstructed image region within the image to which the current block belongs.
- In the image decoding method and device according to the embodiment of the present invention, on the basis of the reconstructed image region within the image to which the current block belongs, and of a reference image of the current block, a motion vector of the first region may be estimated.
- In the image decoding method and device according the embodiment of the present invention, a region positioned at a boundary as a region within the prediction block of the first region or a region positioned at a boundary as a region within the prediction block of the second region may be partitioned into multiple sub-blocks; motion information of a neighboring sub-block of a first sub-block, which is one of the multiple sub-blocks, may be used to generate a prediction block of the first sub-block; and the first sub-block and the prediction block of the first sub-block may be subjected to a weighted sum, so that a prediction block of the first sub-block to which the weighted sum is applied may be obtained.
- In an image encoding method and device according to yet still another embodiment of the present invention, a current block to be encoded is partitioned into multiple regions including a first region and a second region; a prediction block of the first region and a prediction block of the second region are obtained, wherein the prediction block of the first region and the prediction block of the second region are obtained by different inter prediction methods.
- The first region may be a region adjacent to a pre-encoded reconstructed image region within an image to which the current block belongs, and the second region may be a region that is not in contact with the pre-encoded reconstructed image region within the image to which the current block belongs.
- In the image encoding method and device according to the embodiment of the present invention, on the basis of the pre-encoded reconstructed image region within the image to which the current block belongs, and of a reference image of the current block, a motion vector of the first region may be estimated.
- In the image encoding method and device according to the embodiment of the present invention, a region positioned at a boundary as a region within the prediction block of the first region or a region positioned at a boundary as a region within the prediction block of the second region may be partitioned into multiple sub-blocks; motion information of a neighboring sub-block of a first sub-block, which is one of the multiple sub-blocks, may be used to generate a prediction block of the first sub-block; and the first sub-block and the prediction block of the first sub-block may be subjected to a weighted sum, so that a prediction block of the first sub-block to which the weighted sum is applied may be obtained.
- In an image decoding method and device according to yet still another embodiment of the present invention, on the basis of blocks around a current block to be decoded, the current block is partitioned into multiple sub-blocks, and the multiple sub-blocks of the current block are decoded.
- In the image decoding method and device according to the embodiment of the present invention, on the basis of a partitioning structure of neighboring blocks of the current block, the current block may be partitioned into the multiple sub-blocks.
- In the image decoding method and device according to the embodiment of the present invention, on the basis of at least one among the number of the neighboring blocks, a size of the neighboring blocks, a shape of the neighboring blocks, and a boundary between the neighboring blocks, the current block may be partitioned into the multiple sub-blocks.
- In the image decoding method and device according to the embodiment of the present invention, as a region neighbors the current block, a pre-reconstructed pixel region may be partitioned on a per-sub-block basis, and at least one of the multiple sub-blocks of the current block may be decoded using at least one sub-block included in the reconstructed pixel region.
- In the image decoding method and device according to the embodiment of the present invention, on the basis of a partitioning structure of neighboring blocks of the current block, the reconstructed pixel region may be partitioned on a per-sub-block basis.
- In the image decoding method and device according to the embodiment of the present invention, on the basis of at least one among the number of the neighboring blocks, a size of the neighboring blocks, a shape of the neighboring blocks, and a boundary between the neighboring blocks, the reconstructed pixel region may be partitioned on a per-sub-block basis.
- In an image encoding method and device according to yet still another embodiment of the present invention, on the basis of blocks around a current block to be encoded, the current block may be partitioned into multiple sub-blocks, and the multiple sub-blocks of the current block may be encoded.
- In the image encoding method and device according to the embodiment of the present invention, on the basis of a partitioning structure of neighboring blocks of the current block, the current block may be partitioned into the multiple sub-blocks.
- In the image encoding method and device according to the embodiment of the present invention, on the basis of the at least one among the number of the neighboring blocks, a size of the neighboring blocks, a shape of the neighboring blocks, and a boundary between the neighboring blocks, the current block may be partitioned into the multiple sub-blocks.
- In the image encoding method and device according to the embodiment of the present invention, as a region neighbors the current block, a pre-reconstructed pixel region may be partitioned on a per-sub-block basis, and at least one of the multiple sub-blocks of the current block may be encoded using at least one sub-block included in the reconstructed pixel region.
- In the image encoding method and device according to the embodiment of the present invention, on the basis of a partitioning structure of neighboring blocks of the current block, the reconstructed pixel region may be partitioned on a per-sub-block basis.
- In the image encoding method and device according to the embodiment of the present invention, on the basis of at least one among the number of the neighboring blocks, a size of the neighboring blocks, a shape of the neighboring blocks, and a boundary between the neighboring blocks, the reconstructed pixel region may be partitioned on a per-sub-block basis.
- According to the present invention, the amount of encoding information generated as a result of encoding a video may be reduced and thus encoding efficiency may be enhanced. Also, by adaptively decoding an encoded image, reconstruction efficiency of an image may be enhanced and the quality of the reproduced image may be improved.
- Also, in inter prediction according to the present invention, the image encoding device does not need to transmit motion vector information to the image decoding device, so that the amount of encoding information may be reduced and thus encoding efficiency may be enhanced.
- Also, according to the present invention, blocking artifacts may be reduced that may occur when one block is partitioned into multiple regions and encoding or decoding is performed using different types of inter prediction.
-
FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present invention. -
FIG. 2 is a diagram illustrating a method of generating motion information by using motion estimation according to the conventional technology. -
FIG. 3 is a diagram illustrating an example of neighboring blocks that may be used to generate motion information of a current block. -
FIG. 4 is a block diagram illustrating an image decoding device according to an embodiment of the present invention. -
FIGS. 5a and 5b are diagrams illustrating inter prediction using a reconstructed pixel region according to a first exemplary embodiment of the present invention. -
FIG. 6 is a flowchart illustrating an inter prediction method according to the first exemplary embodiment of the present invention. -
FIGS. 7a to 7c are diagrams illustrating examples of reconstructed pixel regions. -
FIG. 8 is a flowchart illustrating a process of determining an inter prediction method according to an embodiment of the present invention. -
FIG. 9 is a diagram illustrating a process of encoding information that indicates an inter prediction method. -
FIG. 10 is a diagram illustrating a process of decoding DMVD indication information encoded as shown inFIG. 9 . -
FIG. 11 is a diagram illustrating affine inter prediction. -
FIGS. 12a and 12b are diagrams illustrating derivation of a motion vector of a control point by using a reconstructed pixel region according to a second exemplary embodiment of the present invention. -
FIG. 13 is a flowchart illustrating an inter prediction method according to the second exemplary embodiment of the present invention. -
FIG. 14 is a flowchart illustrating a process of determining an inter prediction method according to the second exemplary embodiment of the present invention. -
FIG. 15 is a diagram illustrating a process of encoding information that indicates an inter prediction method determined by the process shown inFIG. 14 . -
FIG. 16 is a diagram illustrating a process of decoding DCMVD indication information encoded as shown inFIG. 15 . -
FIG. 17 is a flowchart illustrating an example of an image decoding method in which motion vectors of three control points are derived using a reconstructed pixel region so as to generate a prediction block of a current block. -
FIG. 18 is a diagram illustrating a current block partitioned into multiple regions for inter prediction according to a third exemplary embodiment of the present invention. -
FIG. 19 is a flowchart illustrating an inter prediction method according to the third exemplary embodiment of the present invention. -
FIG. 20 is a diagram illustrating an example of motion estimation and motion compensation using a reconstructed pixel region. -
FIG. 21 is a flowchart illustrating a process of determining an inter prediction method according to an embodiment of the present invention. -
FIG. 22 is a diagram illustrating a process of transmitting, to an image decoding device, information that indicates an inter prediction method. -
FIG. 23 is a diagram illustrating a process of decoding information that indicates which type of inter prediction has been used. -
FIG. 24 is a flowchart illustrating a method of generating an inter prediction block by using information that indicates which type of inter prediction has been used. -
FIG. 25 is a diagram illustrating a method of reducing blocking artifacts according to a fourth exemplary embodiment of the present invention. -
FIG. 26 is a diagram illustrating a method of applying a weighted sum of a sub-block within a prediction block and a sub-block adjacent to the upper side thereof. -
FIG. 27 is a diagram illustrating a method of applying a weighted sum of a sub-block within a prediction block and a sub-block adjacent to the left side thereof. -
FIG. 28 is a flowchart illustrating a process of determining whether or not a weighted sum is applied between sub-blocks. -
FIG. 29 is a flowchart illustrating a process of encoding information that indicates whether or not a weighted sum is applied between sub-blocks. -
FIG. 30 is a flowchart illustrating a process of decoding information that indicates whether or not a weighted sum is applied between sub-blocks. -
FIGS. 31a and 31b are diagrams illustrating inter prediction using a reconstructed pixel region according to a fifth exemplary embodiment of the present invention. -
FIG. 32 is a diagram illustrating an example of a case where motion estimation is further performed on a current block on a per-sub-block basis. -
FIG. 33 is a diagram illustrating an example in which a reconstructed pixel region and a current block are partitioned on a per-sub-block basis. -
FIG. 34 is a flowchart illustrating an example of an inter prediction method using a reconstructed pixel region. -
FIG. 35 is a diagram illustrating an example in which reconstructed blocks neighboring a current block are used to partition a reconstructed pixel region into sub-blocks according to the present invention. -
FIG. 36 is a diagram illustrating an example in which reconstructed blocks neighboring a current block are used to partition a current block into multiple sub-blocks according to the present invention. -
FIG. 37 is a flowchart illustrating a method of partitioning a current block into multiple sub-blocks according to an embodiment of the present invention. -
FIG. 38 is a flowchart illustrating a method of partitioning a reconstructed region used to encode or decode a current block into multiple sub-blocks according to an embodiment of the present invention. -
FIG. 39 is a flowchart illustrating an example of an inter prediction method using the sub-blocks of the current block partitioned as shown inFIG. 36 . -
FIG. 40 is a flowchart illustrating a method of encoding information determined according to inter prediction shown inFIG. 39 . -
FIG. 41 is a flowchart illustrating an example of a method of decoding information encoded by the encoding method shown inFIG. 40 . -
FIGS. 42a and 42b are diagrams illustrating a sixth exemplary embodiment of the present invention. -
FIG. 43 is a flowchart illustrating an example of a method of determining an inter prediction mode according to the sixth exemplary embodiment of the present invention described with reference toFIGS. 42a and 42 b. -
FIG. 44 is a diagram illustrating a process of encoding information determined by the method shown inFIG. 43 . -
FIG. 45 is a diagram illustrating a process of decoding information encoded by the method shown inFIG. 44 . - The present invention may be modified in various ways and implemented by various embodiments, so that specific embodiments are shown in the drawings and will be described in detail. However, the present invention is not limited thereto, and the exemplary embodiments can be construed as including all modifications, equivalents, or substitutes in a technical concept and a technical scope of the present invention. The similar reference numerals refer to the similar elements described in the drawings.
- Terms “first”, “second”, etc. can be used to describe various elements, but the elements are not to be construed as being limited to the terms. The terms are only used to differentiate one element from other elements. For example, the “first” element may be named the “second” element without departing from the scope of the present invention, and similarly the “second” element may also be named the “first” element. The term “and/or” includes a combination of a plurality of items or any one of a plurality of terms.
- It will be understood that when an element is referred to as being “coupled” or “connected” to another element, it can be directly coupled or connected to the other element or intervening elements may be present therebetween. In contrast, it will be understood that when an element is referred to as being “directly coupled” or “directly connected” to another element, there are no intervening elements present.
- The terms used in the present specification are merely used to describe particular embodiments, and are not intended to limit the present invention. An expression used in the singular encompasses the expression of the plural, unless it has a clearly different meaning in the context. In the present specification, it will be understood that terms such as “including”, “having”, etc. are intended to indicate the existence of the features, numbers, steps, actions, elements, parts, or combinations thereof disclosed in the specification, and are not intended to preclude the possibility that one or more other features, numbers, steps, actions, elements, parts, or combinations thereof may exist or may be added.
- Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Hereinafter, the same elements in the drawings are denoted by the same reference numerals, and a repeated description of the same elements will be omitted.
-
FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present invention. - Referring to
FIG. 1 , animage encoding device 100 may include animage partitioning module 101, anintra prediction module 102, aninter prediction module 103, asubtractor 104, atransform module 105, a quantization module 106, anentropy encoding module 107, adequantization module 108, an inverse transform module 109, anadder 110, a filter module 111, and a memory 112. - The constituents shown in
FIG. 1 are independently shown so as to represent different distinctive functions in the image encoding device, which does not mean that each constituent is constituted as separated hardware or a single software constituent unit. In other words, each constituent includes each of enumerated constituents for convenience. Thus, at least two constituents of each constituent may be combined to form one constituent or one constituent may be divided into a plurality of constituents to perform each function. The embodiment where each constituent is combined and the embodiment where one constituent is divided are also included in the scope of the present invention, if not departing from the essence of the present invention. - Also, some of constituents may not be indispensable constituents performing essential functions of the present invention but be selective constituents improving only performance thereof. The present invention may be implemented by including only the indispensable constituents for implementing the essence of the present invention except the constituents used in improving performance. The structure including only the indispensable constituents except the selective constituents used in improving only performance is also included in the scope of the present invention.
- The
image partitioning module 101 may partition an input image into one or more blocks. Here, the input image may have various shapes and sizes, such as a picture, a slice, a tile, a segment, and the like. A block may mean a coding unit (CU), a prediction unit (PU), or a transform unit (TU). The partitioning may be performed on the basis of at least one among a quadtree and a binary tree. Quadtree partitioning is a method of partitioning a parent block into four child blocks of which the width and the height are half of those of the parent block. Binary tree partitioning is a method of partitioning a parent block into two child blocks of which either the width or the height is half of that of the parent block. Through the above-described partitioning based on binary tree, a block may be in a square shape as well as a non-square shape. - Hereinafter, in the embodiment of the present invention, the coding unit may mean a unit of performing encoding or a unit of performing decoding.
- The
prediction modules intra prediction module 102 performing intra prediction and theinter prediction module 103 performing inter prediction. - Whether to perform inter prediction or intra prediction on the prediction unit may be determined, and detailed information (for example, an intra prediction mode, a motion vector, a reference picture, and the like) depending on each prediction method may be determined. Here, a processing unit on which prediction is performed may be different from a processing unit in which the prediction method and the detailed content are determined. For example, the prediction method, the prediction mode, and the like may be determined on a per-prediction unit basis, and prediction may be performed on a per-transform unit basis.
- A residual value (residual block) between the generated prediction block and an original block may be input to the
transform module 105. Further, prediction mode information used for prediction, motion vector information, and the like may be encoded with the residual value by theentropy encoding module 107 and then may be transmitted to a decoder. When a particular encoding mode is used, the original block is intactly encoded and transmitted to the decoding module without generating a prediction block by theprediction module - The
intra prediction module 102 may generate a prediction block on the basis of information on a reference pixel around the current block, which is information on a pixel within a current picture. When the prediction mode of the neighboring block of the current block on which intra prediction is to be performed is inter prediction, a reference pixel included in the neighboring block to which inter prediction has been applied is replaced by a reference pixel within another neighboring block to which intra prediction has been applied. - That is, when the reference pixel is unavailable, at least one reference pixel among available reference pixels is used instead of information on the unavailable reference pixel.
- Prediction modes in intra prediction may include a directional prediction mode using the information on the reference pixel depending on a prediction direction and a non-directional prediction mode not using directivity information in performing prediction. A mode for predicting luma information may be different from a mode for predicting chroma information, and in order to predict the chroma information, intra prediction mode information used to predict the luma information or predicted luma signal information may be utilized.
- The
intra prediction module 102 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation module, and a DC filter. The AIS filter is a filter performing filtering on a reference pixel of the current block, and may adaptively determine whether to apply the filter depending on a prediction mode of a current prediction unit. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter is not applied. - When the intra prediction mode of the prediction unit is a prediction mode in which intra prediction is performed on the basis of a pixel value obtained by interpolating the reference pixel, the reference pixel interpolation module of the
intra prediction module 102 interpolates the reference pixel to generate a reference pixel at a position on a per-fraction basis. When the prediction mode of the current prediction unit is a prediction mode in which the prediction block is generated without interpolating the reference pixel, the reference pixel is not interpolated. The DC filter generates the prediction block through filtering when the prediction mode of the current block is a DC mode. - The
inter prediction module 103 generates the prediction block using a pre-reconstructed reference image stored in the memory 112 and motion information. The motion information may contain, for example, a motion vector, a reference picture index, alist 1 prediction flag, alist 0 prediction flag, and the like. - In the image encoding device, there are two typical methods of generating motion information.
- The first method is a method in which motion information (a motion vector, a reference image index, an inter prediction direction, and the like) is generated using a motion estimation process.
FIG. 2 is a diagram illustrating a method of generating motion information by using motion estimation according to the conventional technology. Motion estimation is a method of generating motion information such as, a motion vector, a reference image index, and an inter prediction direction of a current image region to be encoded using the decoded reference image after being encoded. It is possible that motion estimation is performed in the entire reference image or that in order to reduce the complexity, a search range is set and motion estimation is performed only within the search range. - The second method of generating the motion information is a method in which motion information of neighboring blocks of the current image block to be encoded is used.
-
FIG. 3 is a diagram illustrating an example of neighboring blocks that may be used to generate motion information of a current block.FIG. 3 shows an example of spatial candidate blocks A to E and a temporal candidate block COL as neighboring blocks that may be used to generate the motion information of the current block. The spatial candidate blocks A to E are present in the same image as the current block, but the temporal candidate block COL is present in an image that is different from the image to which the current block belongs. - One piece of motion information of the spatial candidate blocks A to E and the temporal candidate block COL, which neighbor the current block, may be selected as the motion information of the present block. Here, an index may be defined that indicates which block has the motion information used as the motion information of the current block. This index information also belongs to the motion information. In the image encoding device, using the above methods, the motion information may be generated and the prediction block may be generated through motion compensation.
- A residual block may be generated that includes residual value information which is a difference value between the prediction unit generated by the
prediction module transform module 105 for transform. - The
inter prediction module 103 may derive the prediction block on the basis of information on at least one picture among the previous picture and the subsequent picture of the current picture. Further, the prediction block of the current block may be derived on the basis of information on a partial region with encoding completed within the current picture. Theinter prediction module 103 according to an embodiment of the present invention may include a reference picture interpolation module, a motion prediction module, and a motion compensation module. - The reference picture interpolation module may receive reference picture information from the memory and may generate information on a pixel equal to or smaller than an integer pixel in the reference picture. In the case of a luma pixel, a DCT-based 8-tap interpolation filter having different filter coefficients may be used to generate information on a pixel equal to or smaller than an integer pixel on a per-¼ pixel basis. In the case of a chroma signal, a DCT-based 4-tap interpolation filter having different filter coefficients may be used to generate information on a pixel equal to or smaller than an integer pixel on a per-⅛ pixel basis.
- The motion prediction module may perform motion prediction on the basis of the reference picture interpolated by the reference picture interpolation module. As methods of calculating the motion vector, various methods, such as a full search-based block matching algorithm (FBMA), a three step search (TSS) algorithm, a new three-step search (NTS) algorithm, and the like, may be used. The motion vector may have a motion vector value on a per-½ or ¼ pixel basis on the basis of the interpolated pixel. The motion prediction module may predict the prediction block of the current block by using different motion prediction methods. As motion prediction methods, various methods, such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, and the like, may be used.
- The
subtractor 104 performs subtraction on the block to be currently encoded and on the prediction block generated by theintra prediction module 102 or theinter prediction module 103 so as to generate the residual block of the current block. - The
transform module 105 may transform the residual block containing residual data, using a transform method, such as DCT, DST, Karhunen-Loeve transform (KLT), and the like. Here, the transform method may be determined on the basis of the intra prediction mode of the prediction unit that is used to generate the residual block. For example, depending on the intra prediction mode, DCT may be used in the horizontal direction, and DST may be used in the vertical direction. - The quantization module 106 may quantize values transformed into a frequency domain by the
transform module 105. Quantization coefficients may vary according to a block or importance of an image. The value calculated by the quantization module 106 may be provided to thedequantization module 108 and theentropy encoding module 107. - The
transform module 105 and/or the quantization module 106 may be selectively included in theimage encoding device 100. That is, theimage encoding device 100 may perform at least one of transform and quantization on residual data of the residual block, or may encode the residual block by skipping both transform and quantization. Even though theimage encoding device 100 does not perform either transform or quantization or does not perform both transform and quantization, the block that is input to theentropy encoding module 107 is generally referred to as a transform block. Theentropy encoding module 107 entropy encodes the input data. Entropy encoding may use various encoding methods, for example, exponential Golomb coding, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). - The
entropy encoding module 107 may encode a variety of information, such as residual value coefficient information of a coding unit, block type information, prediction mode information, partitioning unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, and the like, from theprediction module entropy encoding module 107, the coefficient of the transform block may be encoded on a per-partial block basis within the transform block on the basis of various types of flags indicating a non-zero coefficient, a coefficient of which the absolute value is higher than one or two, the sign of the coefficient, and the like may be encoded. A coefficient that is not encoded only with the flags may be encoded through the absolute value of the difference between the coefficient encoded through the flag and the actual coefficient of the transform block. Thedequantization module 108 dequantizes the values quantized by the quantization module 106, and the inverse transform module 109 inversely transforms the values transformed by thetransform module 105. The residual value generated by thedequantization module 108 and the inverse transform module 109 may be combined with the prediction unit predicted through a motion estimation module included in theprediction module intra prediction module 102 such that a reconstructed block is generated. Theadder 110 adds the prediction block generated by theprediction module - The filter module 111 may include at least one of a deblocking filter, an offset correction module, and an adaptive loop filter (ALF).
- The deblocking filter may remove block distortion that occurs due to boundaries between the blocks in the reconstructed picture. In order to determine whether to perform deblocking, whether to apply the deblocking filter to the current block may be determined on the basis of the pixels included in several rows and columns in the block. When the deblocking filter is applied to the block, a strong filter or a weak filter is applied depending on required deblocking filtering strength. Further, in applying the deblocking filter, when performing horizontal direction filtering and vertical direction filtering, horizontal direction filtering and vertical direction filtering are configured to be processed in parallel.
- The offset correction module may correct an offset from the original image on a per-pixel basis with respect to the image subjected to deblocking. In order to perform offset correction on a particular picture, it is possible to use a method of separating pixels of the image into the predetermined number of regions, determining a region to be subjected to offset, and applying the offset to the determined region, or a method of applying an offset considering edge information of each pixel.
- Adaptive loop filtering (ALF) may be performed on the basis of the value obtained by comparing the filtered reconstructed image and the original image. The pixels included in the image may be divided into predetermined groups, a filter to be applied to each of the groups may be determined, and filtering may be individually performed on each group. Information on whether to apply ALF of a luma signal may be transmitted for each coding unit (CU), and the shape and the filter coefficient of the ALF filter to be applied may vary depending on each block. Also, regardless of the characteristic of the application target block, the ALF filter in the same form (fixed form) may be applied.
- The memory 112 may store the reconstructed block or picture calculated through the filter module 111, and the stored reconstructed block or picture may be provided to the
prediction module - Next, an image decoding device according to an embodiment of the present invention will be described with reference to the accompanying drawings.
FIG. 4 is a block diagram illustrating animage decoding device 400 according to an embodiment of the present invention. - Referring to
FIG. 4 , theimage decoding device 400 may include anentropy decoding module 401, adequantization module 402, aninverse transform module 403, anadder 404, afilter module 405, amemory 406, andprediction modules - When an image bitstream generated by the
image encoding device 100 is input to theimage decoding device 400, the input bitstream is decoded according to a reverse process of the process performed in theimage encoding device 100. - The
entropy decoding module 401 may perform entropy decoding according to the reverse procedure of the entropy encoding performed by theentropy encoding module 107 of theimage encoding device 100. For example, corresponding to the methods performed by the image encoder, various methods, such as exponential Golomb coding, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC), may be applied. In theentropy decoding module 401, the coefficient of the transform block may be encoded on a per-partial block basis within the transform block on the basis of various types of flags indicating a non-zero coefficient, a coefficient of which the absolute value is higher than one or two, the sign of the coefficient, and the like. A coefficient that is not represented only by the flags may be decoded through the sum of a coefficient represented through the flag and a signaled coefficient. - The
entropy decoding module 401 may decode information related to intra prediction and inter prediction performed in the encoder. Thedequantization module 402 performs dequantization on the quantized transform block to generate the transform block. This operates substantially in the same manner as thedequantization module 108 inFIG. 1 . - The
inverse transform module 403 performs inverse transform on the transform block to generate the residual block. Here, the transform method may be determined on the basis of the prediction method (inter or intra prediction), the size and/or the shape of the block, information on the intra prediction mode, and the like. This operates substantially in the same manner as the inverse transform module 109 inFIG. 1 . - The
adder 404 adds the prediction block generated by theintra prediction module 407 or theinter prediction module 408 and the residual block generated by theinverse transform module 403 so as to generate a reconstructed block. This operates substantially in the same manner as theadder 110 inFIG. 1 . - The
filter module 405 reduces various types of noises occurring in the reconstructed blocks. - The
filter module 405 may include a deblocking filter, an offset correction module, and an ALF. - From the
image encoding device 100, it is possible to receive information on whether or not the deblocking filter is applied to the block or picture and information on whether the strong filter is applied or the weak filter is applied when the deblocking filter is applied. - The deblocking filter of the
image decoding device 400 may receive information related to the deblocking filter from theimage encoding device 100, and theimage decoding device 400 may perform deblocking filtering on the block. - The offset correction module may perform offset correction on the reconstructed image on the basis of the type of offset correction, offset value information, and the like applied to the image in performing encoding.
- The ALF may be applied to the coding unit on the basis of information on whether to apply the ALF, ALF coefficient information, and the like received from the
image encoding device 100. The ALF information may be provided by being included in a particular parameter set. Thefilter module 405 operates substantially in the same manner as the filter module 111 inFIG. 1 . - The
memory 406 stores the reconstructed block generated by theadder 404. This operates substantially in the same manner as the memory 112 inFIG. 1 . - The
prediction module entropy decoding module 401 and of information on the previously decoded block or picture provided from thememory 406. - The
prediction modules intra prediction module 407 and theinter prediction module 408. Although not shown, theprediction module entropy decoding module 401, such as prediction unit information, prediction mode information of an intra prediction method, information related to motion prediction of an inter prediction method, and the like, may separate a prediction unit in a current coding unit, and may determine whether inter prediction is performed on the prediction unit or intra prediction is performed on the prediction unit. By using information required in inter prediction of the current prediction unit provided from theimage encoding device 100, theinter prediction module 408 may perform inter prediction on the current prediction unit on the basis of information included at least one picture among the previous picture and the subsequent picture of the current picture including the current prediction unit. Alternatively, inter prediction may be performed on the basis of information on some pre-reconstructed regions within the current picture including the current prediction unit. - In order to perform inter prediction, on the basis of the coding unit, it may be determined which mode among a skip mode, a merge mode, and an AMVP Mode is used for the motion prediction method of the prediction unit included in the coding unit.
- The
intra prediction module 407 generates the prediction block using the pre-reconstructed pixels positioned near the block to be currently encoded. - The
intra prediction module 407 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation module, and a DC filter. The AIS filter is a filter performing filtering on the reference pixel of the current block, and may adaptively determine whether to apply the filter depending on the prediction mode of the current prediction unit. The prediction mode of the prediction unit provided from theimage encoding device 100 and the AIS filter information may be used to perform AIS filtering on the reference pixel of the current block. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter is not applied. - When the prediction mode of the prediction unit is a prediction mode in which intra prediction is performed on the basis of a pixel value obtained by interpolating the reference pixel, the reference pixel interpolation module of the
intra prediction module 407 interpolates the reference pixel to generate a reference pixel at a position on a per-fraction basis. The generated reference pixel on a per-fraction basis may be used as a prediction pixel of a pixel within the current block. When the prediction mode of the current prediction unit is a prediction mode in which a prediction block is generated without interpolating the reference pixel, the reference pixel is not interpolated. The DC filter may generate a prediction block through filtering when the prediction mode of the current block is a DC mode. - The
intra prediction module 407 operates substantially in the same manner as theintra prediction module 102 inFIG. 1 . - The
inter prediction module 408 generates an inter prediction block using a reference picture stored in thememory 406 and motion information. Theinter prediction module 408 operates substantially in the same manner as theinter prediction module 103 inFIG. 1 . - Hereinafter, various embodiments of the present invention will be described in detail with reference to the drawings.
-
FIGS. 5a and 5b are diagrams illustrating inter prediction using a reconstructed pixel region according to a first exemplary embodiment of the present invention. - In the inter prediction using the reconstructed pixel region according to the embodiment, particularly, the motion vector of the current block may be derived using the reconstructed pixel region.
-
FIG. 5a shows acurrent block 51 to be encoded or decoded and a pre-reconstructedpixel region C 52 as a region adjacent to thecurrent block 51. Thecurrent block 51 and the reconstructedpixel region C 52 are included in thecurrent image 50. Thecurrent image 50 may be a picture, a slice, a tile, a coding tree block, a coding block, or other image regions. The reconstructedpixel region C 52 may correspond to a reconstructed pixel region after being encoded before encoding of thecurrent block 51 in terms of encoding, and may be a pre-reconstructed region before decoding of thecurrent block 51 in terms of decoding. - Before encoding or decoding of the current block, the reconstructed
pixel region C 52 neighbors thecurrent block 51, and thus theimage encoding device 100 and theimage decoding device 400 may use the same reconstructedpixel region C 52. Therefore, without encoding the motion information of thecurrent block 51 by theimage encoding device 100, the reconstructedpixel region C 52 is used such that theimage encoding device 100 and theimage decoding device 400 may generate the motion information of thecurrent block 51 and the prediction block in the same manner. -
FIG. 5b shows an example of motion estimation and motion compensation using a reconstructed pixel region. Areference image 53 shown inFIG. 5b is searched for a region matched with the reconstructedpixel region C 52 shown inFIG. 5a . When a reconstructedpixel region D 54 that is most similar to the reconstructedpixel region C 52 is determined, a displacement between aregion 56, which is at the same position as the reconstructedpixel region C 52, and the reconstructedpixel region D 54 is determined to be amotion vector 57 of the reconstructedpixel region C 52. Themotion vector 57 determined as described above is selected as the motion vector of thecurrent block 51, and aprediction block 58 of thecurrent block 51 may be derived using themotion vector 57. -
FIG. 6 is a flowchart illustrating an inter prediction method according to the first exemplary embodiment of the present invention. - Inter prediction according to the embodiment may be performed by the
inter prediction module 103 of theimage encoding device 100 or by theinter prediction module 408 of theimage decoding device 400. Reference images used in inter prediction are stored in the memory 112 of theimage encoding device 100 or in thememory 406 of theimage decoding device 400. Theinter prediction module 103 or theinter prediction module 408 may generate the prediction block of thecurrent block 51 with reference to the reference image stored in the memory 112 or in thememory 406. - Referring to
FIG. 6 , first, the reconstructedpixel region 52 is selected to be used in deriving of the motion vector of the current block to be encoded or decoded, at step S61. Next, on the basis of the reconstructedpixel region 52 and the reference image of the current block, the motion vector of the reconstructedpixel region 52 is derived at step S63. - As described above with reference to
FIG. 5b , thereference image 53 shown inFIG. 5b is searched for the region matched with the reconstructedpixel region C 52. When the reconstructedpixel region D 54 most similar to the reconstructedpixel region C 52 is determined, the displacement between theregion 56 at the same position as the reconstructedpixel region C 52 and the reconstructedpixel region D 54 is determined to be themotion vector 57 of the reconstructedpixel region C 52. - The
image encoding device 100 or theimage decoding device 400 selects themotion vector 57 of the reconstructedpixel region C 52, determined as described above, as the motion vector of thecurrent block 51 at step S65. Using thismotion vector 57, theprediction block 58 of thecurrent block 51 may be generated. - In the meantime, the reconstructed
pixel region C 52 may be in various shapes and/or sizes.FIGS. 7a to 7c are diagrams illustrating examples of reconstructed pixel regions. The letters M, N, O, and P shown inFIGS. 7a to 7c denotes pixel intervals, respectively, and it is possible that O and P have negative values assuming that the absolute values of O and P are lower than the horizontal or vertical lengths of the current block, respectively. - Also, it is possible that the reconstructed pixel regions at the upper side and the left side of the current block are used as the reconstructed
pixel region C 52 or that the two regions are combined into a single piece to be used as the reconstructedpixel region C 52. Also, it is possible that the reconstructedpixel region C 52 is used by being subjected to subsampling. In this method, only the decoded information around the current block is used to derive the motion information, and thus it is not necessary to transmit the motion information from theencoding device 100 to thedecoding device 400. - According to the embodiment of the present invention, the
decoding device 400 also performs motion estimation, so that if motion estimation is performed on the entire reference image, the complexity may extremely increase. Therefore, by transmitting the search range on a per-block basis or in the parent header or by fixing the search region to be the same in theencoding device 100 and in thedecoding device 400, computational complexity of thedecoding device 400 may be reduced. -
FIG. 8 is a flowchart illustrating a process of determining an inter prediction method according to an embodiment of the present invention. Among the inter prediction using the reconstructed pixel region according to the present invention and the conventional inter prediction method, the optimum method may be determined through rate-distortion optimization (RDO). The process shown inFIG. 8 may be performed by theimage encoding device 100. - Referring to
FIG. 8 , first, inter prediction according to the conventional method is performed to compute cost_A at step S81, and then as described above, inter prediction using the reconstructed pixel region according to the present invention is performed to compute cost_B at step S82. - Afterward, cost_A is compared with cost_B to determine which method is optimum to use, at step S83. When cost_A is lower, it is set to perform inter prediction using the conventional method at step S84. Otherwise, it is set to perform inter prediction using the reconstructed pixel region at step S85.
-
FIG. 9 is a diagram illustrating a process of encoding information that indicates an inter prediction method determined by the process shown inFIG. 8 . Hereinafter, the information indicating the inter prediction method determined by the process shown inFIG. 8 is referred to as decoder-side motion vector derivation (DMVD) indication information. The DMVD indication information or decoder-side motion vector derivation indication information may be information indicating whether inter prediction using the conventional method is performed or inter prediction using the reconstructed pixel region is performed. - Referring to
FIG. 9 , the DMVD indication information indicating the inter prediction method determined by the process shown inFIG. 8 is encoded at step S91. The DMVD indication information may be, for example, a 1-bit flag or one of several indexes. Afterward, the motion information is encoded at step S92, and the algorithm ends. - Alternatively, information indicating whether or not inter prediction using the reconstructed pixel region according to the embodiment of the present invention is used may be generated in the parent header first and then may be decoded. That is, when the information indicating whether or not inter prediction using the reconstructed pixel region is used indicates true, the DMVD indication information is encoded. When the information indicating whether or not inter prediction using the reconstructed pixel region is used indicates false, the DMVD indication information is not present within the bitstream and in this case, the current block is predicted using the conventional inter prediction.
- In the meantime, regarding the parent header, the parent header including the information that indicates whether or not inter prediction using the reconstructed pixel region is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
-
FIG. 10 is a diagram illustrating a process of decoding the DMVD indication information encoded as shown inFIG. 9 . - The
decoding device 400 decodes the DMVD indication information at step S101, decodes the motion information at step S102, and ends the algorithm. - In the case where the information indicating whether or not inter prediction using the reconstructed pixel region is used is present in the parent header of the bitstream, when the information indicating whether or not inter prediction using the reconstructed pixel region is used indicates true, the DMVD indication information is present in the bitstream. When the information indicating whether or not inter prediction using the reconstructed pixel region is used indicates false, the DMVD indication information is not present within the bitstream and in this case, the current block is predicted using the conventional inter prediction.
- Regarding the parent header, the parent header including the information that indicates whether or not inter prediction using the reconstructed pixel region is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
- Hereinafter, the second exemplary embodiment of the present invention will be described with reference to the drawings.
- In the second exemplary embodiment, the inter prediction using the reconstructed pixel region according to the first exemplary embodiment described above is applied to inter prediction using affine transformation. Specifically, in order to derive a motion vector of a control point used for inter prediction using affine transformation, a motion vector derivation method using the reconstructed pixel region is applied. Hereinafter, for convenience of description, according to the second exemplary embodiment of the present invention, inter prediction using affine transformation is simply referred to as affine inter prediction.
-
FIG. 11 is a diagram illustrating affine inter prediction. - In affine inter prediction, motion vectors at four corners of the current block to be encoded or decoded are obtained, and then the motion vectors are used to generate a prediction block. Here, the four corners of the current block may correspond to the control points.
- Referring to
FIG. 11 , a block identified by motion vectors 11-2, 11-3, 11-4, and 11-5 at the four corners (namely, the control points) of the current block (not shown) within the current image may be a prediction block 11-6 of the current block. - This affine inter prediction enables prediction of a block or image region subjected to rotation, zoom-in/zoom-out, translation, reflection, or shear deformation.
-
Equation 1 below is a general determinant of affine transformation. -
-
Equation 1 is an equation representing transform of two-dimensional coordinates, wherein (x, y) denotes original coordinates, (x′, y′) denotes destination coordinates, and a, b, c, d, e, and f denote transform parameters. - In order to apply this affine transformation to video codec, transform parameters need to be transmitted to the image decoding device, which results in enormous increase in overhead. For this reason, in the conventional video codec, affine transformation is simply applied using N reconstructed neighboring control points.
-
Equation 2 below represents a method of deriving a motion vector of an arbitrary sub-block within the current block by using two control points at the top left and the top right of the current block. -
- In
Equation 2, (x, y) denotes the position of the arbitrary sub-block within the current block, W denotes the horizontal length of the current block, (MVx, MVy) denotes the motion vector of the sub-block, (MV0x, MV0y) denotes the motion vector of the top left control point, and (MV1x, MV1y) denotes the motion vector of the top right control point. - Next,
Equation 3 below represents a method of deriving a motion vector of an arbitrary sub-block within the current block by using three control points at the top left, the top right, and the bottom left of the current block. -
- In
Equation 3, (x, y) denotes the position of the arbitrary sub-block, W and H denote the horizontal length and the vertical length of the current block, respectively, (MVx, MVy) denotes the motion vector of the sub-block within the current block, (MV0x, MV0y) denotes the motion vector of the top left control point, (MV1x, MV1y) denotes the motion vector of the top right control point, and (MV2x, MV2y) denotes the motion vector of the bottom left control point. - In the second exemplary embodiment of the present invention, in order to derive the motion vector of the control point used for affine inter prediction, the motion vector derivation method using the reconstructed pixel region according to the first exemplary embodiment is applied. Therefore, the
image encoding device 100 does not need to transmit motion vector information of multiple control points to theimage decoding device 400. -
FIGS. 12a and 12b are diagrams illustrating derivation of the motion vector of the control point by using the reconstructed pixel region according to the second exemplary embodiment of the present invention. - Referring to
FIG. 12a , acurrent block 12 a-2 to be encoded or decoded is included in acurrent image 12 a-1. Four control points for affine inter prediction are denoted by circles at four corners of thecurrent block 12 a-2 inFIG. 12a . Also, in this figure, as regions adjacent to three control points at the top left, the top right, and the bottom left, pre-reconstructed pixel regions a 12 a-3,b 12 a-4, andc 12 a-5 are shown. - In the embodiment, motion vectors of three control points at the top left, the top right, and the bottom left are derived using the reconstructed pixel regions a 12 a-3,
b 12 a-4, andc 12 a-5 as shown inFIG. 12b . However, in the case of the control point at the bottom right of the current block, a reconstructed pixel region may be not present nearby. In this case, by using asub-block d 12 a-6 in an arbitrary size, the motion vector of thesub-block d 12 a-6 obtained using the conventional inter prediction method may be set to be the motion vector of the bottom right control point of the current block. - Referring to
FIG. 12b , areference image 12 b-1 shown inFIG. 12b is searched forregions d 12 b-10,e 12 b-11, andf 12 b-12 matched with the reconstructed pixel regions a 12 a-3,b 12 a-4, andc 12 a-5, respectively, shown inFIG. 12a . The displacements from theregions 12 b-6, 12 b-7, and 12 b-8, which are at the same position as the reconstructed pixel regions a 12 a-3,b 12 a-4, andc 12 a-5, respectively, are determined to bemotion vectors 12 b-2, 12 b-3, and 12 b-4 of the reconstructed pixel regions a 12 a-3,b 12 a-4, andc 12 a-5, respectively. Themotion vectors 12 b-2, 12 b-3, and 12 b-4 determined as described above are determined to be the motion vectors of three control points at the top left, the top right, and the bottom left of thecurrent block 12 a-2. In the meantime, as the motion vector of the control point at the bottom right, the motion vector of thesub-block d 12 a-6 obtained using the conventional inter prediction method may be used. - By using the motion vectors of the four control points derived as described above, a motion vector of an arbitrary sub-block within the current block may be derived as shown in
Equation 4 below. -
- In
Equation 4, (x, y) denotes the position of the arbitrary sub-block within the current block, W and H denote the horizontal length and the vertical length of the current block, respectively, (MVx, MVy) denotes the motion vector of the sub-block within the current block, (MV0x, MV0y) denotes the motion vector of the top left control point, (MV1x, MV1y) denotes the motion vector of the top right control point, (MV2x, MV2y) denotes the motion vector of the bottom left control point, and (MV3x, MV3y) denotes the motion vector of the bottom right control point. - In the meantime, the reconstructed pixel regions a 12 a-3,
b 12 a-4, andc 12 a-5 may be in various sizes and/or shapes as described above with reference toFIGS. 7a to 7c . The size and/or the shape of thesub-block d 12 a-6 may be the same as a preset size and/or a preset shape in theencoding device 100 and thedecoding device 400. Also, on a per-block basis or through the parent header, horizontal and/or vertical size information of thesub-block d 12 a-6 may be transmitted, or size information may be transmitted on a per-exponentiation of two basis. - As described above, when motion vectors are derived from four control points, these vectors are used to derive the motion vector of the
current block 12 a-2 or the motion vector of an arbitrary sub-block within thecurrent block 12 a-2, and this derived motion vector may be used to derive the prediction block of thecurrent block 12 a-2 or the prediction block of an arbitrary sub-block within thecurrent block 12 a-2. Specifically, referring toEquation 4 above, the position of thecurrent block 12 a-2 is coordinates (0, 0), so that the motion vector of thecurrent block 12 a-2 is the motion vector (MV0x, MV0y) of the top left control point. Therefore, the prediction block of thecurrent block 12 a-2 may be obtained using the motion vector of the top left control point. When the current block is a 8×8 block and is partitioned into four 4×4 sub-blocks, the motion vector of the sub-block at the position (3,0) within the current block is obtained by substituting a value of three for the variable x inEquation 4 above, a value of zero for the variable y, and a value of eight for both variables W and H. - Next, with reference to
FIG. 13 , the inter prediction method according to the second exemplary embodiment of the present invention will be described.FIG. 13 is a flowchart illustrating the inter prediction method according to the second exemplary embodiment of the present invention. - Inter prediction according to the embodiment may be performed by the
inter prediction module 103 of theimage encoding device 100 or theinter prediction module 408 of theimage decoding device 400. Reference images used in inter prediction are stored in the memory 112 of theimage encoding device 100 or in thememory 406 of theimage decoding device 400. Theinter prediction module 103 or theinter prediction module 408 may generate the prediction block of thecurrent block 51 with reference to a reference image stored in the memory 112 or thememory 406. - Referring to
FIG. 13 , first, at least one reconstructed pixel region is selected to be used in deriving a motion vector of at least one control point of the current block to be encoded or decoded, at step S131. In the embodiment shown inFIGS. 12a and 12b , to derive motion vectors of three control points at the top left, the top right, and the bottom left of thecurrent block 12 a-2, three reconstructed pixel regions a 12 a-3,b 12 a-4, andc 12 a-5 are selected. However, without being limited thereto, to derive a motion vector of one or two control points among the three control points, one or two reconstructed pixel regions may be selected. - Next, on the basis of the at least one reconstructed pixel region selected at step S131 and the reference image of the current block, a motion vector of at least one reconstructed pixel region is derived at step S133. The
image encoding device 100 or theimage decoding device 400 selects each motion vector of the reconstructedpixel region C 52, determined as described above, as a motion vector of at least one control point of the current block at step S135. At least one motion vector selected as described above may be used to generate the prediction block of the current block. -
FIG. 14 is a flowchart illustrating a process of determining an inter prediction method according to the second exemplary embodiment of the present invention. According to the second exemplary embodiment of the present invention, among the affine inter prediction and the conventional inter prediction, the optimum method may be determined through rate-distortion optimization (RDO). The process shown inFIG. 14 may be performed by theimage encoding device 100. - Referring to
FIG. 14 , first, inter prediction is performed using the conventional method to compute cost_A at step S141, and as described above, according to the second exemplary embodiment of the present invention, affine inter prediction is performed to compute cost_B at step S142. - Afterward, cost_A is compared with cost_B to determine which method is optimum to use, at step S143. When cost_A is lower, it is set to perform inter prediction using the conventional method at step S144. Otherwise, it is set to perform affine inter prediction at step S145 according to the second exemplary embodiment of the present invention.
-
FIG. 15 is a diagram illustrating a process of encoding information that indicates an inter prediction method determined by the process shown inFIG. 14 . Hereinafter, the information indicating the inter prediction method determined by the process shown inFIG. 14 is referred to as decoder-side control point motion vector derivation (DCMVD) indication information. The DCMVD indication information or decoder-side control point motion vector derivation indication information may be information indicating whether inter prediction using the conventional method is performed or affine inter prediction according to the second exemplary embodiment of the present invention is performed. - Referring to
FIG. 15 , the DCMVD indication information indicating the inter prediction method determined by the process shown inFIG. 14 is encoded at step S151. The DCMVD indication information may be, for example, a 1-bit flag or one of several indexes. Afterward, the motion information is encoded at step S152, and the algorithm ends. - In the meantime, according to the second exemplary embodiment of the present invention, the information indicating whether or not affine inter prediction is used may be generated in the parent header first and then may be encoded. That is, according to the second exemplary embodiment of the present invention, when the information indicating whether or not affine inter prediction is used indicates true, the DCMVD indication information is encoded. According to the second exemplary embodiment of the present invention, when the information indicating whether or not affine inter prediction is used indicates false, the DCMVD indication information is not present within the bitstream, and in this case, the current block is predicted using the conventional inter prediction.
- In the meantime, regarding the parent header, the parent header including the information indicating whether or not affine inter prediction according to the present invention is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
-
FIG. 16 is a diagram illustrating a process of decoding the DCMVD indication information encoded as shown inFIG. 15 . - The
decoding device 400 decodes the DCMVD indication information at step S161, decodes the motion information at step S162, and ends the algorithm. - In the case where the information indicating whether or not affine inter prediction according to the second exemplary embodiment of the present invention is used is present in the parent header of the bitstream, when the information indicating whether or not inter prediction using the reconstructed pixel region is used indicates true, the DCMVD indication information is present in the bitstream. According to the second exemplary embodiment of the present invention, when the information indicating whether or not affine inter prediction is used indicates false, the DCMVD indication information is not present within the bitstream, and in this case, the current block is predicted using the conventional inter prediction.
- According to the second exemplary embodiment of the present invention, regarding the parent header, the parent header including the information indicating whether or not affine inter prediction is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
-
FIG. 17 is a flowchart illustrating an example of an image decoding method in which motion vectors of three control points are derived using a reconstructed pixel region so as to generate a prediction block of a current block. The process shown inFIG. 17 relates to the embodiment shown inFIGS. 12a and 12 b. - To derive motion vectors of three control points at the top left, the top right, and the bottom left of the
current block 12 a-2, three reconstructed pixel regions a 12 a-3,b 12 a-4, andc 12 a-5 are selected. However, without being limited thereto, to derive a motion vector of one or two control points among the three control points, one or two reconstructed pixel regions may be selected. - The
image decoding device 400 may determine, on the basis of the DCMVD indication information, which inter prediction is to be performed. When the DCMVD indication information indicates use of affine inter prediction according to the present invention at step S171, the motion vectors of the control points at the top left, the top right, and the bottom left of the current block are estimated and selected using the respective reconstructed pixel regions at step S172. - Afterward, the motion vector obtained by decoding the transmitted motion information in the bitstream is set to be the motion vector of the control point at the bottom right at step S173. Using affine transformation in which the motion vectors of the four control points derived through steps S172 and S173 are used, an inter prediction block of the current block is generated at step S174. When affine inter prediction is not used, the prediction block of the current block is generated at step S175 according to the conventional inter prediction in which the motion information is decoded and the decoded motion information is used.
-
FIG. 18 is a diagram illustrating a current block partitioned into multiple regions for inter prediction according to a third exemplary embodiment of the present invention. -
FIG. 18 shows acurrent block 500 to be encoded or decoded and a pre-reconstructedpixel region C 503 as a region adjacent to thecurrent block 500. Thecurrent block 500 is partitioned into region A 500-a and region B 500-b. - Due to correlation between pixels, the pixels within the reconstructed
pixel region C 503 is likely to be similar to the pixels included in the region A 500-a, but is unlikely to be similar to the pixels included in the region B 500-b. Therefore, in inter prediction on the region A 500-a, motion estimation and motion compensation using the reconstructedpixel region C 503 are performed to find accurate motion while preventing increase in overhead. In the meantime, as the inter prediction method for the region B 500-b, the conventional inter prediction may be applied. -
FIG. 19 is a flowchart illustrating an inter prediction method according to a third exemplary embodiment of the present invention. - Inter prediction according to the embodiment may be performed by the
inter prediction module 103 of theimage encoding device 100 or by theinter prediction module 408 of theimage decoding device 400. The reference images used in inter prediction are stored in the memory 112 of theimage encoding device 100 or thememory 406 of theimage decoding device 400. Theinter prediction module 103 or theinter prediction module 408 may generate, with reference to the reference image stored in the memory 112 or thememory 406, the prediction block of the region A 500-a and the prediction block of the region B 500-b within the current block. - First, as shown in
FIG. 18 , the current block to be encoded or decoded is partitioned into multiple regions including a first region and a second region at step S51. Here, the first region and the second region may correspond to the region A 500-a and the region B 500-b shown inFIG. 5 , respectively. Thecurrent block 500 shown inFIG. 18 is partitioned into two regions, the region A 500-a and the region B 500-b, but may be partitioned into three or more regions and may be partitioned into regions in various sizes and/or shapes. - Next, using different inter prediction methods, a prediction block of the first region and a prediction block of the second region are obtained at step S53. Here, the inter prediction method for the region A 500-a may be, as described above, the method in which motion estimation and motion compensation using the reconstructed
pixel region C 503 are performed. As the inter prediction method for the region B 500-b, the conventional inter prediction may be applied. - As in the embodiment, a method in which the current block is partitioned into multiple regions and the prediction blocks of the respective regions are derived using different inter prediction methods is referred to as a mixed inter prediction.
-
FIG. 20 is a diagram illustrating an example of motion estimation and motion compensation using a reconstructed pixel region. - Referring to
FIG. 20 , areference image 600 is searched for a region matched with the reconstructedpixel region C 503 shown inFIG. 18 . As shown inFIG. 20 , when a reconstructedpixel region D 603 that is most similar to the reconstructedpixel region C 503 is determined, a displacement between aregion 601, which is at the same position as the reconstructedpixel region C 503, and the reconstructedpixel region D 603 is selected as themotion vector 605 of the region A 500-a. - That is, the
motion vector 605 estimated using the reconstructedpixel region C 503 is selected as the motion vector of the region A 500-a of the current block. Using themotion vector 605, the prediction block of the region A 500-a is generated. - In the meantime, as shown in
FIGS. 7a to 7c , the reconstructedpixel region C 503 may be in various shapes and/or sizes. Also, it is possible that the upper and left sides of the reconstructed pixel region are separated for use. Also, it is possible that the reconstructed pixel region is used by being subjected to subsampling. In this method, only the decoded information around the current block is used to derive the motion information, and thus it is not necessary to transmit the motion information from theencoding device 100 to thedecoding device 400. - According to the embodiment of the present invention, the
decoding device 400 also performs motion estimation, so that if motion estimation is performed on the entire reference image, the complexity may extremely increase. Therefore, by transmitting the search range on a per-block basis or in the parent header or by fixing the search range to be the same in theencoding device 100 and in thedecoding device 400, computational complexity of thedecoding device 400 may be reduced. - In the meantime, when estimating and encoding the motion vector of the region B 500-b shown in
FIG. 18 , the motion information of the decoded block within the reconstructedpixel region C 503 is used to predict the motion vector of the region B 500-b and the residual vector corresponding to the difference between the motion vector of the region B 500-b and the prediction motion vector is encoded. - Alternatively, it is possible that the
motion vector 605 estimated as the motion vector of the region A 500-a is used to predict the motion vector of the region B 500-b and the residual vector is encoded. - Alternatively, it is possible that the motion vector of the decoded block within the reconstructed
pixel region C 503 and the estimatedmotion vector 605 of the region A 500-a are used to constitute a motion vector prediction set, the motion vector of the region B 500-b is predicted, and the residual vector is encoded. - Alternatively, it is possible that among the blocks adjacent to the current block, motion information is taken from a preset position to perform block merging. Here, block merging means that neighboring motion information is intactly applied to a block to be encoded. Here, it is also possible that after setting several preset positions, an index indicating at which position block merging is performed is used.
- Further, possibly, the size of the region B 500-b is encoded by the
encoding device 100 to be transmitted to the decoder on a per-block basis or through the parent header, or uses the same preset value or ratio in theencoding device 100 and thedecoding device 400. -
FIG. 21 is a flowchart illustrating a process of determining an inter prediction method according to an embodiment of the present invention. Among the mixed inter prediction according to the present invention and the conventional inter prediction method, the optimum method may be determined through rate-distortion optimization (RDO). The process shown inFIG. 21 may be performed by theimage encoding device 100. - Referring to
FIG. 21 , first, inter prediction is performed using the conventional method to compute cost_A at step S801, and as described above, the mixed inter prediction according to the present invention, the current block is partitioned into at least two regions to be subjected to inter prediction individually, is performed to compute cost_B at step S802. - Afterward, cost_A and cost_B are computed to determine which method is optimum to use, at step S803. When cost_A is lower, it is set to perform the inter prediction using the conventional method at step S804. Otherwise, it is set to perform the mixed inter prediction at step S805.
-
FIG. 22 is a diagram illustrating a process of transmitting, to thedecoding device 400, information that indicates the inter prediction method determined by the process shown inFIG. 21 . - Information indicating which type of inter prediction has been used for the block to be currently encoded is encoded at step S901. This information may be, for example, a 1-bit flag or one of several indexes. Afterward, the motion information is encoded at step S902, and the algorithm ends.
- Alternatively, the information indicating whether or not the mixed inter prediction according to the embodiment of the present invention is used may be generated in the parent header first and then may be encoded. That is, when in the parent header, the information indicating whether or not the mixed inter prediction is used indicates true, the information indicating which type of inter prediction has been used for the block to be currently encoded is encoded. When the information indicating whether or not the mixed inter prediction is used indicates false, the information indicating which type of inter prediction has been used is not present within the bitstream, and in this case, the current block is not partitioned into multiple regions and the current block is predicted using the conventional inter prediction.
- In the meantime, regarding the parent header, the parent header including the information indicating whether or not the mixed inter prediction is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
-
FIG. 23 is a diagram illustrating a process of decoding information encoded in the manner shown inFIG. 21 , namely, the information indicating which type of inter prediction has been used for the block to be currently encoded. - The
decoding device 400 decodes the information indicating which type of inter prediction has been used for the block to be currently encoded, at step S1001, decodes the motion information at step S1002, and ends the algorithm. - In the case where the information indicating whether or not the mixed inter prediction is used is present in the parent header of the bitstream, when the information indicating whether or not the mixed inter prediction is used indicates true, the information indicating which type of inter prediction has been used for the block to be currently encoded is present in the bitstream. When the information indicating whether or not the mixed inter prediction is used indicates false, the information indicating which type of inter prediction has been used is not present within the bitstream, and in this case, the current block is not partitioned into multiple regions and the current block is predicted using the conventional inter prediction.
- The parent header including the information indicating whether or not the mixed inter prediction is used may be a block header, a slice header, a tile header, a picture header, or a sequence header. In the parent header, the information indicating whether or not the mixed inter prediction is used may be transmitted by being included in a block header, a slice header, a tile header, a picture header, or a sequence header.
-
FIG. 24 is a flowchart illustrating a method of generating an inter prediction block by using the information indicating which type of inter prediction has been used. The method shown inFIG. 24 may be performed by thedecoding device 400. - First, it is determined at step S1101 whether or not the information indicating which type of inter prediction has been used indicates use of the mixed inter prediction. When the mixed inter prediction is used for the current block to be decoded, the current block is partitioned into multiple regions at step S1102. For example, the current block may be partitioned into the region A 500-a and the region B 500-b as shown in
FIG. 5 . - Here, it is possible that that the size of each region resulting from the partitioning is signaled from the
encoding device 100 to thedecoding device 400 on a per-block basis or through the parent header or is set to a preset value. - Afterward, according to the method shown in
FIG. 20 , a motion vector of a first region, for example, the region A 500-a, is estimated, and a prediction block is generated at step S1103. - Next, regarding a second region, for example, the region B 500-b, the decoded motion vector is used to generate a prediction block at step S1104, and the algorithm ends.
- When the information indicating which type of inter prediction has been used indicates that the mixed inter prediction is not used or when the information, included in the parent header, indicating whether or not the mixed inter prediction is used indicates false, the conventional inter prediction is applied as the prediction method of the
current block 500. That is, the decoded motion information is used to generate the prediction block of thecurrent block 500 at step S1105, and the algorithm ends. The size of the prediction block is the same as the size of thecurrent block 500 to be decoded. - Hereinafter, the fourth exemplary embodiment of the present invention will be described with reference to the drawings. The fourth exemplary embodiment relates to a method to reduce blocking artifacts that may occur at the boundary of the block when the mixed inter prediction according to the third exemplary embodiment is performed.
-
FIG. 25 is a diagram illustrating a method of reducing blocking artifacts that may occur when the mixed inter prediction according to the present invention is performed. -
Prediction block 1 andprediction block 2 shown inFIG. 25 may correspond to the prediction block of the region A 500-a and the prediction block of the region B 500-b shown inFIG. 18 , respectively. - To summarize the fourth exemplary embodiment of the present invention, first, the regions positioned at the boundaries of the prediction block are partitioned into sub-blocks in a predetermined size. Afterward, the motion information of the sub-block around the sub-block of the prediction block is applied to the sub-block of the prediction block so that a new prediction block is generated. Afterward, a weighted sum of the sub-block of the prediction block and the new prediction block is obtained so that the final sub-block of the prediction block is generated. This is referred to as overlapped block motion compensation (OBMC).
- Referring to
FIG. 25 , in the case of a sub-block P2 present in theprediction block 1 inFIG. 25 , the motion information of the neighboring sub-block A2 is applied to the sub-block P2 to generate a new prediction block of the sub-block P2, and then the weighted sum is applied as shown inFIG. 26 to generate the final sub prediction block. - For convenience of description, it is assumed that the size of each sub-block shown in
FIG. 25 is 4×4; that there are theprediction block 1, eight sub-blocks A1 to A8 adjacent to the upper side thereof, and eight sub-blocks B1 to B8 adjacent to the left side thereof; and that there are theprediction block 2, four sub-blocks C1 to C4 adjacent to the upper side thereof, and four sub-blocks D1 to D8 adjacent to the left side thereof. - For convenience of description, although the horizontal and vertical lengths of each sub-block are assumed to be four, other various values may be encoded on a per-block basis or through the parent header and may be then signaled to the
decoding device 400. Accordingly, theencoding device 100 and thedecoding device 400 may set the size of the sub-block to be the same. Alternatively, it is possible that theencoding device 100 and thedecoding device 400 use sub-blocks in a preset same size. -
FIG. 26 is a diagram illustrating a method of applying a weighted sum of the sub-block within the prediction block and the sub-block adjacent to the upper side thereof. - Referring to
FIG. 26 , the final prediction pixel c is generated using Equation below. -
c=W1×a+(1−W1)×b [Equation 5] - In addition to the prediction pixel c, the remaining 15 pixels may be computed in a manner similar to the above. P2 to P8 in
FIG. 13 are replaced by new prediction pixels to which the weighted sum is applied through the process shown inFIG. 26 . -
FIG. 27 is a diagram illustrating a method of applying a weighted sum of a sub-block within a prediction block and a sub-block adjacent to the left side thereof. The sub-blocks P9 to P15 are replaced by new prediction pixel values to which the weighted sum is applied as shown inFIG. 27 . - Referring to
FIG. 26 , the same weighting factors are applied to the pixels on a per-row basis, and referring toFIG. 27 , the same weighting factors are applied to the pixels on a per-column basis. - In the case of the sub-block P1 in
FIG. 25 , the weighted sum with the pixels within the neighboring sub-block A1 is performed as shown inFIG. 26 , and then the weighted sum with the pixels within the sub-block B1 is performed as shown inFIG. 27 , thereby obtaining final prediction values. - Also in the case of the sub-blocks P16 to P22 present in the
prediction block 2 shown inFIG. 25 , the weighted sum calculation method shown inFIG. 26 orFIG. 27 is used to obtain the final prediction values. Here, the neighboring sub-blocks used for the weighted sum are C1 to C4 or D1 to D4. - In the meantime, not only the pixel values of the sub-blocks P16 to P22 are replaced, but also the pixel values of the neighboring sub-blocks C1 to C4, D1 to D4 may be replaced by new values through the weighted sum calculation. For example, in the case of the sub-block C2, the motion information of the sub-block P17 is applied to the sub-block C2 to generate a prediction sub-block, and then the pixel values within the prediction sub-block and the pixel values of the sub-block C2 are subjected to the weighted sum so that the pixel values of the sub-block C2 to which the weighted sum is applied is generated.
-
FIG. 28 is a flowchart illustrating a process of determining whether or not the weighted sum is applied between sub-blocks at the boundary of the block, when the mixed inter prediction according to the present invention is performed. - The variable BEST_COST storing the optimum cost is initialized to the maximum value, COMBINE_MODE storing whether or not the mixed inter prediction is used is initialized to false, and WEIGHTED_SUM storing whether or not the weighted sum is used between sub-blocks is initialized to false at step S1501. Afterward, inter prediction using the conventional method is performed, and then cost_A is computed at step S1502. The mixed inter prediction is performed, and then cost_B is computed at step S1503. After comparing the two costs at step S1504, when the value of cost_A is lower, COMBINE_MODE is set to false to indicate that the mixed inter prediction is not used and BEST_COST stores cost_A at step S1505.
- When the value of cost_B is lower, COMBINE_MODE is set to true to indicate that the mixed inter prediction is used and BEST_COST stores cost_B at step S1506. Afterward, the weighted sum is applied between the sub-blocks and cost_C is computed at step S1507. After comparing BEST_COST with cost_C at step S1508, when BEST_COST is lower than cost_C, the variable WEIGHTED_SUM is set to false to indicate that the weighted sum is not applied between the sub-blocks at step S1509. Otherwise, the variable WEIGHTED_SUM is set to true to indicate that the weighted sum is applied between the sub-blocks at step S1510 and the algorithm ends.
-
FIG. 29 is a flowchart illustrating a process of encoding information determined by the method inFIG. 27 , namely, information indicating whether or not a weighted sum is applied between sub-blocks. The process shown inFIG. 29 may be performed by theimage encoding device 100. First, theencoding device 100 encodes the information indicating which type of inter prediction has been used at step S1601, and encodes the motion information at step S1602. Afterward, the information indicating whether or not the weighted sum is applied between the sub-blocks is encoded at step S1603. - When the information indicating whether or not the mixed inter prediction is used is present in the parent header of the bitstream, and when the information indicating whether or not the mixed inter prediction is used indicates true, the information indicating whether or not the weighted sum is applied between the sub-blocks is encoded and then included in the bitstream. However, when the information, included in the parent header, indicating whether or not the mixed inter prediction is used indicates false, the information indicating whether or not the weighted sum is applied between the sub-blocks is not present within the bitstream.
-
FIG. 30 is a flowchart illustrating a process of decoding information indicating whether or not a weighted sum is applied between sub-blocks. The process shown inFIG. 30 may be performed by theimage decoding device 400. First, thedecoding device 400 decodes the information indicating which type of inter prediction has been used at step S1701, and decodes the motion information at step S1702. Afterward, the information indicating whether or not the weighted sum is applied between the sub-blocks is decoded at step S1703. - When the information indicating whether or not the mixed inter prediction is used is present in the parent header of the bitstream, and when the information indicating whether or not the mixed inter prediction is used indicates true, the information indicating whether or not the weighted sum is applied between the sub-blocks is encoded and then included in the bitstream.
- However, when the information, included in the parent header, indicating whether or not the mixed inter prediction is used indicates false, the information indicating whether or not the weighted sum is applied between the sub-blocks is not present within the bitstream. In this case, it may be inferred that the information indicating whether or not the weighted sum is applied between the sub-blocks indicates that the weighted sum is not applied between the sub-blocks.
-
FIGS. 31a and 31b are diagrams illustrating inter prediction using a reconstructed pixel region according to the fifth exemplary embodiment of the present invention. In the inter prediction using the reconstructed pixel region according to the present invention, particularly, the reconstructed pixel region may be used to derive the motion vector of the current block. -
FIG. 31a shows acurrent block 252 to be encoded or decode and a pre-reconstructedpixel region C 251 as a region adjacent to thecurrent block 252. The reconstructedpixel region C 251 includes two regions, regions at the left side and the upper side of thecurrent block 252. Thecurrent block 252 and the reconstructedpixel region C 251 are included within thecurrent image 250. Thecurrent image 250 may be a picture, a slice, a tile, a coding tree block, a coding block, or other image regions. The reconstructedpixel region C 251 may correspond to a reconstructed pixel region after being encoded before encoding of thecurrent block 252 in terms of encoding, and may correspond to a pre-reconstructed pixel region before decoding of thecurrent block 252 in terms of decoding. - Before encoding or decoding of the current block, the reconstructed
pixel region C 251 neighbors thecurrent block 252, and thus theimage encoding device 100 and theimage decoding device 400 may use the same reconstructedpixel region C 251. Therefore, without encoding the motion information of thecurrent block 252 by theimage encoding device 100, the reconstructedpixel region C 251 is used such that theimage encoding device 100 and theimage decoding device 400 may generate the motion information of thecurrent block 252 and the prediction block in the same manner. -
FIG. 31b shows an example of motion estimation and motion compensation using a reconstructed pixel region. A reference image 253 shown inFIG. 31b is searched for a region matched with the reconstructedpixel region C 251 shown inFIG. 31a . When a reconstructed pixel region D 256 that is most similar to the reconstructedpixel region C 251 is determined, a displacement between a region 254, which is at the same position as the reconstructedpixel region C 251, and the reconstructed pixel region D 256 is determined to be a motion vector 257 of the reconstructedpixel region C 251. The motion vector 257 determined as described above is selected as the motion vector of thecurrent block 252, and a prediction block of thecurrent block 252 may be derived using the motion vector 257. -
FIG. 32 is a diagram illustrating an example of a case where the motion vector 257 estimated as shown inFIG. 31b is set as an initial motion vector, thecurrent block 252 is partitioned into multiple sub-blocks A to D, and then motion estimation is further performed on a per-sub-block basis. - The sub-blocks A to D may be in an arbitrary size. MV_A to MV_D shown in
FIG. 32 are the initial motion vectors of the sub-blocks A to D, respectively, and are the same as the motion vector 257 shown inFIG. 31 b. - The size of each sub-block may be encoded on a per-block basis or through the parent header and may be transmitted to the
decoding device 400. Alternatively, it is possible that theencoding device 100 and thedecoding device 400 use the same preset size value of the sub-block. - In the meantime, as shown in
FIGS. 7a to 7c , the reconstructedpixel region C 251 may be in various shapes and/or sizes. Also, it is possible that the reconstructed pixel regions at the upper side and the left side of the current block are used as the reconstructed pixel region C or that as shown inFIGS. 31a to 31b , the two regions are combined into a single piece to be used as the reconstructedpixel region C 251. Also, it is possible that the reconstructedpixel region C 251 is used by being subjected to subsampling. - Here, for convenience of description, the description is given assuming that the reconstructed
pixel region C 251 as shown inFIGS. 31a and 31b is used as a reconstructed pixel region. -
FIG. 33 is a diagram illustrating an example in which the reconstructedpixel region C 251 and the current block are partitioned on a per-sub-block basis. Referring toFIG. 33 , the reconstructedpixel region C 251 is partitioned into sub-blocks a 285,b 286,c 287, andd 288, and the current block is partitioned in tosub-blocks A 281,B 282,C 283, andD 284. - As the reconstructed pixel regions for the
sub-block A 281, the sub-blocks a 285 andc 287 may be used. As the reconstructed pixel regions for thesub-block B 282, the sub-blocks b 286 andc 287 may be used. As the reconstructed pixel regions for thesub-block C 283, the sub-blocks a 285 andd 288 may be used. As the reconstructed pixel regions for thesub-block D 284, the sub-blocks b 286 andd 288 may be used. -
FIG. 34 is a flowchart illustrating an example of an inter prediction method using a reconstructed pixel region. Referring toFIGS. 31a and 31b , the reconstructedpixel region 251 of thecurrent block 252 is set at step S291, and then thereconstructed pixel region 251 is used to perform motion estimation on the reference image 253 at step S292. As the result of the motion estimation, the motion vector 257 of the reconstructedpixel region 251 is obtained. Afterward, as shown inFIG. 33 , the reconstructed pixel region is set on a per-sub-block basis at step S293, the motion vector 257 estimated at step S292 is set as a start point, and then motion is estimated on a per-sub-block basis of the current block at step S294. -
FIG. 35 is a diagram illustrating an example of partitioning a reconstructed pixel region into sub-blocks by using reconstructed blocks neighboring a current block according to the present invention. - According to the embodiment of the present invention, the reconstructed neighboring pixel region used for prediction of the current block may be partitioned on the basis of a partitioning structure of reconstructed neighboring blocks. In other words, on the basis of at least one among the number of the reconstructed neighboring blocks, the sizes of the reconstructed neighboring blocks, the shapes of the reconstructed neighboring blocks, and the boundaries between the reconstructed neighboring blocks, the reconstructed pixel region may be partitioned.
- Referring to
FIG. 35 , there arereconstructed block 1 2101 to reconstructedblock 5 2105 around thecurrent block 2100 to be encoded or decoded. When the reconstructed pixel region is set as shown inFIG. 5a , efficiency in motion estimation may decrease due to the dramatic difference in pixel values, which may be present at each of the boundaries of thereconstructed block 1 2101 to thereconstructed block 5 2105. Therefore, as shown inFIG. 35 , it may be efficient to partition the reconstructed pixel region into sub-blocks a to e for use. Depending on how the pre-reconstructed blocks around thecurrent block 2100 are partitioned, the reconstructed pixel region shown inFIG. 35 may be partitioned. - Specifically, the number of reconstructed neighboring blocks may be considered in partitioning of the reconstructed pixel region. Referring to
FIG. 35 , two reconstructed blocks, thereconstructed block 1 2101 and thereconstructed block 2 2102, are present at the upper side of thecurrent block 2100. Three reconstructed blocks, thereconstructed block 3 2103 to thereconstructed block 5 2105, are present at the left side of thecurrent block 2100. Considering this point, the reconstructed pixel region at the upper side of thecurrent block 2100 is partitioned into two sub-blocks, the sub-blocks a and b. The reconstructed pixel region at the left side of thecurrent block 2100 is partitioned into three sub-blocks, the sub-blocks c to e. - Alternatively, the sizes of the reconstructed neighboring blocks may be considered in partitioning of the reconstructed pixel region. For example, the height of the sub-block c of the reconstructed pixel region at the left side of the
current block 2100 is the same as that of thereconstructed block 3 2103. The height of the sub-block d is the same as that of thereconstructed block 4 2104. The height of the sub-block e corresponds to a value obtained by subtracting the height of the sub-block c and the height of the sub-block d from the height of thecurrent block 2100. - Alternatively, the boundaries between the reconstructed neighboring blocks may be considered in partitioning of the reconstructed pixel region. Considering the boundary between the
reconstructed block 1 2101 and thereconstructed block 2 2102 at the upper side of thecurrent block 2100, the reconstructed pixel region at the upper side of thecurrent block 2100 is partitioned into two sub-blocks, the sub-blocks a and b. Considering the boundary between thereconstructed block 3 2103 and thereconstructed block 4 2104 and the boundary between thereconstructed block 4 2104 and thereconstructed block 5 2105 at the left side of thecurrent block 2100, the reconstructed pixel region at the left side of thecurrent block 2100 is partitioned into three sub-blocks, the sub-blocks c to e. - In the meantime, there may be various conditions with respect to which region of the sub-blocks a to e is used to perform motion estimation. For example, it is possible that motion estimation is performed using only one reconstructed pixel region having the largest area, or it is possible that m reconstructed pixel regions from the top and n reconstructed pixel regions from the left side are selected according to the priority and used for motion estimation. Alternatively, it is also possible that a filter such as a low-pass filter is applied between the sub-blocks a to e to relieve the dramatic difference in pixel values and then one
reconstructed pixel region 251 as shown inFIG. 5a is used. -
FIG. 36 is a diagram illustrating an example of partitioning a current block into multiple sub-blocks by using reconstructed blocks neighboring the current block according to the present invention. - The method of partitioning the current block shown in
FIG. 36 into multiple sub-blocks is similar to the method of partitioning the reconstructed pixel region shown inFIG. 35 . That is, the current block to be encoded or decoded may be partitioned on the basis of a partitioning structure of reconstructed neighboring blocks. In other words, on the basis of at least one among the number of the reconstructed neighboring blocks, the sizes of the reconstructed neighboring blocks, the shapes of the reconstructed neighboring blocks, and the boundaries between the reconstructed neighboring blocks, the current block may be partitioned. - The current block shown in
FIG. 36 is partitioned into multiple sub-blocks A to F. Inter prediction may be performed on a per-sub-block basis, wherein the sub-blocks result from the partitioning. Here, inter prediction may be performed using reconstructed regions a and c inFIG. 10 for the sub-block A, using reconstructed regions b and c for the sub-block B, using reconstructed regions a and d for the sub-block C, using reconstructed regions b and d for the sub-block D, using reconstructed regions a and e for the sub-block E, and using reconstructed regions b and e for the sub-block F. - Alternatively, it is possible that priority is set depending on the sizes of the sub-blocks and the reconstructed pixel regions. For example, in the case of the sub-block A shown in
FIG. 36 , because the length is longer than the height, it is possible that the reconstructed region a has priority over the reconstructed region c and inter prediction is performed only using the reconstructed region a. Alternatively, conversely, it is possible that the reconstructed region c has priority depending on the situation, such as image characteristics, and the like. -
FIG. 37 is a flowchart illustrating a method of partitioning a current block into multiple sub-blocks according to an embodiment of the present invention. - Referring to
FIG. 37 , first, on the basis of the neighboring blocks of the current block to be encoded or decoded, the current block is partitioned into multiple sub-blocks at step S2201. The neighboring blocks of the current block are pre-reconstructed blocks as shown inFIG. 36 . As described above referring toFIG. 36 , the current block to be encoded or decoded may be partitioned on the basis of a partitioning structure of reconstructed neighboring blocks. That is, on the basis of at least one among the number of the reconstructed neighboring blocks, the sizes of the reconstructed neighboring blocks, the shapes of the reconstructed neighboring blocks, and the boundaries between the reconstructed neighboring blocks, the current block may be partitioned. - Next, multiple sub-blocks within the current block are encoded or decoded at step S2203. According to the embodiment of the present invention, as described above, each of the sub-blocks A to F of the current block shown in
FIG. 36 may be encoded or decoded using inter prediction. Here, inter prediction may be performed using reconstructed regions a and c inFIG. 35 for the sub-block A, using reconstructed regions b and c for the sub-block B, using reconstructed regions a and d for the sub-block C, using reconstructed regions b and d for the sub-block D, using reconstructed regions a and e for the sub-block E, and using reconstructed regions b and e for the sub-block F. Information related to inter prediction, such as sub block information indicating whether or not partitioning into sub-blocks is performed, which is obtained by performing inter prediction on each of the sub-blocks A to F, motion information, or the like, may be encoded or decoded. - The method shown in
FIG. 37 may be performed by theinter prediction module 103 of theimage encoding device 100 or by theinter prediction module 408 of theimage decoding device 400. The reference images used in inter prediction are stored in the memory 112 of theimage encoding device 100 or in thememory 406 of theimage decoding device 400. Theinter prediction module 103 or theinter prediction module 408 may generate, with reference to the reference image stored in the memory 112 or thememory 406, the prediction block of thecurrent block 51. -
FIG. 38 is a flowchart illustrating a method of partitioning a reconstructed region used in encoding or decoding of a current block into multiple sub-blocks according to an embodiment of the present invention. - Referring to
FIG. 13 , first, on the basis of the neighboring blocks of the current block to be encoded or decoded, the pre-reconstructed pixel region is partitioned into multiple sub-blocks at step S2211. As described above referring toFIG. 35 and/orFIG. 36 , the reconstructed neighboring pixel region used for prediction of the current block may be partitioned on the basis of a partitioning structure of reconstructed neighboring blocks. In other words, on the basis of at least one among the number of the reconstructed neighboring blocks, the sizes of the reconstructed neighboring blocks, the shapes of the reconstructed neighboring blocks, and the boundaries between the reconstructed neighboring blocks, the reconstructed pixel region may be partitioned. - Next, using at least one sub-block included in the reconstructed pixel region, at least one among the multiple sub-blocks within the current block is encoded or decoded at step S2213. For example, as described above referring to
FIG. 36 , inter prediction may be performed using reconstructed regions a and c inFIG. 35 for the sub-block A, using reconstructed regions b and c for the sub-block B, using reconstructed regions a and d for the sub-block C, using reconstructed regions b and d for the sub-block D, using reconstructed regions a and e for the sub-block E, and using reconstructed regions b and e for the sub-block F. Information related to inter prediction, such as sub block information indicating whether or not partitioning into sub-blocks is performed, which is obtained by performing inter prediction on each of the sub-blocks A to F, motion information, or the like, may be encoded or decoded. - The method shown in
FIG. 38 may be performed by theinter prediction module 103 of theimage encoding device 100 or by theinter prediction module 408 of theimage decoding device 400. The reference images used in inter prediction is stored in the memory 112 of theimage encoding device 100 or in thememory 406 of theimage decoding device 400. Theinter prediction module 103 or theinter prediction module 408 may generate, with reference to the reference image stored in the memory 112 or in thememory 406, the prediction block of thecurrent block 51. -
FIG. 39 is a flowchart illustrating an example of an inter prediction method using the sub-blocks of the partitioned current block as shown inFIG. 36 . The method shown inFIG. 39 may be performed by theinter prediction module 103 of theimage encoding device 100. - First, two variables used in this method, DMVD indication information and SUB_BLOCK will be described. The decoder-side motion vector derivation (DMVD) indication information or decoder-side motion vector derivation indication information is information indicating whether the inter prediction using the conventional method is performed or the above-described inter prediction using the reconstructed pixel region according to the present invention is performed. When the DMVD indication information indicates false, it indicates that the inter prediction using the conventional method is performed. When the DMVD indication information indicates true, it indicates that the inter prediction using the reconstructed pixel region according to the present invention is performed.
- The variable SUB_BLOCK indicates whether or not the current block is partitioned into sub-blocks. When the value of SUB_BLOCK indicates false, it indicates that the current block is not partitioned into sub-blocks. Conversely, when the value of SUB_BLOCK indicates true, it indicates that the current block is partitioned into sub-blocks.
- Referring to
FIG. 39 , first, the variable DMVD indication information, which indicates whether or not the inter prediction using the reconstructed pixel region is performed, is set to false and the variable SUB_BLOCK, which indicates whether or not partitioning into sub-blocks is performed, is set to false, and then inter prediction is performed on the current block and cost_1 is computed at step S2301. - Afterward, SUB_BLOCK is set to true and inter prediction is performed, and then cost_2 is computed at step S2302. Next, the DMVD indication information is set to true and SUB_BLOCK is set to false, and then inter prediction is performed and cost_3 is computed at step S2303. Last, the DMVD indication information and SUB_BLOCK are set to true, and then inter prediction is performed and cost_4 is computed at step S2304. The calculated cost_1 to cost_4 are compared with each other, and then the optimum inter prediction method is determined. The DMVD indication information and the SUB_BLOCK information related to the determined optimum inter prediction method are stored, and then the algorithm ends.
-
FIG. 40 is a flowchart illustrating a method of encoding information determined according to inter prediction shown inFIG. 39 . The encoding method inFIG. 40 may be performed by theimage encoding device 100. - In
FIG. 36 , the total number of sub-blocks of the current block is set to six, so that the variable BLOCK NUM, which indicates the total number of sub-blocks to be encoded, is initialized to six and the variable BLOCK INDEX, which indicates the index of the sub-block to be encoded, is initialized to zero at step S2401. Here, the current block is partitioned into the sub-blocks on the basis of the reconstructed blocks around the current block, so that it is not necessary to encode the number of the sub-blocks. Theimage decoding device 400 partitions the current block into the sub-blocks in the same manner as theimage encoding device 100, so that theimage decoding device 400 is capable of determining the number of the sub-blocks that may be present in the current block. - After step S2401, SUB_BLOCK, the information indicating whether or not the current block is partitioned into sub-blocks, is encoded at step S2402. Whether or not the current block is partitioned into the sub-blocks is determined at step S2403, and when the partitioning into the sub-blocks is not performed, the value of the variable BLOCK NUM is changed into one at step S2404.
- Afterward, the DMVD indication information indicating whether or not the inter prediction using the reconstructed pixel region has been used is encoded at step S2405. Whether or not the inter prediction using the reconstructed pixel region has been used is determined at step S2406, and when the inter prediction using the reconstructed pixel region has not been used, the motion information is encoded at step S2407. Conversely, when the inter prediction using the reconstructed pixel region has been used, the value of BLOCK INDEX is increased at step S2408 and is compared with the variable BLOCK NUM at step S2409. When the value of BLOCK INDEX is the same as the value of BLOCK NUM, this means that there is no more sub-block to be encoded in the current block, so that the algorithm ends. When the two values differ, proceeding to the subsequent sub-block to be encoded, which is present within the current block, takes place and then the process repeats from step S2406.
-
FIG. 41 is a flowchart illustrating an example of a method of decoding information encoded by the encoding method shown inFIG. 40 . InFIG. 36 , the total number of sub-blocks of the current block is set to six, so that the variable BLOCK NUM, which indicates the total number of sub-blocks to be decoded, is initialized to six and the variable BLOCK INDEX, which indicates the index of the sub-block to be decoded, is initialized to zero at step S2501. As described above, on the basis of the reconstructed blocks around the current block, theimage decoding device 400 and theimage encoding device 100 partition the current block into the sub-blocks in the same manner, so that the information indicating the number of the sub-blocks does not need to be transmitted to theimage decoding device 400. Theimage decoding device 400 may determine by itself, on the basis of the reconstructed blocks around the current block, the number of the sub-blocks that may be present in the current block. - After step S2501, SUB BLOCK, the information indicating whether or not the current block is partitioned into sub-block, is decoded at step S2502. Whether or not the current block is partitioned into the sub-blocks is determined at step S2403, and when the partitioning into the sub-blocks is not performed, the value of the variable BLOCK NUM is changed into one at step S2404.
- Afterward, the DMVD indication information indicating whether or not the inter prediction using the reconstructed pixel region has been used is decoded at step S2505. Whether or not the inter prediction using the reconstructed pixel region has been used is determined at step S2506, and when the inter prediction using the reconstructed pixel region has not been used, the motion information is decoded at step S2507. Conversely, when the inter prediction using the reconstructed pixel region has been used, the value of BLOCK INDEX is increased at step S2508 and is compared with the variable BLOCK NUM at step S2509. When the value of BLOCK INDEX is the same as the value of BLOCK NUM, this means that there is no more sub-block to be decoded in the current block, so that the algorithm ends. When the two values differ, proceeding to the subsequent sub-block to be decoded, which is present within the current block, takes place and then the process repeats from step S2506.
- Hereinafter, the sixth exemplary embodiment of the present invention will be described with reference to the drawings.
-
FIGS. 42a and 42b are diagrams illustrating the sixth exemplary embodiment of the present invention. - As shown in
FIGS. 42a and 42b , assuming thatreconstructed block 1 2601 to reconstructedblock 6 2606 are present around acurrent block 2600, the reconstructed pixel region may be partitioned into sub-blocks a to f according to the method shown inFIG. 36 . According to the method shown inFIG. 37 , thecurrent block 2600 may be partitioned into sub-blocks A to I. - Here, the sub-blocks F, G, H, and I are spaced apart from the reconstructed pixel region rather than being in contact therewith, so that inter prediction using the reconstructed pixel region may be inaccurate. Therefore, in the case of the sub-blocks F, G, H, and I, the conventional inter prediction is performed, and only in the case of the sub-blocks A to E, inter prediction using the reconstructed pixel region may be used.
- When inter prediction using reconstructed pixel region is performed on the sub-blocks A to E, inter prediction is performed using the reconstructed pixel region adjacent to each sub-block. For example, inter prediction may be performed using reconstructed pixel region b for the sub-block B, using reconstructed pixel region c for the sub-block C, using reconstructed pixel region e for the sub-block D, and using reconstructed pixel region f for the sub-block E. In the case of the sub-block A, according to preset priority, inter prediction may be performed using either the reconstructed pixel region a or d, or using the reconstructed pixel regions a and d.
- Alternatively, possibly, an index indicating which reconstructed pixel region is used for each sub-block when inter prediction using the reconstructed pixel region is performed on the sub-blocks A to E, is encoded. For example, among the reconstructed pixel regions a to f, the reconstructed pixel region b may be used to perform inter prediction on the sub-block A. In the case of the sub-block E, inter prediction may be performed using the reconstructed pixel region c. In this case, according to the horizontal or vertical size of each of the reconstructed pixel regions a to f, the number of, the positions of pixels in each region, and the like, the priority is determined and indexes are assigned.
- In the case of the sub-blocks F to I, encoding or decoding may be possible by performing the conventional inter prediction. Alternatively, as shown in
FIG. 42b , the sub-blocks F to I may be integrated into one and may be encoded or decoded using the conventional inter prediction. -
FIG. 43 is a flowchart illustrating an example of a method of determining an inter prediction mode according to the sixth exemplary embodiment of the present invention described with reference toFIGS. 42a and 42b . For convenience of description, in this example, as shown inFIG. 42b , it is assumed that as the neighboring reconstructed blocks, thereconstructed block 1 2601 to thereconstructed block 6 2606 are present, and that the reconstructed pixel region is partitioned into sub-reconstructed pixel regions a to f. Further, it is assumed that thecurrent block 2600 is partitioned into the sub-blocks A to F. Here, the sub-block F is the one into which the sub-blocks F to I inFIG. 42a are integrated. - Further, a case in which the index indicating which sub-reconstructed region is used when performing inter prediction using the reconstructed pixel region is encoded, will be described as an example. As an example, it will be described that the sub-block F is encoded or decoded by performing the conventional inter prediction. The description is given assuming that among the sub-blocks within the current block, the sub-block F is encoded or decoded last.
- Referring to
FIG. 43 , first, the current block is subjected to inter prediction without being partitioned into sub-blocks, and then cost_1 is computed at step S2701. - Afterward, the sub-blocks A to F are subjected to inter prediction individually, and cost_A to cost_F are computed and then added up to compute cost_2 at step S2702. The computed cost_1 is compared with the computed cost_2 at step S2703. When cost_1 is lower, it is determined at step S2704 that partitioning into sub-blocks is not performed. Otherwise, it is determined at step S2705 that partitioning into sub-blocks is performed and inter prediction is performed, and the algorithm ends.
-
FIG. 44 is a diagram illustrating a process of encoding information determined by the method shown inFIG. 43 . InFIG. 42b , the total number of sub-blocks of the current block is set to six, so that the variable BLOCK_NUM, which indicates the total number of sub-blocks to be encoded, is initialized to six and the variable BLOCK INDEX, which indicates the index of the sub-block to be encoded, is initialized to zero at step S2801. Here, the current block is partitioned into the sub-blocks on the basis of the reconstructed blocks around the current block, so that it is not necessary to encode the number of the sub-blocks. Theimage decoding device 400 partitions the current block into the sub-blocks in the same manner as theimage encoding device 100, so that theimage decoding device 400 is capable of determining the number of the sub-blocks that may be present in the current block. - After step S2801, SUB BLOCK, the information indicating whether or not the current block is partitioned into sub-blocks, is encoded at step S2802. Whether or not the current block is partitioned into the sub-blocks is determined at step S2803, and when the partitioning into the sub-blocks is not performed, the value of the variable BLOCK NUM is changed into one at step S2804.
- Step S2805, at which the value of BLOCK INDEX is compared with the value of BLOCK_NUM−1, is the step of determining whether the conventional inter prediction is used for the block or the inter prediction using the reconstructed pixel region is used for the block. When the two values are the same, it is the last block, namely, the sub-block subjected to the conventional inter prediction, so that the motion information is encoded at step S2806. Otherwise, it is the sub-block subjected to the inter prediction using the reconstructed pixel region, so that the index indicating which sub-reconstructed region is used is encoded at step S2807. Alternatively, it is possible that this step is skipped and the same reconstructed region determined in the encoding device and the decoding device is used.
- Afterward, the index of the sub-block is increased at step S2808, and BLOCK_NUM is compared with BLOCK_INDEX to determine whether or not encoding of all the sub-blocks present in the current block is completed, at step S2809. If not, proceeding to step S2805 takes place and the algorithm continues.
-
FIG. 45 is a diagram illustrating a process of decoding information encoded by the method shown inFIG. 44 . InFIG. 42b , the total number of sub-blocks of the current block is set to six, so that the variable BLOCK_NUM, which indicates the total number of sub-blocks to be encoded, is initialized to six and the variable BLOCK_INDEX, which indicates the index of the sub-block to be encoded, is initialized to zero at step S2901. Here, the current block is partitioned into the sub-blocks on the basis of the reconstructed blocks around the current block, so that it is not necessary to encode the number of the sub-blocks. Theimage decoding device 400 partitions the current block into the sub-blocks in the same manner as theimage encoding device 100, so that theimage decoding device 400 is capable of determining the number of the sub-blocks that may be present in the current block. - After step S2901, SUB BLOCK, the information indicating whether or not the current block is partitioned into sub-blocks, is decoded at step S2902. Whether or not the current block is partitioned into the sub-blocks is determined at step S2903, and when the partitioning into the sub-blocks is not performed, the value of the variable BLOCK_NUM is changed into one at step S2904.
- Step S2905, at which the value of BLOCK_INDEX is compared with the value of BLOCK_NUM−1, is the step of determining whether the conventional inter prediction is used for the block or the inter prediction using the reconstructed pixel region is used for the block. When the two values are the same, it is the last block, namely, the sub-block subjected to the conventional inter prediction, so that the motion information is decoded at step S2906. Otherwise, it is the sub-block subjected to the inter prediction using the reconstructed pixel region, so that the index indicating which sub-reconstructed region is used is decoded at step S2907. Alternatively, it is possible that this step is skipped and the same reconstructed region determined in the encoding device and the decoding device is used. Afterward, the index of the sub-block is increased at step S2908, and BLOCK_NUM is compared with BLOCK_INDEX to determine whether or not decoding of all the sub-blocks present in the current block is completed, at step S2909. If not, proceeding to step S2905 takes place and the algorithm continues.
- Although the exemplary methods described in the present invention are represented as a series of operations for clarity of description, the order of the steps is not limited thereto. When necessary, the steps may be performed simultaneously or in a different order. In order to realize the method according to the present invention, other steps may be added to the illustrative steps, some steps may be excluded from the illustrative steps, or some steps may be excluded while additional steps may be included.
- The various embodiments of the present invention are not intended to list all possible combinations, but to illustrate representative aspects of the present invention. The matters described in the various embodiments may be applied independently or in a combination of two or more.
- Further, the various embodiments of the present invention may be implemented by hardware, firmware, software, or combinations thereof. In the case of implementation by hardware, implementation is possible by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general processors, controllers, micro controllers, microprocessors, or the like.
- The scope of the present invention includes software or machine-executable instructions (for example, an operating system, an application, firmware, a program, or the like) that cause operation according to the methods of the various embodiments to be performed on a device or a computer, and includes a non-transitory computer-readable medium storing such software or instructions to execute on a device or a computer.
- The present invention is applicable to a field of encoding or decoding an image signal.
Claims (17)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/492,779 US20220030249A1 (en) | 2017-01-16 | 2021-10-04 | Image encoding/decoding method and device |
US18/425,273 US20240244230A1 (en) | 2017-01-16 | 2024-01-29 | Image encoding/decoding method and device |
Applications Claiming Priority (9)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR20170007346 | 2017-01-16 | ||
KR20170007347 | 2017-01-16 | ||
KR10-2017-0007346 | 2017-01-16 | ||
KR10-2017-0007348 | 2017-01-16 | ||
KR20170007348 | 2017-01-16 | ||
KR10-2017-0007347 | 2017-01-16 | ||
PCT/KR2018/000750 WO2018131986A1 (en) | 2017-01-16 | 2018-01-16 | Image encoding/decoding method and device |
US201916478259A | 2019-07-16 | 2019-07-16 | |
US17/492,779 US20220030249A1 (en) | 2017-01-16 | 2021-10-04 | Image encoding/decoding method and device |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2018/000750 Continuation WO2018131986A1 (en) | 2017-01-16 | 2018-01-16 | Image encoding/decoding method and device |
US16/478,259 Continuation US20190364284A1 (en) | 2017-01-16 | 2018-01-16 | Image encoding/decoding method and device |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/425,273 Division US20240244230A1 (en) | 2017-01-16 | 2024-01-29 | Image encoding/decoding method and device |
Publications (1)
Publication Number | Publication Date |
---|---|
US20220030249A1 true US20220030249A1 (en) | 2022-01-27 |
Family
ID=62839457
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/478,259 Abandoned US20190364284A1 (en) | 2017-01-16 | 2018-01-16 | Image encoding/decoding method and device |
US17/492,779 Abandoned US20220030249A1 (en) | 2017-01-16 | 2021-10-04 | Image encoding/decoding method and device |
US18/425,273 Pending US20240244230A1 (en) | 2017-01-16 | 2024-01-29 | Image encoding/decoding method and device |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/478,259 Abandoned US20190364284A1 (en) | 2017-01-16 | 2018-01-16 | Image encoding/decoding method and device |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/425,273 Pending US20240244230A1 (en) | 2017-01-16 | 2024-01-29 | Image encoding/decoding method and device |
Country Status (3)
Country | Link |
---|---|
US (3) | US20190364284A1 (en) |
CN (7) | CN116193110A (en) |
WO (1) | WO2018131986A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US12075028B2 (en) | 2019-09-20 | 2024-08-27 | Electronics And Telecommunications Research Institute | Image encoding/decoding method and device, and recording medium storing bitstream |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106331722B (en) | 2015-07-03 | 2019-04-26 | 华为技术有限公司 | Image prediction method and relevant device |
CN117354538A (en) * | 2017-10-27 | 2024-01-05 | 松下电器(美国)知识产权公司 | Encoding device, decoding device, generating device, transmitting device, and storage medium |
US11575925B2 (en) * | 2018-03-30 | 2023-02-07 | Electronics And Telecommunications Research Institute | Image encoding/decoding method and device, and recording medium in which bitstream is stored |
EP3777179A1 (en) * | 2018-04-12 | 2021-02-17 | ARRIS Enterprises LLC | Motion information storage for video coding and signaling |
CN112740694A (en) * | 2018-09-21 | 2021-04-30 | 韩国电子通信研究院 | Method and apparatus for encoding/decoding image and recording medium for storing bitstream |
ES2972076T3 (en) | 2018-11-08 | 2024-06-11 | Guangdong Oppo Mobile Telecommunications Corp Ltd | Image signal encoding/decoding procedure and device for the same |
IL282761B2 (en) | 2018-11-08 | 2024-04-01 | Guangdong Oppo Mobile Telecommunications Corp Ltd | Method for encoding/decoding image signal, and apparatus therefor |
CN110636301B (en) * | 2019-09-18 | 2021-08-03 | 浙江大华技术股份有限公司 | Affine prediction method, computer device, and computer-readable storage medium |
WO2021056215A1 (en) * | 2019-09-24 | 2021-04-01 | 深圳市大疆创新科技有限公司 | Motion estimation method and system, and storage medium |
Citations (99)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6173078B1 (en) * | 1996-10-30 | 2001-01-09 | Matsushita Electric Industrial Co., Ltd. | Image encoder, image encoding method, image decoder, image decoding method, and data recording medium |
US6333949B1 (en) * | 1996-10-24 | 2001-12-25 | Fujitsu Limited | Video coding apparatus and decoding apparatus |
US7773828B2 (en) * | 2005-01-13 | 2010-08-10 | Olympus Imaging Corp. | Method and device for stabilizing an image by applying an affine transform based on a weighted average of motion vectors |
US20100329336A1 (en) * | 2008-02-20 | 2010-12-30 | Samsung Electronics Co., Ltd. | Method and apparatus for encoding and decoding based on inter prediction using image inpainting |
US7929610B2 (en) * | 2001-03-26 | 2011-04-19 | Sharp Kabushiki Kaisha | Methods and systems for reducing blocking artifacts with reduced complexity for spatially-scalable video coding |
US20110150095A1 (en) * | 2009-12-23 | 2011-06-23 | Electronics And Telecommunications Research Institute | Image encoding/decoding apparatus and method |
US20110194613A1 (en) * | 2010-02-11 | 2011-08-11 | Qualcomm Incorporated | Video coding with large macroblocks |
US8085846B2 (en) * | 2004-08-24 | 2011-12-27 | Thomson Licensing | Method and apparatus for decoding hybrid intra-inter coded blocks |
US8102920B2 (en) * | 2007-07-04 | 2012-01-24 | Lg Electronics Inc. | Digital broadcasting system and data processing method |
US20120106645A1 (en) * | 2009-06-26 | 2012-05-03 | Huawei Technologies Co., Ltd | Method, apparatus and device for obtaining motion information of video images and template |
US20120275522A1 (en) * | 2009-10-28 | 2012-11-01 | Sk Telecom Co., Ltd. | Method and apparatus for motion vector encoding/decoding using spatial division, and method and apparatus for image encoding/decoding using same |
US20130121407A1 (en) * | 2010-09-17 | 2013-05-16 | Nec Corporation | Video encoding device and video decoding device |
US20130216150A1 (en) * | 2010-12-08 | 2013-08-22 | Sony Corporation | Image processing device, image processing method, and program |
US20130266070A1 (en) * | 2010-07-15 | 2013-10-10 | Kazushi Sato | Image processing device and image processing metho |
US20130279586A1 (en) * | 2010-10-01 | 2013-10-24 | Kazushi Sato | Image processing device and image processing method |
US20130287093A1 (en) * | 2012-04-25 | 2013-10-31 | Nokia Corporation | Method and apparatus for video coding |
US8644395B2 (en) * | 2009-06-23 | 2014-02-04 | Acer Incorporated | Method for temporal error concealment |
US20140056350A1 (en) * | 2011-11-04 | 2014-02-27 | Panasonic Corporation | Simplifications for boundary strength derivation in deblocking |
US20140140409A1 (en) * | 2011-06-21 | 2014-05-22 | University-Industry Cooperation Group Of Kyung Hee University | Inter-prediction method and apparatus for same |
US8750378B2 (en) * | 2008-09-23 | 2014-06-10 | Qualcomm Incorporated | Offset calculation in switched interpolation filters |
US20140192884A1 (en) * | 2013-01-04 | 2014-07-10 | Canon Kabushiki Kaisha | Method and device for processing prediction information for encoding or decoding at least part of an image |
US8855206B2 (en) * | 2008-07-01 | 2014-10-07 | France Telecom | Method and device for encoding a sequence of images implementing a temporal prediction, corresponding signal, data storage medium, method and decoding device and computer program products |
US20140301463A1 (en) * | 2013-04-05 | 2014-10-09 | Nokia Corporation | Method and apparatus for video coding and decoding |
US20150063464A1 (en) * | 2013-08-30 | 2015-03-05 | Qualcomm Incorporated | Lookup table coding |
US9058659B2 (en) * | 2009-12-01 | 2015-06-16 | Humax Holdings Co., Ltd. | Methods and apparatuses for encoding/decoding high resolution images |
US9088795B2 (en) * | 2002-11-25 | 2015-07-21 | Godo Kaisha Ip Bridge 1 | Picture coding and decoding system for coding and decoding macroblocks in direct mode |
US9148667B2 (en) * | 2013-02-06 | 2015-09-29 | Qualcomm Incorporated | Intra prediction mode decision with reduced storage |
US9154803B2 (en) * | 2011-05-20 | 2015-10-06 | Kt Corporation | Method and apparatus for intra prediction within display screen |
US9165379B2 (en) * | 2010-09-02 | 2015-10-20 | Lg Electronics Inc. | Method for encoding and decoding video, and apparatus using same |
US20160029033A1 (en) * | 2013-04-05 | 2016-01-28 | Samsung Electronics Co., Ltd. | Interlayer video encoding method and apparatus and interlayer video decoding method and apparatus for compensating luminance difference |
US9288485B2 (en) * | 2010-09-17 | 2016-03-15 | Nec Corporation | Video image encoding and decoding device using adaptive pseudo random noise injection during planar mode filtering |
US9300943B2 (en) * | 2011-01-12 | 2016-03-29 | Panasonic Intellectual Property Corporation Of America | Image coding method, image decoding method, image coding apparatus, and image decoding apparatus |
US9324165B2 (en) * | 2010-07-10 | 2016-04-26 | Huawei Technologies Co., Ltd. | Method and device for generating a predicted value of an image using interpolation and motion vectors |
US9338476B2 (en) * | 2011-05-12 | 2016-05-10 | Qualcomm Incorporated | Filtering blockiness artifacts for video coding |
US9357214B2 (en) * | 2012-12-07 | 2016-05-31 | Qualcomm Incorporated | Advanced merge/skip mode and advanced motion vector prediction (AMVP) mode for 3D video |
US9357195B2 (en) * | 2012-08-16 | 2016-05-31 | Qualcomm Incorporated | Inter-view predicted motion vector for 3D video |
US20160227244A1 (en) * | 2013-09-13 | 2016-08-04 | Canon Kabushiki Kaisha | Method, apparatus and system for encoding and decoding video data |
US20160309169A1 (en) * | 2010-04-13 | 2016-10-20 | Ge Video Compression, Llc | Inter-plane prediction |
US9491459B2 (en) * | 2012-09-27 | 2016-11-08 | Qualcomm Incorporated | Base layer merge and AMVP modes for video coding |
US9491458B2 (en) * | 2012-04-12 | 2016-11-08 | Qualcomm Incorporated | Scalable video coding prediction with non-causal information |
US9491460B2 (en) * | 2013-03-29 | 2016-11-08 | Qualcomm Incorporated | Bandwidth reduction for video coding prediction |
US9491480B2 (en) * | 2010-03-08 | 2016-11-08 | Sk Telecom Co., Ltd. | Motion vector encoding/decoding method and apparatus using a motion vector resolution combination, and image encoding/decoding method and apparatus using same |
US20160330458A1 (en) * | 2013-09-03 | 2016-11-10 | Kt Corporation | Scalable video signal encoding/decoding method and apparatus |
US20160345011A1 (en) * | 2014-04-28 | 2016-11-24 | Panasonic Intellectual Property Corporation Of America | Image coding method and decoding method related to motion estimation on decoder side |
US9525872B2 (en) * | 2009-06-30 | 2016-12-20 | Qualcomm Incorporated | Video coding based on first order prediction and pre-defined second order prediction mode |
US9609324B2 (en) * | 2008-10-24 | 2017-03-28 | Sk Telecom Co., Ltd. | Image encoding/decoding method and device using coefficients of adaptive interpolation filter |
US9699450B2 (en) * | 2012-10-04 | 2017-07-04 | Qualcomm Incorporated | Inter-view predicted motion vector for 3D video |
US20170214937A1 (en) * | 2016-01-22 | 2017-07-27 | Mediatek Inc. | Apparatus of Inter Prediction for Spherical Images and Cubic Images |
US9729881B2 (en) * | 2010-05-19 | 2017-08-08 | Sk Telecom Co., Ltd. | Video encoding/decoding apparatus and method |
US9736454B2 (en) * | 2012-09-21 | 2017-08-15 | Nokia Technologies Oy | Method and apparatus for video coding |
US9749658B2 (en) * | 2013-09-30 | 2017-08-29 | Sun Patent Trust | Image coding method, image decoding method, image coding apparatus, and image decoding apparatus |
US9788015B2 (en) * | 2008-10-03 | 2017-10-10 | Velos Media, Llc | Video coding with large macroblocks |
US9807427B2 (en) * | 2010-04-13 | 2017-10-31 | Ge Video Compression, Llc | Inheritance in sample array multitree subdivision |
US9819954B2 (en) * | 2009-05-12 | 2017-11-14 | Lg Electronics Inc. | Method and apparatus for decoding a video signal |
US9819963B2 (en) * | 2011-07-12 | 2017-11-14 | Electronics And Telecommunications Research Institute | Inter prediction method and apparatus for same |
US9832467B2 (en) * | 2014-10-07 | 2017-11-28 | Qualcomm Incorporated | Deblock filtering for intra block copying |
US20170353722A1 (en) * | 2014-12-22 | 2017-12-07 | B<>Com | A method for encoding a digital image, decoding method, devices, and associated computer programs |
US20170359577A1 (en) * | 2014-10-07 | 2017-12-14 | Samsung Electronics Co., Ltd. | Method and device for encoding or decoding multi-layer image, using interlayer prediction |
US20180007379A1 (en) * | 2015-01-21 | 2018-01-04 | Samsung Electronics Co., Ltd. | Method and apparatus for decoding inter-layer video, and method and apparatus for encoding inter-layer video |
US9866863B1 (en) * | 2014-03-11 | 2018-01-09 | Google Inc. | Affine motion prediction in video coding |
US9906790B2 (en) * | 2014-03-14 | 2018-02-27 | Qualcomm Incorporated | Deblock filtering using pixel distance |
US20180063534A1 (en) * | 2016-08-25 | 2018-03-01 | Intel Corporation | Method and system of video coding with context decoding and reconstruction bypass |
US9912955B2 (en) * | 2010-04-16 | 2018-03-06 | Sk Telecom Co., Ltd. | Video encoding/decoding method using motion information candidate group for batch mode |
US9918105B2 (en) * | 2014-10-07 | 2018-03-13 | Qualcomm Incorporated | Intra BC and inter unification |
US20180131943A1 (en) * | 2015-04-27 | 2018-05-10 | Lg Electronics Inc. | Method for processing video signal and device for same |
US20180160113A1 (en) * | 2015-06-05 | 2018-06-07 | Intellectual Discovery Co., Ltd. | Method and device for encoding and decoding intra-frame prediction |
US20180176596A1 (en) * | 2015-06-05 | 2018-06-21 | Intellectual Discovery Co., Ltd. | Image encoding and decoding method and image decoding device |
US20180176601A1 (en) * | 2015-06-11 | 2018-06-21 | Intellectual Discovery Co., Ltd. | Method for encoding and decoding image using adaptive deblocking filtering, and apparatus therefor |
US10033997B2 (en) * | 2010-06-23 | 2018-07-24 | Panasonic Intellectual Property Management Co., Ltd. | Image decoding apparatus, image decoding method, integrated circuit, and program |
US20180220149A1 (en) * | 2015-08-04 | 2018-08-02 | Lg Electronics Inc. | Inter prediction method and device in video coding system |
US20180249156A1 (en) * | 2015-09-10 | 2018-08-30 | Lg Electronics Inc. | Method for processing image based on joint inter-intra prediction mode and apparatus therefor |
US10148981B2 (en) * | 2015-03-20 | 2018-12-04 | Mediatek Singapore Pte. Ltd. | Methods of palette coding with inter-prediction in video coding |
US10154271B2 (en) * | 2013-07-12 | 2018-12-11 | Samsung Electronics Co., Ltd. | Method and apparatus for inter-layer decoding video using depth-based disparity vector, and method and apparatus for inter-layer encoding video using depth-based disparity vector |
US20180359469A1 (en) * | 2015-11-24 | 2018-12-13 | Samsung Electronics Co., Ltd. | Video encoding method and apparatus, and video decoding method and apparatus |
US10182234B2 (en) * | 2013-10-14 | 2019-01-15 | Samsung Electronics Co., Ltd. | Method and apparatus for applying view synthesized prediction according to illumination compensation |
US20190028731A1 (en) * | 2016-01-07 | 2019-01-24 | Mediatek Inc. | Method and apparatus for affine inter prediction for video coding system |
US10212419B2 (en) * | 2012-10-01 | 2019-02-19 | Ge Video Compression, Llc | Scalable video coding using derivation of subblock subdivision for prediction from base layer |
US20190068967A1 (en) * | 2016-02-03 | 2019-02-28 | Sharp Kabushiki Kaisha | Predicted-image generation device, video decoding device, and video coding device |
US10250881B2 (en) * | 2010-04-16 | 2019-04-02 | Sk Telecom Co., Ltd. | Video encoding/decoding apparatus and method using batch mode together with skip mode |
US20190124332A1 (en) * | 2016-03-28 | 2019-04-25 | Lg Electronics Inc. | Inter-prediction mode based image processing method, and apparatus therefor |
US10313682B2 (en) * | 2013-08-26 | 2019-06-04 | Qualcomm Incorporated | Determining regions when performing intra block copying |
US10321151B2 (en) * | 2014-04-01 | 2019-06-11 | Mediatek Inc. | Method of adaptive interpolation filtering in video coding |
US10321162B2 (en) * | 2015-06-08 | 2019-06-11 | Qualcomm Incorporated | Derivation of color gamut scalability parameters and tables in scalable video coding |
US10321145B2 (en) * | 2014-10-07 | 2019-06-11 | Samsung Electronics Co., Ltd. | Method and device for encoding or decoding image by means of view merging prediction |
US10368084B2 (en) * | 2014-11-27 | 2019-07-30 | Kt Corporation | Video signal processing method and device |
US10375405B2 (en) * | 2012-10-05 | 2019-08-06 | Qualcomm Incorporated | Motion field upsampling for scalable coding based on high efficiency video coding |
US10491892B2 (en) * | 2010-05-26 | 2019-11-26 | Lg Electronics Inc. | Method and apparatus for processing a video signal |
US10554966B2 (en) * | 2014-10-07 | 2020-02-04 | Samsung Electronics Co., Ltd. | Multi-view image encoding/decoding method and apparatus |
US10574999B2 (en) * | 2016-05-05 | 2020-02-25 | Intel Corporation | Method and system of video coding with a multi-pass prediction mode decision pipeline |
US10582215B2 (en) * | 2015-08-07 | 2020-03-03 | Lg Electronics Inc. | Inter prediction method and apparatus in video coding system |
US10621614B2 (en) * | 2010-04-13 | 2020-04-14 | Ge Video Compression, Llc | Region merging and coding parameter reuse via merging |
US10645421B2 (en) * | 2015-12-02 | 2020-05-05 | Nec Corporation | Video encoding method, video encoding device, video decoding method, video decoding device, program, and video system |
US10666940B2 (en) * | 2014-11-06 | 2020-05-26 | Samsung Electronics Co., Ltd. | Video encoding method and apparatus, and video decoding method and apparatus |
US10687053B2 (en) * | 2015-11-19 | 2020-06-16 | Electronics And Telecommunications Research Institute | Method and apparatus for image encoding/decoding |
US10708589B2 (en) * | 2014-03-19 | 2020-07-07 | Samsung Electronics Co., Ltd. | Video encoding and decoding method and apparatus involving boundary filtering |
US10798403B2 (en) * | 2016-01-29 | 2020-10-06 | Sharp Kabushiki Kaisha | Prediction image generation device, video decoding device, and video coding device |
US10979707B2 (en) * | 2015-09-06 | 2021-04-13 | Mediatek Inc. | Method and apparatus of adaptive inter prediction in video coding |
US11153562B2 (en) * | 2015-09-14 | 2021-10-19 | Mediatek Singapore Pte. Ltd. | Method and apparatus of advanced deblocking filter in video coding |
US11234003B2 (en) * | 2016-07-26 | 2022-01-25 | Lg Electronics Inc. | Method and apparatus for intra-prediction in image coding system |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101390193B1 (en) * | 2007-01-18 | 2014-05-02 | 삼성전자주식회사 | Method and apparatus for encoding and decoding based on motion estimation |
JP4427553B2 (en) * | 2007-02-06 | 2010-03-10 | シャープ株式会社 | Video decoding device |
KR101671460B1 (en) * | 2009-09-10 | 2016-11-02 | 에스케이 텔레콤주식회사 | Motion Vector Coding Method and Apparatus and Video Coding Method and Apparatus by Using Same |
JP5762243B2 (en) * | 2011-10-18 | 2015-08-12 | 日本電信電話株式会社 | Video encoding method, apparatus, video decoding method, apparatus, and program thereof |
CN104244002B (en) * | 2013-06-14 | 2019-02-05 | 北京三星通信技术研究有限公司 | The acquisition methods and device of motion information in a kind of video coding/decoding |
KR101676791B1 (en) * | 2015-04-14 | 2016-11-16 | 삼성전자주식회사 | Method for decoding an image |
WO2016175550A1 (en) * | 2015-04-27 | 2016-11-03 | 엘지전자 주식회사 | Method for processing video signal and device for same |
CN106331722B (en) * | 2015-07-03 | 2019-04-26 | 华为技术有限公司 | Image prediction method and relevant device |
-
2018
- 2018-01-16 CN CN202310214058.5A patent/CN116193110A/en active Pending
- 2018-01-16 CN CN202310193767.XA patent/CN116170585A/en active Pending
- 2018-01-16 CN CN201880007055.1A patent/CN110178371A/en active Pending
- 2018-01-16 CN CN202310179294.8A patent/CN116233416A/en active Pending
- 2018-01-16 WO PCT/KR2018/000750 patent/WO2018131986A1/en active Application Filing
- 2018-01-16 CN CN202310188377.3A patent/CN116170584A/en active Pending
- 2018-01-16 CN CN202310188219.8A patent/CN116170583A/en active Pending
- 2018-01-16 CN CN202310213959.2A patent/CN116193109A/en active Pending
- 2018-01-16 US US16/478,259 patent/US20190364284A1/en not_active Abandoned
-
2021
- 2021-10-04 US US17/492,779 patent/US20220030249A1/en not_active Abandoned
-
2024
- 2024-01-29 US US18/425,273 patent/US20240244230A1/en active Pending
Patent Citations (102)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6333949B1 (en) * | 1996-10-24 | 2001-12-25 | Fujitsu Limited | Video coding apparatus and decoding apparatus |
US6173078B1 (en) * | 1996-10-30 | 2001-01-09 | Matsushita Electric Industrial Co., Ltd. | Image encoder, image encoding method, image decoder, image decoding method, and data recording medium |
US7929610B2 (en) * | 2001-03-26 | 2011-04-19 | Sharp Kabushiki Kaisha | Methods and systems for reducing blocking artifacts with reduced complexity for spatially-scalable video coding |
US9088795B2 (en) * | 2002-11-25 | 2015-07-21 | Godo Kaisha Ip Bridge 1 | Picture coding and decoding system for coding and decoding macroblocks in direct mode |
US8085846B2 (en) * | 2004-08-24 | 2011-12-27 | Thomson Licensing | Method and apparatus for decoding hybrid intra-inter coded blocks |
US7773828B2 (en) * | 2005-01-13 | 2010-08-10 | Olympus Imaging Corp. | Method and device for stabilizing an image by applying an affine transform based on a weighted average of motion vectors |
US8102920B2 (en) * | 2007-07-04 | 2012-01-24 | Lg Electronics Inc. | Digital broadcasting system and data processing method |
US20100329336A1 (en) * | 2008-02-20 | 2010-12-30 | Samsung Electronics Co., Ltd. | Method and apparatus for encoding and decoding based on inter prediction using image inpainting |
US8855206B2 (en) * | 2008-07-01 | 2014-10-07 | France Telecom | Method and device for encoding a sequence of images implementing a temporal prediction, corresponding signal, data storage medium, method and decoding device and computer program products |
US8750378B2 (en) * | 2008-09-23 | 2014-06-10 | Qualcomm Incorporated | Offset calculation in switched interpolation filters |
US9788015B2 (en) * | 2008-10-03 | 2017-10-10 | Velos Media, Llc | Video coding with large macroblocks |
US9609324B2 (en) * | 2008-10-24 | 2017-03-28 | Sk Telecom Co., Ltd. | Image encoding/decoding method and device using coefficients of adaptive interpolation filter |
US9819954B2 (en) * | 2009-05-12 | 2017-11-14 | Lg Electronics Inc. | Method and apparatus for decoding a video signal |
US8644395B2 (en) * | 2009-06-23 | 2014-02-04 | Acer Incorporated | Method for temporal error concealment |
US20120106645A1 (en) * | 2009-06-26 | 2012-05-03 | Huawei Technologies Co., Ltd | Method, apparatus and device for obtaining motion information of video images and template |
US9525872B2 (en) * | 2009-06-30 | 2016-12-20 | Qualcomm Incorporated | Video coding based on first order prediction and pre-defined second order prediction mode |
US20120275522A1 (en) * | 2009-10-28 | 2012-11-01 | Sk Telecom Co., Ltd. | Method and apparatus for motion vector encoding/decoding using spatial division, and method and apparatus for image encoding/decoding using same |
US9058659B2 (en) * | 2009-12-01 | 2015-06-16 | Humax Holdings Co., Ltd. | Methods and apparatuses for encoding/decoding high resolution images |
US20110150095A1 (en) * | 2009-12-23 | 2011-06-23 | Electronics And Telecommunications Research Institute | Image encoding/decoding apparatus and method |
US20110194613A1 (en) * | 2010-02-11 | 2011-08-11 | Qualcomm Incorporated | Video coding with large macroblocks |
US9491480B2 (en) * | 2010-03-08 | 2016-11-08 | Sk Telecom Co., Ltd. | Motion vector encoding/decoding method and apparatus using a motion vector resolution combination, and image encoding/decoding method and apparatus using same |
US10621614B2 (en) * | 2010-04-13 | 2020-04-14 | Ge Video Compression, Llc | Region merging and coding parameter reuse via merging |
US9807427B2 (en) * | 2010-04-13 | 2017-10-31 | Ge Video Compression, Llc | Inheritance in sample array multitree subdivision |
US20160309169A1 (en) * | 2010-04-13 | 2016-10-20 | Ge Video Compression, Llc | Inter-plane prediction |
US10250881B2 (en) * | 2010-04-16 | 2019-04-02 | Sk Telecom Co., Ltd. | Video encoding/decoding apparatus and method using batch mode together with skip mode |
US9912955B2 (en) * | 2010-04-16 | 2018-03-06 | Sk Telecom Co., Ltd. | Video encoding/decoding method using motion information candidate group for batch mode |
US9729881B2 (en) * | 2010-05-19 | 2017-08-08 | Sk Telecom Co., Ltd. | Video encoding/decoding apparatus and method |
US10491892B2 (en) * | 2010-05-26 | 2019-11-26 | Lg Electronics Inc. | Method and apparatus for processing a video signal |
US10033997B2 (en) * | 2010-06-23 | 2018-07-24 | Panasonic Intellectual Property Management Co., Ltd. | Image decoding apparatus, image decoding method, integrated circuit, and program |
US9324165B2 (en) * | 2010-07-10 | 2016-04-26 | Huawei Technologies Co., Ltd. | Method and device for generating a predicted value of an image using interpolation and motion vectors |
US20130266070A1 (en) * | 2010-07-15 | 2013-10-10 | Kazushi Sato | Image processing device and image processing metho |
US9165379B2 (en) * | 2010-09-02 | 2015-10-20 | Lg Electronics Inc. | Method for encoding and decoding video, and apparatus using same |
US20130121407A1 (en) * | 2010-09-17 | 2013-05-16 | Nec Corporation | Video encoding device and video decoding device |
US9288485B2 (en) * | 2010-09-17 | 2016-03-15 | Nec Corporation | Video image encoding and decoding device using adaptive pseudo random noise injection during planar mode filtering |
US20130279586A1 (en) * | 2010-10-01 | 2013-10-24 | Kazushi Sato | Image processing device and image processing method |
US20130216150A1 (en) * | 2010-12-08 | 2013-08-22 | Sony Corporation | Image processing device, image processing method, and program |
US9300943B2 (en) * | 2011-01-12 | 2016-03-29 | Panasonic Intellectual Property Corporation Of America | Image coding method, image decoding method, image coding apparatus, and image decoding apparatus |
US9338476B2 (en) * | 2011-05-12 | 2016-05-10 | Qualcomm Incorporated | Filtering blockiness artifacts for video coding |
US9154803B2 (en) * | 2011-05-20 | 2015-10-06 | Kt Corporation | Method and apparatus for intra prediction within display screen |
US20140140409A1 (en) * | 2011-06-21 | 2014-05-22 | University-Industry Cooperation Group Of Kyung Hee University | Inter-prediction method and apparatus for same |
US9819963B2 (en) * | 2011-07-12 | 2017-11-14 | Electronics And Telecommunications Research Institute | Inter prediction method and apparatus for same |
US20140056350A1 (en) * | 2011-11-04 | 2014-02-27 | Panasonic Corporation | Simplifications for boundary strength derivation in deblocking |
US9491458B2 (en) * | 2012-04-12 | 2016-11-08 | Qualcomm Incorporated | Scalable video coding prediction with non-causal information |
US20130287093A1 (en) * | 2012-04-25 | 2013-10-31 | Nokia Corporation | Method and apparatus for video coding |
US9357195B2 (en) * | 2012-08-16 | 2016-05-31 | Qualcomm Incorporated | Inter-view predicted motion vector for 3D video |
US9736454B2 (en) * | 2012-09-21 | 2017-08-15 | Nokia Technologies Oy | Method and apparatus for video coding |
US9491459B2 (en) * | 2012-09-27 | 2016-11-08 | Qualcomm Incorporated | Base layer merge and AMVP modes for video coding |
US10212419B2 (en) * | 2012-10-01 | 2019-02-19 | Ge Video Compression, Llc | Scalable video coding using derivation of subblock subdivision for prediction from base layer |
US9699450B2 (en) * | 2012-10-04 | 2017-07-04 | Qualcomm Incorporated | Inter-view predicted motion vector for 3D video |
US10375405B2 (en) * | 2012-10-05 | 2019-08-06 | Qualcomm Incorporated | Motion field upsampling for scalable coding based on high efficiency video coding |
US9357214B2 (en) * | 2012-12-07 | 2016-05-31 | Qualcomm Incorporated | Advanced merge/skip mode and advanced motion vector prediction (AMVP) mode for 3D video |
US20140192884A1 (en) * | 2013-01-04 | 2014-07-10 | Canon Kabushiki Kaisha | Method and device for processing prediction information for encoding or decoding at least part of an image |
US9148667B2 (en) * | 2013-02-06 | 2015-09-29 | Qualcomm Incorporated | Intra prediction mode decision with reduced storage |
US9491460B2 (en) * | 2013-03-29 | 2016-11-08 | Qualcomm Incorporated | Bandwidth reduction for video coding prediction |
US20160029033A1 (en) * | 2013-04-05 | 2016-01-28 | Samsung Electronics Co., Ltd. | Interlayer video encoding method and apparatus and interlayer video decoding method and apparatus for compensating luminance difference |
US20140301463A1 (en) * | 2013-04-05 | 2014-10-09 | Nokia Corporation | Method and apparatus for video coding and decoding |
US10334263B2 (en) * | 2013-04-05 | 2019-06-25 | Samsung Electronics Co., Ltd. | Interlayer video encoding method and apparatus and interlayer video decoding method and apparatus for compensating luminance difference |
US10154271B2 (en) * | 2013-07-12 | 2018-12-11 | Samsung Electronics Co., Ltd. | Method and apparatus for inter-layer decoding video using depth-based disparity vector, and method and apparatus for inter-layer encoding video using depth-based disparity vector |
US10313682B2 (en) * | 2013-08-26 | 2019-06-04 | Qualcomm Incorporated | Determining regions when performing intra block copying |
US20150063464A1 (en) * | 2013-08-30 | 2015-03-05 | Qualcomm Incorporated | Lookup table coding |
US20160330458A1 (en) * | 2013-09-03 | 2016-11-10 | Kt Corporation | Scalable video signal encoding/decoding method and apparatus |
US20160227244A1 (en) * | 2013-09-13 | 2016-08-04 | Canon Kabushiki Kaisha | Method, apparatus and system for encoding and decoding video data |
US9749658B2 (en) * | 2013-09-30 | 2017-08-29 | Sun Patent Trust | Image coding method, image decoding method, image coding apparatus, and image decoding apparatus |
US10182234B2 (en) * | 2013-10-14 | 2019-01-15 | Samsung Electronics Co., Ltd. | Method and apparatus for applying view synthesized prediction according to illumination compensation |
US9866863B1 (en) * | 2014-03-11 | 2018-01-09 | Google Inc. | Affine motion prediction in video coding |
US9906790B2 (en) * | 2014-03-14 | 2018-02-27 | Qualcomm Incorporated | Deblock filtering using pixel distance |
US10708589B2 (en) * | 2014-03-19 | 2020-07-07 | Samsung Electronics Co., Ltd. | Video encoding and decoding method and apparatus involving boundary filtering |
US10321151B2 (en) * | 2014-04-01 | 2019-06-11 | Mediatek Inc. | Method of adaptive interpolation filtering in video coding |
US20160345011A1 (en) * | 2014-04-28 | 2016-11-24 | Panasonic Intellectual Property Corporation Of America | Image coding method and decoding method related to motion estimation on decoder side |
US10554966B2 (en) * | 2014-10-07 | 2020-02-04 | Samsung Electronics Co., Ltd. | Multi-view image encoding/decoding method and apparatus |
US9832467B2 (en) * | 2014-10-07 | 2017-11-28 | Qualcomm Incorporated | Deblock filtering for intra block copying |
US10321145B2 (en) * | 2014-10-07 | 2019-06-11 | Samsung Electronics Co., Ltd. | Method and device for encoding or decoding image by means of view merging prediction |
US9918105B2 (en) * | 2014-10-07 | 2018-03-13 | Qualcomm Incorporated | Intra BC and inter unification |
US20170359577A1 (en) * | 2014-10-07 | 2017-12-14 | Samsung Electronics Co., Ltd. | Method and device for encoding or decoding multi-layer image, using interlayer prediction |
US10666940B2 (en) * | 2014-11-06 | 2020-05-26 | Samsung Electronics Co., Ltd. | Video encoding method and apparatus, and video decoding method and apparatus |
US10368084B2 (en) * | 2014-11-27 | 2019-07-30 | Kt Corporation | Video signal processing method and device |
US20170353722A1 (en) * | 2014-12-22 | 2017-12-07 | B<>Com | A method for encoding a digital image, decoding method, devices, and associated computer programs |
US20180007379A1 (en) * | 2015-01-21 | 2018-01-04 | Samsung Electronics Co., Ltd. | Method and apparatus for decoding inter-layer video, and method and apparatus for encoding inter-layer video |
US10148981B2 (en) * | 2015-03-20 | 2018-12-04 | Mediatek Singapore Pte. Ltd. | Methods of palette coding with inter-prediction in video coding |
US20180131943A1 (en) * | 2015-04-27 | 2018-05-10 | Lg Electronics Inc. | Method for processing video signal and device for same |
US20180160113A1 (en) * | 2015-06-05 | 2018-06-07 | Intellectual Discovery Co., Ltd. | Method and device for encoding and decoding intra-frame prediction |
US20180176596A1 (en) * | 2015-06-05 | 2018-06-21 | Intellectual Discovery Co., Ltd. | Image encoding and decoding method and image decoding device |
US10321162B2 (en) * | 2015-06-08 | 2019-06-11 | Qualcomm Incorporated | Derivation of color gamut scalability parameters and tables in scalable video coding |
US20180176601A1 (en) * | 2015-06-11 | 2018-06-21 | Intellectual Discovery Co., Ltd. | Method for encoding and decoding image using adaptive deblocking filtering, and apparatus therefor |
US20180220149A1 (en) * | 2015-08-04 | 2018-08-02 | Lg Electronics Inc. | Inter prediction method and device in video coding system |
US10728571B2 (en) * | 2015-08-04 | 2020-07-28 | Lg Electronics Inc. | Inter prediction method and device in video coding system |
US10582215B2 (en) * | 2015-08-07 | 2020-03-03 | Lg Electronics Inc. | Inter prediction method and apparatus in video coding system |
US10979707B2 (en) * | 2015-09-06 | 2021-04-13 | Mediatek Inc. | Method and apparatus of adaptive inter prediction in video coding |
US20180249156A1 (en) * | 2015-09-10 | 2018-08-30 | Lg Electronics Inc. | Method for processing image based on joint inter-intra prediction mode and apparatus therefor |
US11153562B2 (en) * | 2015-09-14 | 2021-10-19 | Mediatek Singapore Pte. Ltd. | Method and apparatus of advanced deblocking filter in video coding |
US10687053B2 (en) * | 2015-11-19 | 2020-06-16 | Electronics And Telecommunications Research Institute | Method and apparatus for image encoding/decoding |
US20180359469A1 (en) * | 2015-11-24 | 2018-12-13 | Samsung Electronics Co., Ltd. | Video encoding method and apparatus, and video decoding method and apparatus |
US10645421B2 (en) * | 2015-12-02 | 2020-05-05 | Nec Corporation | Video encoding method, video encoding device, video decoding method, video decoding device, program, and video system |
US20190158870A1 (en) * | 2016-01-07 | 2019-05-23 | Mediatek Inc. | Method and apparatus for affine merge mode prediction for video coding system |
US20190028731A1 (en) * | 2016-01-07 | 2019-01-24 | Mediatek Inc. | Method and apparatus for affine inter prediction for video coding system |
US20170214937A1 (en) * | 2016-01-22 | 2017-07-27 | Mediatek Inc. | Apparatus of Inter Prediction for Spherical Images and Cubic Images |
US10798403B2 (en) * | 2016-01-29 | 2020-10-06 | Sharp Kabushiki Kaisha | Prediction image generation device, video decoding device, and video coding device |
US20190068967A1 (en) * | 2016-02-03 | 2019-02-28 | Sharp Kabushiki Kaisha | Predicted-image generation device, video decoding device, and video coding device |
US20190124332A1 (en) * | 2016-03-28 | 2019-04-25 | Lg Electronics Inc. | Inter-prediction mode based image processing method, and apparatus therefor |
US10574999B2 (en) * | 2016-05-05 | 2020-02-25 | Intel Corporation | Method and system of video coding with a multi-pass prediction mode decision pipeline |
US11234003B2 (en) * | 2016-07-26 | 2022-01-25 | Lg Electronics Inc. | Method and apparatus for intra-prediction in image coding system |
US20180063534A1 (en) * | 2016-08-25 | 2018-03-01 | Intel Corporation | Method and system of video coding with context decoding and reconstruction bypass |
Non-Patent Citations (1)
Title |
---|
Huang et al. "Control-Point Representation and Differential Coding Affine-Motion Compensation", IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, No. 10, Oct. 2013 (Year: 2013) * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US12075028B2 (en) | 2019-09-20 | 2024-08-27 | Electronics And Telecommunications Research Institute | Image encoding/decoding method and device, and recording medium storing bitstream |
Also Published As
Publication number | Publication date |
---|---|
CN116170585A (en) | 2023-05-26 |
CN116170584A (en) | 2023-05-26 |
CN116233416A (en) | 2023-06-06 |
CN116193109A (en) | 2023-05-30 |
US20190364284A1 (en) | 2019-11-28 |
WO2018131986A1 (en) | 2018-07-19 |
CN116170583A (en) | 2023-05-26 |
CN116193110A (en) | 2023-05-30 |
CN110178371A (en) | 2019-08-27 |
US20240244230A1 (en) | 2024-07-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220030249A1 (en) | Image encoding/decoding method and device | |
US12096002B2 (en) | Method and apparatus for encoding/decoding an image | |
US11368690B2 (en) | Method for decoding video signal by deriving reference sample for intra prediction | |
US11533470B2 (en) | Method and apparatus for encoding/decoding an image signal | |
JP6025726B2 (en) | Video decoding device | |
US8165195B2 (en) | Method of and apparatus for video intraprediction encoding/decoding | |
US20190104304A1 (en) | Method and apparatus for processing intra-prediction-based video signal | |
KR102502216B1 (en) | Method and apparatus for encoding/decoding an image | |
US11659178B2 (en) | Image encoding method/device, image decoding method/device, and recording medium in which bitstream is stored | |
US20240015286A1 (en) | Image encoding method/device, image decoding method/device and recording medium having bitstream stored thereon | |
US20210168398A1 (en) | Video coding method and device using merge candidate | |
US10742971B2 (en) | Inter prediction method and device that performs prediction by applying weights to motion information of a current block | |
US20190335173A1 (en) | Image encoding method/device, image decoding method/device and recording medium having bitstream stored therein | |
US20220272361A1 (en) | Method and apparatus for encoding/decoding image signal | |
KR20240124260A (en) | Method and apparatus for encoding/decoding an image | |
US11909970B2 (en) | Image encoding method/device, image decoding method/device, and recording medium in which bitstream is stored | |
US20220086461A1 (en) | Image encoding/decoding method and apparatus | |
KR102009634B1 (en) | Method and apparatus for coding image compensation information and decoding using the same | |
KR102435445B1 (en) | Method and apparatus for encoding/decoding a video signal | |
KR20200076807A (en) | A method and an apparatus for processing a block divided into sub-block units |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |