KR102152144B1 - Method Of Fast And High Efficiency Video Codec Image Coding Based On Object Information Using Machine Learning - Google Patents

Abstract

The present invention relates to a high-speed, high-efficiency video codec image encoding method based on object information using machine learning.
Here, the method of encoding a high-speed, high-efficiency video codec image based on object information using machine learning of the present invention includes the step (A) in which a video image generator captures a certain area over time and generates different video images over time;
(B) step of extracting object information of the learned object by machine learning the specific object by the object information providing unit;
The object region extraction unit divides the image image at regular intervals from the image image generator, receives the image image in which a plurality of first blocks are formed, receives the object information from the object information provider, and compares the stored reference object and object information. , (C) extracting the unmatched object as a non-learning object and extracting the matched object as a learning object;
(D) step of dividing a plurality of first blocks included in a learning object into a plurality of learning object sub-blocks, and encoding the learning object sub-blocks by giving weights to the learning object sub-blocks; And
The non-learning object encoding unit includes step (E) encoding the first block included in the non-learning object.

Description

Method Of Fast And High Efficiency Video Codec Image Coding Based On Object Information Using Machine Learning

The present invention relates to a technique for quickly and efficiently encoding an image image using an object learned by machine learning.

Machine learning methods are used to classify various objects from one image and to extract classified object information. More specifically, the machine learning method detects people, cars, bicycles, cars, etc. from a single image and classifies them according to objects.

Machine learning classifies and detects objects, learns about objects, and can more accurately classify and detect objects based on the learned data. Typically, deep learning technology is an example of a machine learning method.

Currently, machine learning technology is used as an application that can detect illegal parking enforcement, illegal garbage dumping, and product defects in that it is coupled to a camera or video input device to extract a specific object from an image.

However, the machine learning technology developed up to now is proceeding unnecessary classification and unnecessary encoding in the process of classifying and encoding objects. This increases the complexity of machine learning and lowers the coding efficiency.

Korean Patent Registration 10-1851099 (announcement date 2018.04.20)

Accordingly, the problem to be solved by the present invention is to solve this problem, and the present invention increases the encoding efficiency of an image based on object information learned through machine learning, and reduces the complexity of image encoding. .

The problem to be solved of the present invention is not limited to the problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In the object information-based high-speed and high-efficiency video codec image encoding method using machine learning of the present invention for achieving the above-described problem, an image image generator generates different image images according to time by photographing a certain area according to time. Step (A);

(B) step of extracting object information of the learned object by machine learning the specific object by the object information providing unit;

The object region extracting unit divides the image image at regular intervals from the image image generator, receives an image image in which a plurality of first blocks are formed, receives the object information from the object information provider, and stores the reference object and the (C) step of extracting an unmatched object as a non-learning object and extracting the matched object as a learning object in preparation for object information;

(D) step of dividing a plurality of first blocks included in the learning object into a plurality of learning object sub-blocks, and encoding the learning object sub-blocks by giving weights to the learning object sub-blocks; And

And (E) encoding the first block included in the non-learning object by the non-learning object encoding unit.

In the step (C), when the object region extracting unit extracts the learning object, when extracting the learning object, the step of setting the learning object region larger than the size of the learning object may be further included.

The object information-based high-speed, high-efficiency video codec image encoding method using machine learning according to the present invention classifies learning objects and non-learning objects from image images based on objects learned through machine learning, and differentiates learning and non-learning objects from each other. It is encoded in the encoding process.

That is, according to the present invention, different encodings are performed on one image image, and encoding efficiency of the image image may be improved. In addition, according to the present invention, different block division processes are performed on one video image, the divided blocks are encoded, and the encoding speed for the video image can be increased.

1 is a block diagram of a high-speed, high-efficiency video codec image encoding system based on object information using machine learning according to an embodiment of the present invention.
2 is a diagram showing an image image captured by an image image generator.
3 is a diagram showing an example of a device in which an image image generating unit and an object information providing unit are combined.
4 is a diagram illustrating a state in which the object region extraction unit of FIG. 1 processes the second image image of FIG. 2.
5 is a diagram showing a process in which the first block of FIG. 4 is divided into learning object sub-blocks.
6 is a diagram illustrating an encoding process of a learning object encoder in a high-speed, high-efficiency video codec image encoding system based on object information using machine learning.
7 is a diagram showing an area of a learning object included in an image image by an object area extraction unit.
8 is a diagram showing weights of a learning object area.
9 is a diagram showing a process of dividing a continuous image according to time by a learning object encoding unit and a non-learning object encoding unit.
10 is a diagram showing a method of early termination of a coding unit division structure using a SKIP mode.
11 is a diagram showing a method of ending a coding unit division structure limitation.
12 is a flowchart of a method for ending a motion prediction search range limitation.
13 is a diagram illustrating a method of searching for advanced motion vector prediction (AMVP).
14 is a flowchart illustrating a method of encoding a high-speed, high-efficiency video codec image based on object information using machine learning according to an embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings.

In this specification, before describing the object information-based high-speed high-efficiency video codec image encoding method using machine learning, the object information-based high-speed high-efficiency video codec image encoding method using machine learning is performed so that the description of the present invention may be concise and clear. A high-speed, high-efficiency video codec image encoding system based on object information using machine learning will be described first.

Accordingly, all descriptions of the object information-based high-speed high-efficiency video codec image encoding system using machine learning described throughout the specification can be directly applied to the object information-based high-speed high-efficiency video codec image encoding method using machine learning.

In addition, the steps of the method described above in this specification are each step performed by a computer. Therefore, the object information-based high-speed high-efficiency video codec image encoding method using machine learning of the present invention is a high-speed high-efficiency video codec image encoding based on object information using machine learning that performs the object information-based high-speed high-efficiency video codec image encoding method using machine learning. It can be processed in a system ie computer.

Hereinafter, a high-speed, high-efficiency video codec image encoding system based on object information using machine learning according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 11. And based on this, a method of encoding a high-speed, high-efficiency video codec image based on object information using machine learning will be described in detail with reference to FIG. 12.

First, a high-speed, high-efficiency video codec image encoding system based on object information using machine learning according to an embodiment of the present invention will be described in detail with reference to FIG. 1.

High-speed and high-efficiency video codec video encoding system based on object information using machine learning (1) classifies learning and non-learning objects from image images based on objects learned through machine learning, and encodes learning and non-learning objects differently. It is encoded as a process.

That is, according to the present invention, different encodings are performed on one image image, and encoding efficiency of the image image may be improved. In addition, according to the present invention, different block division processes are performed on one video image, the divided blocks are encoded, and the encoding speed for the video image can be increased.

The object information-based high-speed and high-efficiency video codec image encoding system 1 using machine learning includes an image image generator 10, an object information providing unit 20, an object region extraction unit 30, and a learning object encoding unit 40. And a non-learning object encoding unit 50.

In addition, the object information-based high-speed and high-efficiency video codec image encoding system 1 using machine learning may include an encoding end determination unit 60.

The image image generating unit 10 photographs a certain area according to time and generates different image images according to time. Such an image image generator 10 may be a camera capable of generating an image image by photographing a certain area in which an object is located. As an example, the image image generation unit 10 photographs a certain area including a plurality of objects matched with the object information, and, as shown in FIG. 2A, the first image 110 and the FIG. 2 As shown in (b), the second image image 120 may be generated. At this time, the plurality of objects may be a car object, a bicycle object, and a dog object corresponding to the object information machine-learned by the object information providing unit 20. Several such objects may exist in a block in a block-based encoding codec. In this case, object information about the object, for example, flag information, may be included in a bitstream for coding and transmitting an image.

The object information providing unit 20 extracts an object included in an image (or image frame) image, performs machine learning, and extracts object information from the learned object.

The object information providing unit 20 may be installed in the image image generating unit 10. In other words, the image image generating unit 10 and the object information providing unit 20 may be formed as an integrated device as one object. For example, as shown in FIG. 3, the object information providing unit 20 for machine learning may be formed by a CCTV camera installed in the image generating unit 10.

In addition, the object information providing unit 20, as shown in Figure 3, before generating the image frame in the image image generator 10, as an example of various specific objects, to machine learning objects such as bicycles, cars, dogs, etc. I can. In this case, the object information providing unit 20 may machine learn the object through a machine learning technology such as a deep learning technology. In addition, the object information providing unit 20 makes it possible to use object information necessary for encoding an image image.

The object information providing unit 20 may detect an object learned from an image image captured through the image image generator 10 and extract information, that is, object information from the object.

The image image generating unit 10 transmits the generated image image and the object information providing unit 20 to the object region extracting unit 30 the extracted object information.

The object region extracting unit 30 divides the image image at regular intervals as shown in FIG. 4 to receive an image image in which a plurality of first blocks are formed.

The object region extraction unit 30 compares the stored reference object, that is, machine-learned object information and object information extracted from an image image. In this case, the object region extraction unit 30 extracts the unmatched object as a non-learning object Ba1, and extracts the matched object as a learning object Aa1, Ab1, Ac1.

The learning object encoding unit 40 is an example of a learning object extracted from the object region extraction unit 30, and divides the first block Aa11 included in the puppy learning object Aa1 into a plurality of learning object sub-blocks Aa21. can do.

Here, the learning object sub-block Aa21 has a second block Aa12 whose horizontal and vertical length is 1/2 of the first block and the horizontal and vertical length of the second block as shown in FIG. It may be a third block Aa13 of /2, and a fourth block Aa14 of 1/2 of the horizontal and vertical lengths of the third block.

The learning object encoding unit 40 may give the fourth block a weight greater than that of the third block, the third block with a weight greater than the second block, and the second block with a weight greater than that of the first block. In addition, the weighted block can be encoded. For example, the learning object encoding unit 40 increases and assigns a weight of 1 to a block formed by dividing the first block from the first block, and assigns a weight of 0 to the first block, a weight of 1 to the second block, and a third block. A weight 2 may be applied to the block and a weight 3 may be applied to the fourth block. And each block can be coded. In other words, the learning object encoding unit 40 forms the third block and the fourth block to which the learning object is detected by being divided in a large amount from the first block to form the third block and the fourth block to which the large weight is assigned, and encodes the divided blocks.

More specifically, when performing encoding, the learning object encoder 40 divides the image into a unit of a largest coding unit (LCU), which is a basic unit of a coding unit (CU), and performs encoding. Perform. Here, the coding unit (CU) plays a similar role to a macroblock (MB: Macro Block, hereinafter'MB'), which is a basic block in the existing video codec H.264 /AVC. However, unlike a macroblock having a fixed size of 16x16, the coding unit may be variably sized. In addition, the maximum coding unit (LCU) may be divided into several coding units (CU) having a size smaller than that of the maximum coding unit in order to efficiently encode an image. The maximum coding unit having a size of 64x64 may be divided into a plurality of coding units (CUs) in various ways. The maximum coding unit having a size of 64x64 may be divided into a plurality of coding units as shown in FIG. 5.

Hereinafter, a process in which the first block is divided into learning object sub-blocks will be described with reference to FIG. 5. Here, ① of FIG. 5 represents a first block Aa11 included in the puppy learning object Aa1 of FIG. 4. As shown in ② of FIG. 5, the first block Aa11 may be divided into coding units (CUs) having a size of 32x32 with a maximum coding unit of 1 division depth. As shown in ④, ⑧, and ⑫ of FIG. 5, the 32x32-sized coding units (CUs) may be divided into 16x16-sized coding units (CUs) having a split depth of 2 and a 32x32-sized coding unit (CU). . In addition, the coding units (CUs) having a size of 16x16 may be divided into coding units (CUs) having a division depth of 3 having a size of 8x8 as shown in ⑥ and ⑩ of FIG. 5.

The maximum coding unit (LCU) may be divided into a plurality of coding units (CU) as described above. The split structure of the LCU may be split information of the coding unit. The learning object encoding unit 40 generates various LCU division structures, stores them in the LCU division structure candidate, and determines the optimal LCU division structure. It is possible to select one of the LCU split structure candidates as the optimal LCU split structure in units of the coding unit (LCU).

The selection of a coding unit (CU) candidate is determined by a rate-distortion optimization method, and through this, a partition structure having the best coding efficiency is determined.

The division structure of the maximum coding unit (LCU) is based on the division structure of the maximum coding unit (LCU) according to the characteristics of an image in units of the maximum coding unit (LCU), and encoding efficiency can be improved by performing encoding. In addition, the encoding process of the learning object encoding unit may appear as shown in FIG. 6.

The encoding process of the learning object encoder may be an HEVC video codec encoding process. In the HEVC video codec encoding process, as shown in FIG. 6, a block image is received and coding units and structures, inter prediction, interpolation, filtering, and transformation are performed. can do.

The non-learning object encoding unit 50 may encode a first block included in the non-learning object extracted by the object region extracting unit 30. In addition, the non-learning object encoding unit 50 overlaps the first video image 110 and the second video image 120 so that the learning object of the second video image is in the region included in the non-learning object of the first video image. In the case of overlapping, that is, when the learning object moves and enters the non-learning object, the first block Aa11 of the learning object of the second image image can be divided into the size of the first block or less like the learning object encoding unit. That is, the non-learning object encoding unit 50 may divide the first block into second blocks whose horizontal and vertical lengths of the first block are 1/2. In particular, since the non-learning object encoding unit 50 includes a size value of a divided block limiting the division of the first block, it can divide the first block until it corresponds to the size value of the divided block.

In addition, the non-learning object encoding unit 50 may give the second block a greater weight than the first block. For example, the non-learning object encoding unit 50 may assign a weight of 0 to the first block and a weight of 1 to the second block. The non-learning object encoding unit 50 may encode the divided blocks. As such, the non-learning object encoding unit 50 is not divided much in the first block in the area where the object is not detected and the movement of the object is small or the pixel change is small, and is divided once even if the object is divided, and a small weight is given. A second block is formed and the divided blocks are encoded.

Hereinafter, other features of the object region extraction unit will be described with reference to FIGS. 7 and 8. The object region extracting unit 30 may include the learning object and set the learning object region OA to be larger than the size of the learning objects Aa1, Ab1, and Ac1 as shown in FIG. 7. More specifically, the object region extracting unit 30 may set an area surrounded by the first block of one layer along the outer side of the learning object as the learning object region.

8A is the first step of obtaining the area of the non-learning object in the object area extracting unit. In the first block, that is, the maximum coding unit including the object area in the image image divided by the maximum coding unit (LCU) ( LCU) is assigned a weight of '2', and other LCUs, that is, the first block, are assigned a weight of '0'. (b) is the second step of obtaining the area of the non-learning object in the object area extraction unit, and the weight is set to '1' to the adjacent maximum coding unit (LCU) of the maximum coding unit (LCU) given a weight of '2'. do. Through this process, it is possible to accurately obtain the non-learning object region excluding the object region to which the weight is assigned to 2 and the adjacent region to the object region to which the weight is assigned to 1.

Hereinafter, another feature of the learning object encoding unit 40 and the non-learning object encoding unit 50 will be described with reference to FIG. 9.

9A and 9B are image images taken with a time difference. This video image is a continuous video coded with a high-speed and high-efficiency video codec on a time axis, and shows an example of a coding unit (CU) division structure thereof. The coding unit (CU) split structure shows characteristics determined according to the characteristics of an image. More specifically, the division structure of the coding unit shows a marked difference in the divisional structure of the maximum coding unit (LCU) in a region with a lot of motion or complexities such as an object or its boundary compared to the divisional structure in a non-learning object region such as a background. .

As shown in Fig. 9, the maximum coding unit (LCU) is located in the area where the unlearned object is located as shown in (1), and unlike the maximum coding unit (LCU) in the area where the learning object is located as shown in (2). Since there is little change in pixels, the coding is performed by being divided into coding units (CU) having a relatively large size. As an example, even in the area where the learning object is located as shown in (2), if a part of the object with the same pixel moves, the moving part due to the small change in the pixel is divided into less by the LCU and encoding can be performed. have.

In other words, through the comparison of the maximum coding unit (LCU) of (1) and (2) of FIGS. 9A and 9B, the division structure for the LCU at the same location in time has considerable similarity. Can be seen. This point can be good information that can predict the shape of the partition structure in advance in determining the coding unit (CU) partition structure.

In particular, this object information allows the maximum coding unit (LCU) located in the non-learning object area to be expected to have a partition structure in a form that has a large size and a large coding unit (CU), and the location is the same in time. It can be expected that the partition structure will be similar if the object information is referred to the LCU of.

Therefore, by performing encoding on all coding units (CU) using such object information, it is possible to predict the partition structure according to the object and background region in the conventional method of determining the partition structure, thereby reducing the coding complexity. have.

In the above-described maximum coding unit and coding unit split structure, a method of early termination of a coding unit split structure using a SKIP mode, a coding unit split structure restriction method, and a motion prediction search range restriction method may be applied.

First, a description will be given of the application of the above-described maximum coding unit and coding unit split structure to the method of early termination of the coding unit split structure using the SKIP mode as shown in FIG. 10.

In the method of early termination of the coding unit division structure using the SKIP mode, the block in the non-learning object region has a large change in motion, so the coding is performed in the SKIP mode. In this method, when the current coding unit (CU) is in the background, when the optimal coding unit (CU) is determined in the SKIP mode of 2Nx2N size during the coding process, no further coding unit (CU) division is performed. It ends. Through this method, the coding speed of the block can be improved by reducing the computational complexity of the division of the block.

In more detail, a method of early termination of the SKIP mode-using coding unit division structure may be described, and the method of early termination of the SKIP mode-using coding unit division structure may be performed in a series of seven steps.

First, a first step of determining whether an area of the current coding unit CU is a non-learning object area, and determining an optimal prediction unit (PU) mode starts with a series of steps. After the first step, a second step of determining whether the current coding unit CU is the smallest coding unit CU is performed. At this time, if the coding unit is not the smallest coding unit, the third step is performed. On the other hand, when the coding unit is the coding unit of the smallest size, step 6 is performed.

The process proceeds to the third step, and if the coding unit is a coding unit belonging to the non-learning object region, the fourth step is performed. If not, proceed to step 5.

The process proceeds to the fourth step, and it is determined whether the current coding unit is determined as the SKIP mode, and if it is determined as the SKIP mode, the sixth step is performed.

When proceeding to the fifth step, the first step is performed by dividing into four coding units having a size of half the width and half the length of the current coding unit.

When the division of the coding unit is finished, the sixth step of encoding the coding unit of the current size and storing it in the coding unit division structure candidate proceeds. Thereafter, the seventh, which selects the most efficient split structure of the CU in terms of image quality and bit quantity through rate-distortion optimization among coding unit (CU) candidates currently stored in the maximum coding unit (LCU). Go through the steps. Then, the series of steps is finished with the seventh step.

In addition, the coding unit division structure as shown in FIG. 9 may be applied to the coding unit division structure limitation method as shown in FIG. 11.

The coding unit partitioning structure restriction method is based on the fact that the block of the non-learning object region has a structure similar to that of the coding unit at the temporal position, and if the partitioning structure of the coding unit at the temporal position is simple, the coding unit to be encoded is divided. Structure also uses simple points.

In this method, when the current coding unit corresponds to the non-learning object region, the current maximum coding unit (LCU) is also within the range if the minimum size of the maximum coding unit (LCU) in temporal correlation is not less than the size of the set coding unit. It ends without performing the following coding unit division.

Hereinafter, a method of limiting the coding unit division structure will be described in more detail. However, in order to concise and clarify the description of the method of restricting the divisional structure of the coding unit, the size of the coding unit is set to 32x32.

The method of limiting the coding unit division structure can proceed a series of steps in 8 steps.

First, it starts with a first step of determining whether the region of the current coding unit is a non-object region, and determining an optimal prediction unit mode. After the first step, a second step of determining whether the current coding unit is a coding unit having the smallest size is performed. In this case, if the coding unit is not the smallest size, step 3 is performed, and if the coding unit is the smallest size, step 7 is performed.

In the third step, it is determined whether the current coding unit belongs to the non-learning object area. In this case, if the current coding unit belongs to the non-learning object region, step 4 is performed.

In step 4, it is determined whether the size of the current coding unit is less than 32x32, and if the size of the coding unit is less than 32x32, step 5 is performed, and if the size of the coding unit is greater than or equal to 32x32, step 6 is performed. .

In the fifth step, it is determined whether the size of the minimum coding unit (CU) of the maximum coding unit (LCU) co-located from the reference frame is greater than or equal to 32x32. At this time, if the size of the maximum coding unit is greater than or equal to 32x32, step 7 is performed, and if the size of the maximum coding unit is less than 32x32, step 6 is performed.

In the sixth step, the current coding unit is divided into four coding units having a horizontal half and a vertical half, and the first step is performed.

On the other hand, in the seventh step, the current size is encoded and stored in the partition structure candidate of the coding unit, and the eighth step is performed.

In the eighth step, the most efficient partitioning structure of the coding unit in terms of image quality and bit amount is selected through a rate-distortion optimization method among coding unit candidates currently stored in the LCU. This eighth step ends a series of steps. In addition, the coding unit division structure as shown in FIG. 9 can be applied to the method of limiting the motion prediction search range as shown in FIG. 12.

The motion prediction search range limitation termination method is, for example, a method of limiting the motion prediction search range to 1/2 compared to the existing motion prediction search range when motion prediction is performed in a process of encoding a non-learning object region.

The method of limiting the motion prediction search range proceeds in the order of processing as shown in FIG. 12. In particular, the method of limiting the motion prediction search range is to set the motion prediction range to a value of 1/2 compared to the previous one when the current prediction unit (PU) performs motion search, and when the current coding unit corresponds to the background area in the video image. , The motion prediction search range is set to 32, which is 1/2 from the set value 64, and in other cases, the existing method is followed. Such a motion prediction search range limitation method can reduce computational complexity compared to conventional methods. Here, the existing method may be Advanced Motion Vector Prediction (AMVP) and Merge.

Among these existing methods, Advanced Motion Vector Prediction (AMVP), as shown in FIG. 13, uses a diamond search method at positions in the motion search area from the reference frame to determine the position where the block closest to the current PU exists. It is a method of finding an optimal motion vector by comparing it with surrounding positions in detail around the corresponding position using a two-point search method in a second order.

In addition, the merge method can be encoded in the SKIP mode. In the SKIP mode, only motion information excluding the residual signal of the corresponding PU is encoded, so that the pixel values of the current PU and the reference block are the same, and no pixel information is added. Bring it as it is. Moreover, the SKIP mode is applied only to the prediction unit (PU) of 2Nx2N, and whether the corresponding PU is encoded in the SKIP mode is determined using SKIP_FLAG.

In addition, all of the methods applied to the above-described coding unit division structure may have different application ranges according to a block size or CU depth. As for the variable (ie, size or depth information) that determines the application range, the encoder and the decoder may be set to use a predetermined value, or a value determined according to a profile or level may be used. In addition, if the encoder writes the variable value in the bitstream, the decoder may obtain and use this value from the bitstream.

When the range of application varies depending on the depth of the coding unit, as illustrated in the table below, Method A is applied only to a depth above a given depth, Method B is applied only to a given depth or less, and Method C is applied only to a given depth. There may be.

When the depth of a given coding unit is 2, it is an example of a range determination method to which the methods of the present invention are applied. (O: applies to the depth, X: does not apply to the depth.) Coding unit depth indicating coverage Method A Method B Method C 0 X O X One X O X 2 O O O 3 O X X 4 O X X

If the methods of the present invention are not applied to all depths, an arbitrary flag may be used to indicate. In addition, a value that is one greater than the maximum value of the depth of the coding unit (CU) may be expressed by signaling a depth value of the coding unit (CU) indicating the coverage. The above-described method is different for the color difference block according to the size of the luminance block. Can be applied. In addition, it can be applied differently to the luminance signal image and the color difference image.

Luminance block size Color difference block size Apply luminance Apply color difference Methods 4 (4x4, 4x2, 2x4) 2 (2x2) O or X O or X A 1, 2, ... 4 (4x4, 4x2, 2x4) O or X O or X Me 1, 2, ... 8 (8x8, 8x4, 4x8, 2x8, etc.) O or X O or X Da 1, 2, ... 16 (16x16, 16x8, 4x16, 2x16, etc.) O or X O or X D 1, 2, .. 32 (32x32) O or X O or X M 1, 2, ... 8 (8x8, 8x4, 2x8, etc.) 2 (2x2) O or X O or X Bar 1, 2, ... 4 (4x4, 4x2, 2x4) O or X O or X Company 1, 2, .. 8 (8x8, 8x4, 4x8, 2x8, etc.) O or X O or X Ah 1, 2, .. 16 (16x16, 16x8, 4x16, 2x16, etc.) O or X O or X Now 1, 2, .. 32 (32x32) O or X O or X Car 1, 2, .. 16 (16x16, 8x16, 4x16, etc.) 2 (2x2) O or X O or X Get 1, 2, .. 4 (4x4, 4x2, 2x4) O or X O or X Par 1, 2, .. 8 (8x8, 8x4, 4x8, 2x8, etc.) O or X O or X Ha 1, 2, .. 16 (16x16, 16x8, 4x16, 2x16, etc.) O or X O or X Dog 1, 2, .. 32 (32x32) O or X O or X My 1st, 2nd, ...

Table 2 shows an example of a combination of methods.

Looking at the method "four 1" among the modified methods in Table 2, when the size of the luminance block is 8 (8x8, 8x4, 2x8, etc.), and the size of the color difference block is 4 (4x4, 4x2, 2x4) The method of the specification can be applied to a luminance signal and a color difference signal. Looking at the method "wave 2" among the above modified methods, when the size of the luminance block is 16 (16x16, 8x16, 4x16, etc.), and the size of the color difference block is 4 (4x4, 4x2, 2x4), The method of the specification may be applied to a luminance signal and not to a color difference signal. As other modified methods, the method of the specification may be applied only to the luminance signal and may not be applied to the color difference signal. Conversely, the method of the specification may be applied only to the color difference signal and not to the luminance signal.

Hereinafter, based on the description of the object information-based high-speed high-efficiency video codec image encoding system 1 using machine learning for detecting objects described so far, high-speed high-efficiency video based on object information using machine learning according to an embodiment of the present invention The codec video encoding method will be described in detail.

The description of the components constituting the object information-based high-speed and high-efficiency video codec image encoding system using the above-described machine learning and the features of the components are intact in the object information-based high-speed high-efficiency video codec image encoding method using machine learning to be described later. Can be applied.

A high-speed, high-efficiency video codec image encoding method based on object information using machine learning is based on the flowchart of FIG. 14.

A high-speed, high-efficiency video codec image encoding method based on object information using machine learning proceeds to steps (A) to (F), and may encode an image image.

Hereinafter, each step of a method of encoding a high-speed, high-efficiency video codec image based on object information using machine learning will be described.

The object information-based high-speed, high-efficiency video codec image encoding method using machine learning begins with step (A) in which the image image generator 10 photographs a certain area over time and generates different image images over time (S110). ).

Thereafter, the object information providing unit 20 proceeds to step (B) in which a specific object is machine-learned and object information of the learned object is extracted (S120).

Thereafter, the object region extraction unit 30 divides the image image at regular intervals from the image image generation unit 10, receives an image image in which a plurality of first blocks are formed, and receives the object information from the object information providing unit 20. The input and stored reference object and object information are compared, and the non-matching object is extracted as a non-learning object, and the matching object is extracted as a learning object, and the process proceeds to step (C) (S200).

Thereafter, the learning object encoding unit 40 divides the learning object into a plurality of first blocks into a plurality of learning object sub-blocks, and weights the learning object sub-blocks for encoding (D) (S410) or non-learning. The object encoding unit 50 proceeds to step (E) of encoding the first block included in the non-learning object (S420). Thereafter, the encoding end determination unit 60 may proceed to step (F), which is repeated until the encoding of the image image ends in step (C), step (D), and step (E) (S500). ). At the end of step (F), a series of steps of a method of encoding a high-speed high-efficiency video codec image based on object information using machine learning may be ended.

In addition, in the object information-based high-speed, high-efficiency video codec image encoding method using machine learning, in step (C), when the object region extracting unit 30 extracts the learning object, the learning object region is set to be larger than the size of the learning object. You can proceed with the steps.

The object information-based high-speed and high-efficiency video codec image encoding method using machine learning is through these steps, classifying learning and non-learning objects in images based on machine-learned data, and encoding learning and non-learning objects in different ways. By encoding by using, encoding efficiency for an image can be improved, and an encoding speed for an image can be increased. Such a method according to the present invention may be produced as a program for execution on a computer and stored in a computer-readable recording medium. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., including those implemented in the form of a carrier wave (for example, transmission over the Internet). can do.

The computer-readable recording medium is distributed over a computer system connected by a network, and computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes and code segments for implementing the method can be easily deduced by programmers in the art to which the present invention belongs.

The embodiments of the present invention have been described above with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features. You can understand. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting.

1: High-speed and high-efficiency video codec image coding system based on object information using machine learning
10: image image generator 110: first image image
120: second video image
20: object information provision unit
30: object region extraction unit 40: learning object encoding unit
50: Non-learning object encoding unit 60: Encoding end determination unit

Claims (6)

(A) step of generating different image images according to time by photographing a certain area according to time by the image image generator;
(B) step of extracting object information of the learned object by machine learning the specific object by the object information providing unit;
The object region extracting unit divides the image image at regular intervals from the image image generator, receives an image image in which a plurality of first blocks are formed, receives the object information from the object information providing unit, and stores the reference object and the object (C) extracting an unmatched object as a non-learning object in preparation for information and extracting the matching object as a learning object;
(D) step of dividing a plurality of first blocks included in the learning object into a plurality of learning object sub-blocks, and then encoding the learning object sub-blocks by giving weights to the learning object sub-blocks; And
Including the step (E) of encoding a first block included in the non-learning object encoding unit,
In the step (A), the image image generator generates different first and second image images over time,
In the step (E), the non-learning object encoding unit overlaps the first video image and the second video image, so that the learning object of the second video image is in an area included in the non-learning object of the first video image. In the case of overlapping, the first block of the learning object of the second video image is divided into a size equal to or less than the size of the first block,
In the step (C), when the object region extraction unit is extracted as a learning object, further comprising the step of setting a learning object region larger than the size of the learning object, including the learning object,
In the step (E), the non-learning object encoding unit includes a size value of a divided block limiting the division of the first block, and divides the first block until it corresponds to the size value of the divided block. ,
The second block is encoded by dividing the first block into second blocks whose horizontal and vertical lengths of the first block are 1/2,
Object information-based high speed using machine learning that allows the first block and the second block to be encoded differently by giving a weight of '1' to the second block and a weight of '0' to the first block High-efficiency video codec video encoding method. delete delete delete delete delete

Images