CN103024395A - Device and method for multi-view video coding mode selection - Google Patents

Abstract

The present invention proposes a method and device for selecting a multi-viewpoint video encoding mode. Wherein, the method includes the following steps: input a video image and a corresponding depth map, and divide each frame of the video image into a plurality of basic coding units, namely macroblocks; judge the position of the currently coded macroblock, and set the prediction SKIP mode The rate-distortion threshold and calculate the rate-distortion cost of the current macroblock in SKIP mode; calculate the motion range of the current macro block according to the rate-distortion threshold and the rate-distortion cost; and select the corresponding candidate mode according to the motion range of the current macro block. According to the method of the embodiment of the present invention, by predicting the SKIP mode based on the block-based adaptive threshold, a large amount of unnecessary calculations are avoided, and the motion vector of the current macroblock is predicted according to the motion vector of the adjacent macroblock and the corresponding macroblock of the adjacent view. The motion range, and then check the corresponding candidate patterns according to the size of the motion range, so that the search range can be reduced and the calculation time can be reduced.

Description

Method and device for selecting multi-view video encoding mode

technical field

The present invention relates to the technical field of video coding, in particular to a method and device for selecting a multi-viewpoint video coding mode.

Background technique

With the continuous increase of people's demand for stereoscopic vision perception, multi-view video technology has attracted more and more domestic and foreign scholars and industrial circles. Compared with ordinary two-dimensional video, multi-view video increases the depth information of the scene, thereby providing users with a sense of visual reality and lifelikeness, but at the same time it also increases at least double the video data to be processed. Therefore, for multi-view video coding, the prediction method of disparity estimation is introduced to remove the redundancy between views.

Although existing video coding standards, such as H.264, have obvious advantages in coding efficiency, the high computational complexity they introduce to improve compression efficiency has become an unbearable defect in practical applications. For example, in the JMVM model of H.264/MVC, in order to obtain higher video coding efficiency, JMVM provides 7 block sizes (16×16, 16×8, 8×16, 8×8, 8×4, 4×8, and 4×4) for motion prediction and disparity prediction mode selection. For the macroblock coding of the inter frame, there are 11 candidate modes: SKIP, inter_16×16, inter_16×8, inter_8×16, inter_8×8, inter_8×4, inter_4×8, inter_4×4, intra_4×4, intra_8× 8,, intra_16×16, when selecting the encoding mode, it is necessary to perform motion estimation on different partition modes, use rate distortion optimization (RDO) to calculate the encoding cost, compare and select the smallest encoding mode as the optimal encoding mode. This multi-modal motion estimation and mode decision has high computational complexity, and its encoding time accounts for more than half of the total processing time. It can be seen that a reasonable and efficient mode decision algorithm is of great significance for video coding.

Nowadays, in order to meet the needs of wider-angle multi-viewpoint video image generation, in addition to inputting multi-viewpoint video, the corresponding depth video is usually input during encoding, so that synthetic viewpoint video images from more angles can be generated at the output end. Although this encoding mode meets the needs of wide-angle and even holographic video, the huge amount of data and calculations have greatly restricted the real-time application of video.

Currently, there are many fast mode selection methods based on single-view video coding. However, most of the existing fast single-view video coding algorithms are not suitable for multi-view video coding due to the introduction of the prediction method of disparity estimation and the introduction of video depth information in the multi-view coding framework.

Contents of the invention

The object of the present invention is to solve at least one of the above-mentioned technical drawbacks.

In order to achieve the above object, an embodiment of the present invention proposes a method for selecting a multi-viewpoint video coding mode, including the following steps: S1: Input a video image and a corresponding depth map, and divide each frame of the video image into A plurality of basic coding units, that is, macroblocks; S2: determine the position of the currently encoded macroblock, set the rate-distortion threshold for predicting SKIP mode and calculate the rate-distortion cost of the current macroblock in SKIP mode; S3: according to the Calculate the motion range of the current macroblock based on the rate-distortion threshold and the rate-distortion cost; and S4: Select a corresponding candidate mode according to the motion range of the current macroblock.

According to the method of the embodiment of the present invention, by predicting the SKIP mode based on the block-based adaptive threshold, a large amount of unnecessary calculations are avoided, and the motion vector of the current macroblock is predicted according to the motion vector of the adjacent macroblock and the corresponding macroblock of the adjacent view. The motion range, and then check the corresponding candidate patterns according to the size of the motion range, so that the search range can be reduced and the calculation time can be reduced.

In one embodiment of the present invention, the step S2 specifically includes: S21: If the frame where the current macroblock is located is an I frame or the current macroblock is located in the first row or first column of the frame, use the rate-distortion optimization method to check For all intra-frame and inter-frame modes, select the mode with the smallest rate-distortion cost as the best coding mode; S22: If the viewpoint of the current macroblock is an independent coding viewpoint, and the current frame is not an I frame and the current macroblock is not in the frame In the first row or column, calculate the first rate-distortion cost of the current macroblock using SKIP mode and set the first rate-distortion threshold used to predict the SKIP mode; and S23: If the view of the current macroblock is a joint predictive coding view , while the frame is not an I frame and the current macroblock is not in the first row or column of the frame, then set the second rate-distortion threshold used to predict the SKIP mode and calculate the second rate for the current macroblock to use the SKIP mode Distortion cost.

In an embodiment of the present invention, the step S3 specifically includes: comparing the first rate-distortion cost with the first rate-distortion threshold; when the first rate-distortion cost is less than the first rate-distortion threshold , setting the SKIP mode as the best coding mode of the current macroblock; and calculating a first motion magnitude of the current macroblock when the first rate-distortion cost is greater than the first rate-distortion threshold.

In an embodiment of the present invention, the step S3 further includes: comparing the second rate-distortion cost with the second rate-distortion threshold; when the second rate-distortion cost is smaller than the second rate-distortion threshold , setting SKIP mode as the best coding mode of the current macroblock; and calculating a first motion magnitude of the current macroblock when the second rate-distortion cost is greater than the second rate-distortion threshold.

In order to achieve the above object, another embodiment of the present invention proposes a multi-view video coding mode selection device, including a division module, which is used to input a video image and a corresponding depth map, and divide each frame of the video image into It is a plurality of basic coding units, that is, macroblocks; a judgment setting module is used to judge the position of the macroblock currently encoded, set the rate-distortion threshold for predicting the SKIP mode and calculate the rate-distortion cost under the SKIP mode for the current macroblock a calculation module, configured to calculate the motion amplitude of the current macroblock according to the rate-distortion threshold and the rate-distortion cost; and a selection module, configured to select a corresponding candidate mode according to the motion amplitude of the current macroblock.

According to the device of the embodiment of the present invention, by predicting the SKIP mode based on the block-based adaptive threshold, a large number of unnecessary calculations are avoided, and the motion vector of the current macroblock is predicted according to the motion vector of the adjacent macroblock and the corresponding macroblock of the adjacent view. The motion range, and then check the corresponding candidate patterns according to the size of the motion range, so that the search range can be reduced and the calculation time can be reduced.

In one embodiment of the present invention, the judgment setting module specifically includes: a selection unit, used to determine the usage rate when the frame where the current macroblock is located is an I frame or the current macroblock is in the first row or first column of the frame where the macroblock is located. The distortion optimization method checks all intra-frame and inter-frame modes, and selects the mode with the smallest rate-distortion cost as the best encoding mode; the first calculation unit is used when the view of the current macroblock is an independent encoding view, and the frame is not an I frame And the current macroblock is not in the first row or the first column of the frame, then calculate the first rate-distortion cost of selecting the SKIP mode for the current macroblock and set the first rate-distortion threshold for predicting the SKIP mode; and the second calculation unit , used to set the second rate used to predict the SKIP mode when the view of the current macroblock is the joint predictive coding view, and the frame is not an I frame and the current macroblock is not in the first row or column of the frame Distortion threshold and calculate the second rate-distortion cost when the current macroblock selects the SKIP mode.

In an embodiment of the present invention, the calculation module specifically includes: a first comparison unit, configured to compare the first rate-distortion cost with the first rate-distortion threshold; a first processing unit, configured to When the first rate-distortion cost is less than the first rate-distortion threshold, the SKIP mode is the best coding mode for the current macroblock; and a second processing unit is configured to use when the first rate-distortion cost is greater than the first rate-distortion rate When the distortion threshold is set, the first motion magnitude of the current macroblock is calculated.

In an embodiment of the present invention, the calculation module specifically further includes: a second comparison unit, configured to compare the second rate-distortion cost with the second rate-distortion threshold; a third processing unit, configured to When the second rate-distortion cost is less than the second rate-distortion threshold, set the SKIP mode as the best coding mode for the current macroblock; and a fourth processing unit is configured to use when the second rate-distortion cost is greater than the second When the rate-distortion threshold is set, the first motion magnitude of the current macroblock is calculated.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

FIG. 1 is a flowchart of a method for selecting a multi-viewpoint video coding mode according to an embodiment of the present invention;

FIG. 2 is a specific flowchart of a method for selecting a multi-viewpoint video coding mode according to an embodiment of the present invention;

FIG. 3 is a positional diagram of a macroblock and its temporal and spatial adjacent macroblocks in an independently encoded viewpoint according to an embodiment of the present invention;

FIG. 4 is a diagram of the positional relationship between a macroblock and its adjacent macroblocks in time, space, and view in a joint predictive coding view according to an embodiment of the present invention; and

Fig. 5 is a frame diagram of an apparatus for selecting a multi-view video coding mode according to an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

FIG. 1 is a flowchart of a method for selecting a multi-viewpoint video coding mode according to an embodiment of the present invention. Fig. 2 is a specific flowchart of a method for selecting a multi-viewpoint video coding mode according to an embodiment of the present invention. As shown in Figure 1 and Figure 2, the method for selecting a multi-viewpoint video coding mode according to an embodiment of the present invention includes the following steps:

S101. Input a video image and a corresponding depth map, and divide each frame of the video image into multiple basic coding units, namely macroblocks.

S102. Determine the position of the currently coded macroblock, set the rate-distortion threshold for predicting the SKIP mode, and calculate the rate-distortion cost when the current macroblock selects the SKIP mode.

If the frame where the current macroblock is located is an I frame or the current macroblock is in the first row or column of the frame, use the rate-distortion optimization method to check all intra-frame inter-frame modes, and select the mode with the smallest rate-distortion cost as the best encoding mode.

通过如下公式计算：If the viewpoint of the current macroblock is an independently coded viewpoint, and the frame is not an I frame and the current macroblock is not in the first row or column of the frame, calculate the first rate-distortion cost RD of the current macroblock using SKIP mode _skip1 and set the first rate-distortion threshold used to predict SKIP mode

Calculated by the following formula:

${\mathrm{<span\; class="notranslate">\; T</span>}}_{{\mathrm{<span\; class="notranslate">\; skip</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; RD</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; t</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; SKIP</span>}<span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; ,</span>$

Among them, RDcost(SKIP) _i is to calculate the left macroblock (macroblock 1 in Fig. 3), the upper macroblock (macroblock 2 in Fig. 3), and the upper right macroblock of the current macroblock (macroblock 0 in Fig. 3), respectively. The rate-distortion cost of the block (macroblock 3 in Figure 3) and the macroblock at the same position as the current macroblock in the previous frame (macroblock 4 in Figure 3) in SKIP mode. W _i is the corresponding weight coefficient, which is obtained from offline training.

The definition of I _i is as follows:

If the view of the current macroblock is a joint predictive coding view, and the frame is not an I frame and the current macroblock is not in the first row or column of the frame, then set the second rate-distortion cost RD used to predict the SKIP mode _skip2 and calculate the second rate-distortion threshold T _skip2 for which the current macroblock selects the SKIP mode.

${\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="T\; skip"\; filenames="HDA00002693098900021.png"\; state="\{\{state\}\}">T</figure-callout></span><figure-callout\; id="2"\; label="T\; skip"\; filenames="HDA00002693098900021.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{{\mathrm{<span\; class="notranslate">\; </span>}}_{}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 13</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; RD</span>}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; t</span>}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; SKIP</span>}<span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 13</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; ,</span>$

Wherein, RDcost(SKIP) _i are rate-distortion costs of macroblock 1 to macroblock 13 in SKIP mode in FIG. 4 respectively. These macroblocks are the left macroblock (macroblock 1), the upper macroblock (macroblock 2), and the upper right macroblock (macroblock 3) of the current macroblock 0 in the current view video. The macroblock with the same macroblock position (macroblock 4) and the macroblock (macroblock 5) whose adjacent coded viewpoint has been displaced by parallax and its 8 adjacent macroblocks (macroblock 6~macroblock 13). The disparity between macroblock 0 and macroblock 5 is derived from the depth of the current macroblock at the current view.

W _i is the corresponding weight coefficient, which is obtained from offline training. The definition of I _i is as follows:

Step S103, calculating the motion amplitude of the current macroblock according to the rate-distortion threshold and the rate-distortion cost.

Compare the first rate-distortion cost RD _skip1 with the first rate-distortion threshold

时，将SKIP模式为当前宏块的最佳编码模式。When the first rate-distortion cost RD _skip1 is less than the first rate-distortion threshold

When , the SKIP mode is the best coding mode for the current macroblock.

时，计算当前宏块的第一运动幅度A_MV1，A_MV1的计算公式如下：When the first rate-distortion cost RD _skip1 is greater than the first rate-distortion threshold

, calculate the first motion amplitude A _MV1 of the current macroblock, and the calculation formula of A _MV1 is as follows:

First take the motion vectors MV _i , (i=1, 2, 3, 4) of macroblock 1 to macroblock 4 in Fig. 3, and record MV _i = (xi _, y _i ). but, $A_{\mathrm{MV}1}=\left|\frac{\underset{i=1}{\overset{4}{\mathrm{\&Sigma;}}}{W}_i\&Center\; Dot;K_i\&Center\; Dot;x_i}{\underset{i=1}{\overset{4}{\mathrm{\&Sigma;}}}{W}_i}\right|+\left|\frac{\underset{i=1}{\overset{4}{\mathrm{\&Sigma;}}}{W}_i\&Center\; Dot;K_i\&Center\; Dot;{\mathrm{the\; y}}_i}{\underset{i=1}{\overset{4}{\mathrm{\&Sigma;}}}{W}_i}\right|,$ Among them, W _i is the corresponding weight coefficient, which is obtained from offline training, and the definition of K _i is as follows:

Compare the second rate-distortion cost RD _skip2 with the second rate-distortion threshold T _skip2 .

When the second rate-distortion cost RD _skip2 is smaller than the second rate-distortion threshold T _skip2 , the SKIP mode is the best coding mode for the current macroblock.

When the second rate-distortion cost is greater than the second rate-distortion threshold, the second motion amplitude A _MV2 of the current macroblock is calculated, and the calculation formula of A _MV2 is as follows:

First take the motion vectors MV _i , (i=1, 2, . . . , 13) of macroblock 1 to macroblock 13 in Fig. 4, and record MV _i = (xi _, y _i ). but, $A_{\mathrm{<figure-callout\; id="2"\; label="MV"\; filenames="HDA00002693098900021.png"\; state="\{\{state\}\}">MV</figure-callout>}2}=\left|\frac{\underset{i=1}{\overset{13}{\mathrm{\&Sigma;}}}{W}_i\&CenterDot;K_i\&Center\; Dot;x_i}{\underset{i=1}{\overset{13}{\mathrm{\&Sigma;}}}{W}_i}\right|+\left|\frac{\underset{i=1}{\overset{13}{\mathrm{\&Sigma;}}}{W}_i\&Center\; Dot;K_i\&Center\; Dot;{\mathrm{the\; y}}_i}{\underset{i=1}{\overset{13}{\mathrm{\&Sigma;}}}{W}_i}\right|,$ Among them, W _i is the corresponding weight coefficient, which is obtained from offline training, and the definition of K _i is as follows:

S104. Select a corresponding candidate mode according to the magnitude of motion of the current macroblock.

A corresponding candidate mode is selected according to the magnitude of the motion amplitude A _MV1 or A _MV2 of the current macroblock.

(1) When A _MV1 ≤ 1, take SKIP and ME16×16 as candidate modes and select the mode with the smallest rate-distortion as the best coding mode for the current macroblock.

(2) When 1<A _MV1 ≤2, use ME16×8 and ME8×16 as candidate modes for rate-distortion optimization calculation and select the mode with the smallest rate-distortion cost as the best coding mode.

(3) When 2<A _MV1 ≤5, select the MV_P8×8 mode to encode the current macroblock and use the rate-distortion optimization method to select the segmentation of the 8×8 sub-macroblock with the smallest rate-distortion cost.

(4) For A _MV1 >5, select DE (16×16, 16×8, 8×16, 8×8), intra_4×4, intra_8×8, intra_16×16 as candidate modes and select the one with the smallest rate-distortion mode is the best coding mode for the current macroblock.

(5) When A _MV2 ≤ 1, take SKIP and ME16×16 as candidate modes and select the mode with the smallest rate-distortion as the best coding mode for the current macroblock.

(6) When 1<A _MV2 ≤2, use ME16×8 and ME8×16 as candidate modes for rate-distortion optimization calculation and select the mode with the smallest rate-distortion cost as the best coding mode.

(7) When 2<A _MV2 ≤ 5, select the MV_P8×8 mode to encode the current macroblock and use the rate-distortion optimization method to select the segmentation of the 8×8 sub-macroblock with the smallest rate-distortion cost.

(8) For A _MV2 >5, select DE (16×16, 16×8, 8×16, 8×8), intra_4×4, intra_8×8, intra_16×16 as candidate modes and select the one with the smallest rate distortion mode is the best coding mode for the current macroblock.

After the encoding of the current macroblock is completed, the encoding of the next macroblock is continued until multiple macroblocks of each frame of the video image are completed.

According to the method of the embodiment of the present invention, by predicting the SKIP mode based on the block-based adaptive threshold, a large amount of unnecessary calculations are avoided, and the motion vector of the current macroblock is predicted according to the motion vector of the adjacent macroblock and the corresponding macroblock of the adjacent view. The motion range, and then check the corresponding candidate patterns according to the size of the motion range, so that the search range can be reduced and the calculation time can be reduced.

Fig. 5 is a frame diagram of an apparatus for selecting a multi-view video coding mode according to an embodiment of the present invention. As shown in FIG. 5 , the device for selecting a multi-view video coding mode according to an embodiment of the present invention includes a division module 100 , a judgment setting module 200 , a calculation module 300 and a selection module 400 .

The division module 100 is used to input video images and corresponding depth maps, and divide each frame of the video images into a plurality of basic coding units, namely macroblocks.

The judgment setting module 200 is used for judging the position of the currently coded macroblock, setting the rate-distortion threshold for predicting the SKIP mode, and calculating the rate-distortion cost when the current macroblock selects the SKIP mode.

In one embodiment of the present invention, the judgment setting module 200 includes a selection unit 210 , a first calculation unit 220 and a second calculation unit 230 .

The selection unit 210 is used to check all the intra-frame and inter-frame modes using the rate-distortion optimization method when the frame where the current macroblock is located is an I frame or the current macroblock is in the first row or column of the frame, and the selection rate-distortion cost is the smallest The mode of is the best encoding mode.

The first calculation unit 220 is used to calculate the SKIP mode for the current macroblock when the viewpoint of the current macroblock is an independently coded viewpoint, and the frame is not an I frame and the current macroblock is not in the first row or the first column of the frame. The first rate-distortion cost and set the first rate-distortion threshold used to predict the SKIP mode.

First Rate Distortion Threshold Calculated by the following formula: $T_{{\mathrm{skip}}_1}=\frac{\underset{i=1}{\overset{4}{\mathrm{\&Sigma;}}}{W}_i\&Center\; Dot;I_i\&Center\; Dot;\mathrm{RD}\cos t{\left(\mathrm{SKIP}\right)}_i}{\underset{i=1}{\overset{4}{\mathrm{\&Sigma;}}}{W}_i\&Center\; Dot;I_i}.$

Among them, RDcost(SKIP) _i is to calculate the left macroblock (macroblock 1 in Fig. 3), the upper macroblock (macroblock 2 in Fig. 3), and the upper right macroblock of the current macroblock (macroblock 0 in Fig. 3), respectively. The rate-distortion cost of the block (macroblock 3 in Figure 3) and the macroblock at the same position as the current macroblock in the previous frame (macroblock 4 in Figure 3) in SKIP mode. W _i is the corresponding weight coefficient, which is obtained from offline training.

The definition of I _i is as follows:

The second computing unit 230 is used to set the SKIP mode for prediction when the viewpoint of the current macroblock is the joint predictive coding viewpoint, and the frame is not an I frame and the current macroblock is not in the first row or column of the frame The second rate-distortion threshold of the current macroblock and calculate the second rate-distortion cost for the SKIP mode selected.

The second rate-distortion threshold T _skip2 is calculated by the following formula: $T_{{\mathrm{skip}}_2}=\frac{\underset{i=1}{\overset{13}{\mathrm{\&Sigma;}}}{W}_i\&CenterDot;I_i\&CenterDot;\mathrm{RD}\cos t{\left(\mathrm{SKIP}\right)}_i}{\underset{i=1}{\overset{13}{\mathrm{\&Sigma;}}}{W}_i\&CenterDot;I_i},$

Wherein, RDcost(SKIP) _i are rate-distortion costs of macroblock 1 to macroblock 13 in SKIP mode in FIG. 4 respectively. These macroblocks are the left macroblock (macroblock 1), the upper macroblock (macroblock 2), and the upper right macroblock (macroblock 3) of the current macroblock 0 in the current view video. The macroblock with the same macroblock position (macroblock 4) and the macroblock (macroblock 5) whose adjacent coded viewpoint has been displaced by parallax and its 8 adjacent macroblocks (macroblock 6~macroblock 13). The disparity between macroblock 0 and macroblock 5 is derived from the depth of the current macroblock at the current view.

W _i is the corresponding weight coefficient, which is obtained from offline training. The definition of I _i is as follows:

The calculation module 300 is used for calculating the motion amplitude of the current macroblock according to the rate-distortion threshold and the rate-distortion cost.

In an embodiment of the present invention, the calculation module 300 includes a first comparison unit 310 , a first processing unit 320 , a second processing unit 330 , a second comparison unit 340 , a third processing unit 350 and a fourth processing unit 360 .

The first comparison unit 310 is configured to compare the first rate-distortion cost with the first rate-distortion threshold.

The first processing unit 320 is configured to set the SKIP mode as the best coding mode for the current macroblock when the first rate-distortion cost is smaller than the first rate-distortion threshold.

The second processing unit 330 is configured to calculate a first motion magnitude of the current macroblock when the first rate-distortion cost is greater than a first rate-distortion threshold.

The second comparing unit 340 is used for comparing the second rate-distortion cost and the second rate-distortion threshold.

The third processing unit 350 is configured to set the SKIP mode as the best coding mode for the current macroblock when the second rate-distortion cost is smaller than the second rate-distortion threshold.

The fourth processing unit 360 is configured to calculate the first motion magnitude of the current macroblock when the second rate-distortion cost is greater than the second rate-distortion threshold.

The selecting module 400 is used to select corresponding candidate modes according to the magnitude of motion of the current macroblock.

A corresponding candidate mode is selected according to the magnitude of the motion amplitude A _MV1 or A _MV2 of the current macroblock.

(1) When A _MV1 ≤ 1, take SKIP and ME16×16 as candidate modes and select the mode with the smallest rate-distortion as the best coding mode for the current macroblock.

(2) When 1<A _MV1 ≤2, use ME16×8 and ME8×16 as candidate modes for rate-distortion optimization calculation and select the mode with the smallest rate-distortion cost as the best coding mode.

(3) When 2<A _MV1 ≤5, select the MV_P8×8 mode to encode the current macroblock and use the rate-distortion optimization method to select the segmentation of the 8×8 sub-macroblock with the smallest rate-distortion cost.

(4) For A _MV1 >5, select DE (16×16, 16×8, 8×16, 8×8), intra_4×4, intra_8×8, intra_16×16 as candidate modes and select the one with the smallest rate-distortion mode is the best coding mode for the current macroblock.

(5) When A _MV2 ≤ 1, take SKIP and ME16×16 as candidate modes and select the mode with the smallest rate-distortion as the best coding mode for the current macroblock.

(6) When 1<A _MV2 ≤2, use ME16×8 and ME8×16 as candidate modes for rate-distortion optimization calculation and select the mode with the smallest rate-distortion cost as the best coding mode.

(7) When 2<A _MV2 ≤ 5, select the MV_P8×8 mode to encode the current macroblock and use the rate-distortion optimization method to select the segmentation of the 8×8 sub-macroblock with the smallest rate-distortion cost.

(8) For A _MV2 >5, select DE (16×16, 16×8, 8×16, 8×8), intra_4×4, intra_8×8, intra_16×16 as candidate modes and select the one with the smallest rate distortion mode is the best coding mode for the current macroblock.

After the encoding of the current macroblock is completed, the encoding of the next macroblock is continued until multiple macroblocks of each frame of the video image are completed.

According to the device of the embodiment of the present invention, by predicting the SKIP mode based on the block-based adaptive threshold, a large number of unnecessary calculations are avoided, and the motion vector of the current macroblock is predicted according to the motion vector of the adjacent macroblock and the corresponding macroblock of the adjacent view. The motion range, and then check the corresponding candidate patterns according to the size of the motion range, so that the search range can be reduced and the calculation time can be reduced.

It should be understood that the specific operation process of each module and unit in the device embodiment of the present invention may be the same as that described in the method embodiment, and will not be described in detail here.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be construed as limitations to the present invention. Variations, modifications, substitutions, and modifications to the above-described embodiments are possible within the scope of the present invention.

Claims (8)

1. A method for selecting a multi-viewpoint video encoding mode, characterized in that it comprises the following steps: S1: Input a video image and a corresponding depth map, and divide each frame of the video image into a plurality of basic coding units, namely macroblocks; S2: Determine the position of the currently encoded macroblock, set the rate-distortion threshold for predicting the SKIP mode, and calculate the rate-distortion cost when the current macroblock selects the SKIP mode; S3: Calculate the motion range of the current macroblock according to the rate-distortion threshold and the rate-distortion cost; and S4: Select a corresponding candidate mode according to the magnitude of the motion amplitude of the current macroblock. 2. The method for selecting a multi-viewpoint video coding mode according to claim 1, wherein said step S2 specifically comprises: S21: If the frame where the current macroblock is located is an I frame or the current macroblock is in the first row or column of the frame, then use the rate-distortion optimization method to check all intra-frame inter-frame modes, and select the mode with the smallest rate-distortion cost as best encoding mode; S22: If the viewpoint of the current macroblock is an independently coded viewpoint, and the frame is not an I frame and the current macroblock is not in the first row or column of the frame, calculate the first rate-distortion of the current macroblock using SKIP mode cost and set the first rate-distortion threshold used to predict the SKIP mode; and S23: If the viewpoint of the current macroblock is the joint predictive coding viewpoint, and the frame is not an I frame and the current macroblock is not in the first row or the first column of the frame, set the second rate-distortion used to predict the SKIP mode Threshold and calculate the second rate-distortion cost of the SKIP mode for the current macroblock. 3. The method for selecting a multi-viewpoint video coding mode as claimed in claim 1, wherein said step S3 specifically comprises: comparing the first rate-distortion penalty to the first rate-distortion threshold; When the first rate-distortion cost is less than the first rate-distortion threshold, the SKIP mode is the best coding mode for the current macroblock; and When the first rate-distortion cost is greater than the first rate-distortion threshold, calculate a first motion magnitude of the current macroblock. 4. The method for selecting a multi-viewpoint video coding mode according to claim 1, wherein said step S3 further comprises: comparing the second rate-distortion penalty to the second rate-distortion threshold; When the second rate-distortion cost is less than the second rate-distortion threshold, the SKIP mode is the best coding mode for the current macroblock; and When the second rate-distortion cost is greater than the second rate-distortion threshold, calculate the first motion magnitude of the current macroblock. 5. A device for selecting a multi-viewpoint video encoding mode, comprising: A division module for inputting a video image and a corresponding depth map, and dividing each frame of the video image into a plurality of basic coding units, namely macroblocks; A judgment setting module, configured to judge the position of the currently encoded macroblock, set the rate-distortion threshold for predicting the SKIP mode and calculate the rate-distortion cost for the current macroblock in the SKIP mode; a calculation module, configured to calculate the motion amplitude of the current macroblock according to the rate-distortion threshold and the rate-distortion cost; and The selection module is used to select a corresponding candidate mode according to the magnitude of the motion of the current macroblock. 6. The multi-viewpoint video coding mode selection device as claimed in claim 5, wherein the judgment setting module specifically includes: The selection unit is used to check all intra-frame and inter-frame modes using the rate-distortion optimization method when the frame where the current macroblock is located is an I frame or the current macroblock is in the first row or first column of the frame, and the selection rate-distortion cost is the smallest The mode of is the best coding mode; The first calculation unit is used to calculate the SKIP mode for the current macroblock when the viewpoint of the current macroblock is an independent coding viewpoint, and the frame is not an I frame and the current macroblock is not in the first row or the first column of the frame. The first rate-distortion cost and set the first rate-distortion threshold used to predict the SKIP mode; and The second calculation unit is used to set the SKIP mode for prediction when the viewpoint of the current macroblock is the joint predictive coding viewpoint, and the frame is not an I frame and the current macroblock is not in the first row or the first column of the frame The second rate-distortion threshold of the current macroblock and calculate the second rate-distortion cost for the SKIP mode selected. 7. The multi-viewpoint video coding mode selection device as claimed in claim 5, wherein the calculation module specifically includes: a first comparison unit, configured to compare the first rate-distortion cost with the first rate-distortion threshold; A first processing unit, configured to set the SKIP mode as the best coding mode for the current macroblock when the first rate-distortion cost is less than the first rate-distortion threshold; and A second processing unit, configured to calculate a first motion magnitude of the current macroblock when the first rate-distortion cost is greater than the first rate-distortion threshold. 8. The multi-viewpoint video coding mode selection device as claimed in claim 5, wherein the calculation module specifically further comprises: a second comparing unit, configured to compare the second rate-distortion cost with the second rate-distortion threshold; A third processing unit, configured to set the SKIP mode as the best coding mode for the current macroblock when the second rate-distortion cost is smaller than the second rate-distortion threshold; and A fourth processing unit, configured to calculate a first motion magnitude of the current macroblock when the second rate-distortion cost is greater than the second rate-distortion threshold.

Images