CN104144347B - A kind of H.264/AVC video I frame error recovery methods based on hiding reversible data - Google Patents

Abstract

The present invention discloses a H.264/AVC video I frame error recovery method based on reversible data hiding. At the encoding end, first extract the feature vector of the macroblock and determine the host vector of the macroblock, and then convert the previous macroblock The feature vector is embedded into the host vector of the current macroblock; at the decoding end, the feature information is extracted from the macroblock in the I frame embedded with feature information, and the coding mode and brightness prediction mode of the macroblock are determined, and then the non- Correctly decode the block for error recovery; the advantage is that at the encoding end, the feature information of the extracted macroblock is determined by the luminance prediction mode of the sub-block of the macroblock or by the encoding mode and luminance prediction mode of the macroblock, which can effectively recover when there is a scene change. The missing macroblock, and when embedding the feature vector into the host vector, according to the encoding mode of the previous macroblock, the method of generalized differential expansion is used to embed, which not only makes the embedding capacity controllable, but also realizes the extraction of feature information. I-frame raw data before embedding feature information.

Description

A H.264/AVC Video I Frame Error Recovery Method Based on Reversible Data Hiding

technical field

The invention relates to an error recovery and covert communication technology in the video network transmission process, in particular to an H.264/AVC video I frame error recovery method based on reversible data hiding.

Background technique

With the maturity of multimedia technology and the drive of user needs, wireless video communication has received more and more attention. However, the transmission channel of the wireless communication network and the Internet is not reliable. When the H.264/AVC video stream is transmitted on the communication channel, especially on a narrowband channel with large noise interference or a channel that may lose packets (Internet) During transmission, problems such as channel interference, network congestion, and routing delays can cause random bit errors, burst errors, and other bit errors.

The efficient H.264/AVC video coding standard relies heavily on the integrity of the video code stream. Once packet loss or code error occurs, it will have a great impact on the video quality. This is because: on the one hand, H.264/AVC adopts variable length coding (VLC, Variable Length Coding) to improve coding efficiency, but because each VLC code word of variable length coding has different lengths, so if some of them If there is an error in the bit, the decoder cannot correctly skip the erroneous codeword, which will cause the VLC codeword to lose synchronization, resulting in the inability to decode the subsequent bit stream correctly; on the other hand, H.264/AVC uses intra-frame prediction, motion estimation, Motion vector prediction and other prediction technologies make the following data refer to the previous data when encoding. If some part of the data is wrong, not only will it not be able to decode correctly, but the subsequent data will also be affected, and this effect will continue. Continues for subsequent multiple frames. All of the above situations will directly lead to wrong decoding of video information, sharply reduce the reconstruction quality of video signal, and even lead to complete failure of the entire video communication in severe cases. Therefore, how to reduce or eliminate the impact of transmission errors is an important research direction in low-bit-rate video communication applications.

Although there have been some error recovery methods for H.264/AVC video, such as error recovery using video content correlation at the decoding end; interactive error recovery with a feedback channel between the encoding end and decoding end; using information Concealment technology extracts feature information suitable for error recovery at the encoding end, and passes the feature information to the decoding end for error recovery through information hiding. Among them, the method of error recovery based on information hiding technology is a new way of dealing with video error recovery because the information at the encoding end is richer and more accurate than that at the decoding end. Lin et al. cited a reversible data hiding method based on differential expansion to embed feature information, but since the feature information is a pixel value, the number of embedded feature information bits is very large, so multi-layer difference is used, and the calculation is relatively complicated. Chen et al. proposed an effective error recovery method, which uses the calculated motion vector (Motion vector, MV) of each macroblock in the I frame as important feature data, and embeds it into the same frame in a loop manner In other macroblocks, the embedding method adopts the odd-even embedding method. Since the original quantized DCT (Discrete Cosine Transform) coefficient value of the video carrier is permanently changed after the information is embedded, the quality of the reconstructed video will be affected after the information is extracted. On this basis, Chung et al. used the reversible information hiding method of histogram translation to embed the motion vector of the macroblock into the quantized DCT coefficient with a value of zero, but this method needs to modify many quantized DCT coefficients, which will not only affect the quality of the video imperceptible, and will increase the transmission bit rate. Therefore, error recovery methods based on information hiding techniques still have room for improvement in terms of extracting effective features, selecting appropriate embedding methods and embedding locations, and improving video reconstruction quality.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for recovering H.264/AVC video I frame errors based on reversible data hiding. When there is a scene change, the extracted feature information can efficiently recover lost macroblocks in the I frame. , which improves the quality of I frame error recovery, and after extracting the feature information, it can restore the original data of the I frame macroblock before embedding the feature information.

The technical scheme that the present invention solves the problems of the technologies described above is: a kind of H.264/AVC video I frame error recovery method based on reversible data hiding, it is characterized in that comprising the following steps:

① At the encoding end, embed feature information into each macroblock except the first macroblock in the I frame to obtain an I frame embedded with feature information. The specific process is as follows:

①-1. Define the kth macroblock that is currently pre-encoded in the I frame as the current macroblock, where 1≤k≤K, where K represents the total number of macroblocks contained in the I frame, and k The initial value is 1;

①-2. Convert the decimal value of the CBP of the current macroblock into a binary string composed of 6-bit binary values, and record it as A, A=a ₁ a ₂ a ₃ a ₄ a ₅ a ₆ , where a ₁ is The highest binary value of A, a ₆ is the lowest binary value of A;

①-3. Extract the feature vector of the current macroblock: if the encoding mode of the current macroblock is Intra4×4, then extract the respective luminance predictions of the four 4×4 blocks numbered 0, 4, 8, and 12 in the current macroblock The digital identification of the mode, and then quantize the digital identification of the respective luma prediction modes of the four 4×4 blocks into 4 bits, and then the four 4×4 blocks are numbered in the order of the current macroblock. The luminance prediction mode of the 4×4 block is arranged with digital signs represented by bits to form a one-dimensional vector containing 16 elements, and then the one-dimensional vector is used as the feature vector of the current macroblock, which is recorded as in, with Correspondingly represent the first element, the second element and the 16th element in the feature vector w ^Intra4×4 (k) of the current macroblock;

If the encoding mode of the current macroblock is Intra16×16, the number 9 is used to identify the encoding mode of the current macroblock, and then the digital identification of the encoding mode of the current macroblock is quantized to 4 bits, and the brightness prediction mode of the current macroblock is The digital identification of the current macroblock is quantized into 4 bits, and then the current macroblock coding mode and the current macroblock's luminance prediction mode are arranged in order to form a one-dimensional matrix containing 8 elements. vector, and then use the one-dimensional vector as the feature vector of the current macroblock, denoted as w ^Intra16×16 (k), in, with Correspondingly represent the first element, the second element and the eighth element in the feature vector w ^Intra16×16 (k) of the current macroblock;

①-4. Determine the host vector of the current macroblock: calculate the sum of the absolute values of all AC DCT coefficients of each 4×4 block in the current macroblock, and embed the 4×4 block with the largest sum value as the feature information into the block, Then scan all the quantized DCT coefficients in the feature information embedding block in zig-zag mode, and then arrange the 8th to 16th quantized DCT coefficients in the feature information embedding block in the order of zig-zag scanning to form a set containing 9 The one-dimensional vector of the element, and then use the one-dimensional vector as the host vector of the current macroblock, which is recorded as x(k), x(k)=(x ₁ x ₂ ... x ₉ ), where x ₁ , x ₂ and x ₉ corresponds to the first element, the second element and the ninth element in the host vector x(k) of the current macroblock;

①-5. Embed the eigenvector of the previous macroblock of the current macroblock into the host vector of the current macroblock: determine whether the current macroblock is the first macroblock in the I frame, and if so, perform Do not process, directly execute steps ①-7; otherwise, if the coding mode of the previous macroblock of the current macroblock is Intra4×4, use the method of two generalized difference extensions to convert the feature vector of the previous macroblock of the current macroblock w ^Intra4×4 (k-1) is embedded into the host vector x(k) of the current macroblock to obtain the vector embedded with feature information corresponding to the current macroblock then use Replace all the elements in the current macroblock in sequence with the 8th to 16th quantized DCT coefficients in the feature information embedding block, and then perform steps ①-6; if the coding mode of the previous macroblock of the current macroblock is Intra16 ×16, then use a generalized differential extension method to embed the feature vector w ^Intra16×16 (k-1) of the previous macroblock of the current macroblock into the host vector x(k) of the current macroblock to obtain the current macroblock The corresponding vector embedded with feature information then use All elements in replace the 8th to 16th quantized DCT coefficients in the feature information embedding block in the current macroblock in order, and then perform steps ①-6;

①-6. Modify the CBP of the current macroblock, and then perform steps ①-7;

①-7. Entropy encoding is performed on the current macroblock;

①-8. Make k=k+1, use the next pre-encoded macroblock in the I frame as the current macroblock, and return to step ①-2 to continue until all pre-encoded macroblocks in the I frame are processed Complete, obtain the coded stream of the I frame that is embedded with feature information, wherein, "=" in k=k+1 is the assignment symbol;

② At the decoding end, feature information is extracted from each macroblock in the I frame embedded with feature information, and the coding mode and brightness prediction mode of each macroblock are determined, and then error recovery is performed on incorrectly decoded blocks, specifically The process is:

②-1. Perform entropy decoding on each macroblock in the I frame embedded with feature information, then determine whether each macroblock after entropy decoding is a correct decoded block, and then determine the first macroblock after entropy-decoding The vector containing feature information of each correctly decoded block except the first macroblock after entropy decoding extracts the feature information from the vector containing feature information of each correctly decoded block except the first macroblock after entropy decoding, and determines The encoding mode and brightness prediction mode of the previous macroblock of each correctly decoded block outside the first macroblock after entropy decoding, assuming that the k'th macroblock after entropy decoding is a correctly decoded block, then

The specific process of determining the vector containing feature information of the macroblock is: calculating the sum of the absolute values of all the quantized DCT coefficients of each 4×4 block in the macroblock which are dequantized as AC DCT coefficients, and the sum The 4×4 block with the largest value is used as the feature information extraction block, and then all quantized DCT coefficients in the feature information extraction block are scanned in zig-zag mode, and then the eighth one in the feature information extraction block is scanned in the order of zig-zag mode The 16th quantized DCT coefficients are arranged to form a one-dimensional vector containing 9 elements, and then the one-dimensional vector is used as the vector containing feature information of the macroblock, which is denoted as in, with Corresponding representation The 1st element, the 2nd element and the 9th element in ;

The vector containing feature information from the macroblock Extract feature information from the macroblock, and determine the encoding mode and brightness prediction mode of the previous macroblock of the macroblock;

Above, 2≤k'≤K;

②-2. Define the kth macroblock in the decoded I frame as the current macroblock, wherein, 1≤k≤K, and the initial value of k is 1;

②-3, judge whether the current macroblock is the last macroblock in the decoded I frame, if the current macroblock is not the last macroblock, then perform step ②-4, if the current macroblock is the last macroblock, then Execute steps ②-5;

②-4. Determine whether the current macroblock is a correctly decoded block or an incorrectly decoded block. If the current macroblock is a correctly decoded block, then do not process the current macroblock, and then perform step ②-6;

If the current macroblock is an incorrectly decoded block, then determine whether the next macroblock of the current macroblock is a correctly decoded block or an incorrectly decoded block, and if the next macroblock of the current macroblock is a correctly decoded block, use the current macroblock The encoding mode and brightness prediction mode restore the current macroblock, and then perform step ②-6; if the next macroblock of the current macroblock is an incorrectly decoded block, use bilinear interpolation to restore the current macroblock;

②-5. Determine whether the current macroblock is a correctly decoded block or an incorrectly decoded block. If the current macroblock is a correctly decoded block, the current macroblock is not processed. So far, all incorrectly decoded blocks in the decoded I frame are completed recovery; if the current macroblock is an incorrectly decoded block, the bilinear interpolation method is used to restore the current macroblock, and the recovery of all incorrectly decoded blocks in the decoded I frame is completed so far;

②-6, make k=k+1, use the next macroblock to be processed in the decoded I frame as the current macroblock, then return to step ②-3 to continue execution, wherein, " in k=k+1 =" is an assignment symbol.

In the step ①-5, the method of two generalized differential extensions is used to embed the feature vector w ^Intra4×4 (k-1) of the previous macroblock of the current macroblock into the host vector x(k) of the current macroblock The specific process is:

A1. Carry out positive transformation to x(k) to obtain vector y(k), y(k)=(y ₁ y ₂ ... y ₉ ), wherein, y ₁ , y ₂ and y ₉ correspond to y(k) The 1st element, the 2nd element and the 9th element of the , y ₂ ＝x ₂ -x ₁ , y _i' ＝ _xi' -x ₁ , y ₉ ＝x ₉ -x ₁ , 2≤i'≤9, α _i is weight, symbol is the rounding down symbol;

A2. Embed the first to eighth elements of w ^Intra4×4 (k-1) into y(k) to get a vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with Correspondingly represents the first element, the i'-1th element and the 8th element in w ^Intra4×4 (k-1);

A3, yes Perform an inverse transformation to obtain a vector embedded with part of the feature information in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the ,

2≤i'≤9;

A4, yes Perform forward transformation to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight;

A5. Embed the 9th to 16th elements in w ^Intra4×4 (k-1) into , get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with Correspondingly represent the 9th element, the i'+7th element and the 16th element in w ^Intra4×4 (k-1);

A6, yes Perform inverse transformation to obtain the vector embedded with feature information corresponding to the current macroblock in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9.

In the step ①-5, the feature vector w ^Intra16×16 (k-1) of the previous macroblock of the current macroblock is embedded into the host vector x(k) of the current macroblock by using a generalized differential extension method The specific process is:

B1. Carry out positive transformation on x(k) to obtain vector y(k), y(k)=(y ₁ y ₂ ... y ₉ ), wherein, y ₁ , y ₂ and y ₉ correspond to y(k) The 1st element, the 2nd element and the 9th element of the , y _i' ＝x _i' -x ₁ , y ₉ ＝x ₉ -x ₁ , 2≤i'≤9, α _i is weight, symbol is the rounding down symbol;

B2. Embed w ^Intra16×16 (k-1) into y(k) to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with Correspondingly represents the first element, the i'-1th element and the 8th element in w ^Intra16×16 (k-1);

B3, yes Perform inverse transformation to obtain the vector embedded with feature information corresponding to the current macroblock in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9.

The specific process of modifying the CBP of the current macroblock in the described step 1.-6 is:

①-6a. If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the upper left corner of the current macroblock, then count the 8×8 at the upper left corner of the current macroblock after embedding the feature information Whether all the quantized DCT coefficients in the block are all 0, if all are 0, set a ₆ in A to 0, a ₃ , a ₄ , a ₅ remain unchanged, if not all 0, set a 6 in A a ₆ is set to 1, a ₃ , a ₄ , a ₅ remain unchanged;

If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block in the upper right corner of the current macroblock, then count all the 8×8 blocks in the upper right corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₅ in A to 0, a ₃ , a ₄ , a ₆ remain unchanged, if they are not all 0, set a ₅ in A to 0 1, a ₃ , a ₄ , a ₆ remain unchanged;

If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the lower left corner of the current macroblock, then count all the 8×8 blocks at the lower left corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₄ in A to 0, a ₃ , a ₅ , a ₆ remain unchanged, if they are not all 0, set a ₄ in A to 0 1, a ₃ , a ₅ , a ₆ remain unchanged;

If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the lower right corner of the current macroblock, then count all the 8×8 blocks at the lower right corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₃ in A to 0, a ₄ , a ₅ , a ₆ remain unchanged, if they are not all 0, set a ₃ in A to 0 1, a ₄ , a ₅ , a ₆ remain unchanged;

①-6b. Convert the modified A into a decimal number to obtain the modified CBP of the current macroblock.

In the step ②-1, the vector containing feature information from the macroblock The specific process of extracting feature information from the macroblock and determining the encoding mode and brightness prediction mode of the previous macroblock of the macroblock is as follows:

②-1a, yes Perform forward transformation to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight, symbol is the rounding down symbol;

②-1b, from The feature information consisting of 8 feature information bits is extracted from the feature information, which is denoted as in, with Corresponding representation The 1st element, the 2nd element and the 8th element in the , 1≤j≤8, the symbol "||" is the absolute value operation symbol;

②-1c. Construct a new vector z(k'), z(k')=(z ₁ z ₂ ... z ₉ ), where z ₁ , z ₂ and z ₉ correspond to the z(k') 1st element, 2nd element and 9th element, 2≤i'≤9;

②-1d. Perform inverse transformation on z(k') to obtain vector f(k'), f(k')=(f ₁ f ₂ …f ₉ ), where f ₁ , f ₂ and f ₉ correspond to represent The 1st element, the 2nd element and the 9th element in f(k'), f ₂ =z ₂ +f ₁ , f _i' =z _i' +f ₁ , f ₉ =z ₉ +f ₁ , 2≤i'≤9;

②-1e, will be made by The first element in 2nd element 3rd element and the 4th element The composed binary string is converted into a decimal value. If the decimal value corresponds to the number 9, it means that the coding mode of the previous macroblock of this macroblock is Intra16×16, and then replace the macro with all elements in f(k') in order The feature information in the previous macroblock of the block is extracted from the 8th to 16th quantized DCT coefficients in the block, and then the brightness prediction mode of the previous macroblock of the macroblock is determined, and the brightness of the previous macroblock of the macroblock is determined. The numerical identification of the predictive mode is given by The 5th element in 6th element 7th element and the 8th element The decimal value converted from the composed binary string ends feature information extraction and determination of encoding mode and brightness prediction mode;

If the decimal value does not correspond to the number 9, it means that the coding mode of the previous macroblock of this macroblock is Intra4×4, and then perform step ②-1f;

②-1f, perform positive transformation on f(k') to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight, symbol is the rounding down symbol;

②-1g, from The feature information consisting of 8 feature information bits is extracted from the feature information, which is denoted as in, with Corresponding representation The 1st element, the 2nd element and the 8th element in the , 1≤j≤8, the symbol "||" is the absolute value operation symbol;

②-1h. Construct a new vector z'(k'), z'(k')=(z ₁ 'z ₂ '...z ₉ '), where z ₁ ', z ₂ ' and z ₉ ' correspond to Indicates the 1st element, the 2nd element and the 9th element in z'(k'), 2≤i'≤9;

②-1i. Perform inverse transformation on z'(k') to obtain vector x(k'), x(k')=(x ₁ x ₂ ... x ₉ ), where x ₁ , x ₂ and x ₉ correspond to Represents the 1st element, the 2nd element and the 9th element in x(k'), x ₂ =z ₂ '+x ₁ , x _i' =z _i' '+x ₁ , x ₉ =z ₉ '+x ₁ , 2≤i'≤9;

②-1j, replacing the 8th to 16th quantized DCT coefficients in the feature information extraction block in the previous macroblock of the macroblock in sequence with all elements in x(k');

②-1k, construct a vector w(k') containing 16 elements, the first 8 elements of w(k') are The 8 elements of w(k'), the last 8 elements of w(k') are 8 elements of

②-11. Determine the luminance prediction mode of the 4×4 block numbered 0 in the previous macroblock of the macroblock, and the number of the luminance prediction mode of the 4×4 block is identified by the first in w(k') The decimal value converted from the binary string composed of the first element, the second element, the third element and the fourth element;

Determine the luminance prediction mode of the 4×4 block numbered 4 in the previous macroblock of the macroblock, and the number of the luminance prediction mode of the 4×4 block is identified by the fifth element in w(k'), the first The decimal value converted from the binary string composed of 6 elements, the 7th element and the 8th element;

Determine the luminance prediction mode of the 4×4 block numbered 8 in the previous macroblock of the macroblock, the number of the luminance prediction mode of the 4×4 block is identified by the 9th element in w(k'), the 9th element The decimal value converted from the binary string composed of 10 elements, the 11th element and the 12th element;

Determine the luminance prediction mode of the 4×4 block numbered 12 in the previous macroblock of the macroblock, the number of the luminance prediction mode of the 4×4 block is identified by the 13th element in w(k'), the th The decimal value converted from the binary string composed of 14 elements, the 15th element and the 16th element;

End feature information extraction and determination of encoding mode and brightness prediction mode.

The specific process of using the encoding mode and brightness prediction mode of the current macroblock in the described step ②-4 to restore the current macroblock is:

②-4a. If the encoding mode of the current macroblock is Intra16×16, use the brightness prediction mode of the current macroblock to predict the pixel value of each pixel in the current macroblock, and then use The predicted value of the pixel is used as the final restored pixel value of the corresponding pixel to complete the restoration of the current macroblock;

②-4b. If the encoding mode of the current macroblock is Intra4×4, use the brightness prediction mode of the 4×4 block numbered 0 in the current macroblock to each of the 8×8 blocks where the 4×4 block is located The pixel value of each pixel in the 4×4 block is predicted, and then the predicted value of each pixel in each 4×4 block is used as the final restored pixel corresponding to each pixel in the 4×4 block value;

Use the brightness prediction mode of the 4×4 block numbered 4 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then Taking the predicted value of each pixel in each 4×4 block as the final restored pixel value of each pixel in the corresponding 4×4 block;

Use the brightness prediction mode of the 4×4 block numbered 8 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then Taking the predicted value of each pixel in each 4×4 block as the final restored pixel value of each pixel in the corresponding 4×4 block;

Use the brightness prediction mode of the 4×4 block numbered 12 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then The predicted value of each pixel in each 4×4 block is used as the final restored pixel value of each pixel in the corresponding 4×4 block.

Compared with the prior art, the present invention has the advantages of:

1) At the coding end, the feature information of the extracted macroblock is determined by the brightness prediction mode of the sub-block of the macroblock (the coding mode is Intra4×4) or is determined by the coding mode and brightness prediction of the macroblock (the coding mode is Intra16×16) The mode is determined, which is not only convenient for extraction, but also can effectively restore the lost macroblock when there is a scene change, thereby improving the quality of I frame error recovery.

2) At the encoding end, when embedding the feature vector into the host vector, according to the encoding mode of the previous macroblock of the currently processed macroblock, the method of two generalized differential extensions or one generalized differential extension is used for embedding. The differential expansion method not only makes the embedding capacity controllable, but also can restore the original data of the I frame before embedding the feature information after extracting the feature information.

3) The difference object of the generalized difference extension method adopted by the method of the present invention is a quantized DCT coefficient, and there is no overflow in the difference, and the computational complexity is greatly reduced.

4) The method of the present invention carries out I frame error recovery by means of information hiding technology, makes full use of the information of encoding end, compared with only carrying out error concealment at decoding end, the available video resources are richer, more accurate, and more flexible.

Description of drawings

Fig. 1 is the overall realization block diagram of the inventive method;

Fig. 2 is a schematic diagram of numbering of 16 4×4 blocks in a macroblock;

When Fig. 3 a is encoding quantization parameter QP=28, the I frame of Foreman standard test sequence is under different macroblock loss rates, adopts the comparison schematic diagram of the error recovery objective quality after the method of the present invention and existing JM method and RH method processing;

When Fig. 3 b is the encoding quantization parameter QP=28, the I frame of Carphone standard test sequence is under different macroblock loss rates, adopts the comparison schematic diagram of the error recovery objective quality after the method of the present invention and existing JM method and RH method processing;

Fig. 3c is when the encoding quantization parameter QP=28, the I frame of the Container standard test sequence is under different macroblock loss rates, adopts the method of the present invention and the existing JM method and the RH method after the error recovery objective quality comparison schematic diagram;

Figure 3d is a schematic diagram of comparing the objective quality of error recovery after the method of the present invention and the existing JM method and RH method are used for the I frame of the Akyio standard test sequence under different macroblock loss rates when the coding quantization parameter QP=28;

Fig. 3 e is when the encoding quantization parameter QP=28, the I frame of the Silent standard test sequence is under different macroblock loss rates, adopts the method of the present invention and the existing JM method and the RH method after the error recovery objective quality comparison schematic diagram;

Fig. 3 f is when coding quantization parameter QP=28, the I frame of Mobile standard test sequence is under different macroblock loss rates, adopts the error recovery objective quality comparison schematic diagram after the method of the present invention and existing JM method and RH method to process;

Fig. 4 a is when coding quantization parameter QP=38, and the I frame of Foreman standard test sequence is under different macroblock loss rates, adopts the comparison schematic diagram of the error recovery objective quality after the method of the present invention and existing JM method and RH method to process;

When Fig. 4 b is the encoding quantization parameter QP=38, the I frame of Carphone standard test sequence is under different macroblock loss rates, adopts the comparison schematic diagram of the error recovery objective quality after the method of the present invention and existing JM method and RH method processing;

Fig. 4c is when the encoding quantization parameter QP=38, the I frame of the Container standard test sequence is under different macroblock loss rates, adopts the method of the present invention and the existing JM method and the RH method after the error recovery objective quality comparison schematic diagram;

Figure 4d is a schematic diagram of comparing the objective quality of error recovery after the method of the present invention and the existing JM method and RH method are used for the I frame of the Akyio standard test sequence under different macroblock loss rates when the coding quantization parameter QP=38;

Fig. 4e is when the quantization parameter QP=38 of encoding, the I frame of Silent standard test sequence is under different macroblock loss rate, adopts the method of the present invention and the existing JM method and the RH method to process the error recovery objective quality comparative schematic diagram;

Fig. 4 f is when coding quantization parameter QP=38, the I frame of Mobile standard test sequence is under different macroblock loss rates, adopts the error recovery objective quality comparison schematic diagram after the method of the present invention and existing JM method and RH method to process;

Fig. 5 a is the original picture of the 20th frame (I frame) of Carphone standard test sequence;

Fig. 5b is the figure after block loss in Fig. 5a (block loss rate is 20%);

Fig. 5c is the figure after adopting existing JM method to restore Fig. 5b;

Fig. 5d is the figure after adopting existing RH method to restore Fig. 5b;

Fig. 5e is the figure after adopting the method of the present invention to restore Fig. 5b;

Figure 6a is a schematic diagram of comparing the objective quality of error recovery after processing the method of the present invention with the existing JM method and RH method for the first 150 frames of the composite sequence under different macroblock loss rates when the coding quantization parameter QP=28;

Figure 6b is a schematic diagram of comparing the objective quality of error recovery after processing the first 150 frames of the synthesized sequence under different macroblock loss rates when the coding quantization parameter QP=38, using the method of the present invention and the existing JM method and RH method;

Figure 7a is the original picture of the 29th frame of the synthesized sequence;

Figure 7b is the original picture of the 30th frame (I frame) of the synthetic sequence;

Fig. 7c is the picture after the block loss of the image shown in Fig. 7b (block loss rate is 20%);

Figure 7d is the subjective quality map after recovering Figure 7c using the existing JM method;

Figure 7e is the subjective quality map after recovering Figure 7c using the existing RH method;

Fig. 7f is a subjective quality map after restoration of Fig. 7c by the method of the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

A kind of H.264/AVC video I frame error recovery method based on reversible data hiding that the present invention proposes, its overall realization block diagram is as shown in Figure 1, and it comprises the following steps:

① At the encoding end, embed feature information into each macroblock except the first macroblock in the I frame to obtain an I frame embedded with feature information. The specific process is as follows:

①-1. Define the kth macroblock that is currently pre-encoded in the I frame as the current macroblock, where 1≤k≤K, where K represents the total number of macroblocks contained in the I frame, and k The initial value is 1.

①-2. Convert the decimal value of the CBP of the current macroblock into a binary string composed of 6-bit binary values, and record it as A, A=a ₁ a ₂ a ₃ a ₄ a ₅ a ₆ , where a ₁ is The highest binary value of A, a ₆ is the lowest binary value of A.

The upper two bits of the binary string corresponding to the CBP of the macroblock are used to represent chroma, and the lower four bits are used to represent brightness; a ₆ represents the statistics of the quantized DCT coefficients of the 8×8 block at the upper left corner of the macroblock, if a ₆ is 0, it means that the quantized DCT coefficients of the four 4×4 blocks in the 8×8 block are all 0, if a ₆ is 1, it means that the quantization of the four 4×4 blocks in the 8×8 block The DCT coefficients are not all 0; a ₅ represents the statistics of the quantized DCT coefficients of the 8×8 block at the upper right corner of the macroblock, and a ₄ represents the statistics of the quantized DCT coefficients of the 8×8 block at the lower left corner of the macroblock , a ₃ represents the statistics of the quantized DCT coefficients of the 8×8 block at the lower right corner of the macroblock.

①-3. Extract the feature vector of the current macroblock: if the encoding mode of the current macroblock is Intra4×4, then extract the four 4s numbered 0, 4, 8, and 12 (as shown in Figure 2) in the current macroblock ×4 block’s respective luminance prediction mode digital identification, then quantize the digital identification of these four 4×4 blocks’ respective luminance prediction modes into 4 bits, and then press the four 4×4 blocks in the current macroblock The sequence of numbering arranges the luminance prediction modes of these four 4×4 blocks with digital identifiers represented by bits to form a one-dimensional vector containing 16 elements, and then uses the one-dimensional vector as the feature vector of the current macroblock, which is recorded as in, with Correspondingly represent the first element, the second element and the 16th element in the feature vector w ^Intra4×4 (k) of the current macroblock.

If the encoding mode of the current macroblock is Intra16×16, the number 9 is used to identify the encoding mode of the current macroblock, and then the digital identification of the encoding mode of the current macroblock is quantized to 4 bits, and the brightness prediction mode of the current macroblock is The digital identification of the current macroblock is quantized into 4 bits, and then the current macroblock coding mode and the current macroblock's luminance prediction mode are arranged in order to form a one-dimensional matrix containing 8 elements. vector, and then use the one-dimensional vector as the feature vector of the current macroblock, denoted as w ^Intra16×16 (k), in, with Correspondingly represent the first element, the second element and the eighth element in the feature vector w ^Intra16×16 (k) of the current macroblock.

In H.264/AVC video, there are 9 kinds of luminance prediction modes for the 4×4 blocks in the macroblock whose encoding mode is Intra4×4, which are represented by numbers 0 to 8 respectively; the macroblock whose encoding mode is Intra16×16 There are 4 types of brightness prediction modes for a block, which are represented by numbers 0 to 3 respectively.

①-4. Determine the host vector of the current macroblock: calculate the sum of the absolute values of all AC DCT coefficients of each 4×4 block in the current macroblock, and embed the 4×4 block with the largest sum value as the feature information into the block, Then scan all the quantized DCT coefficients in the feature information embedding block in zig-zag mode, and then embed the feature information into the 8th to 16th quantized DCT coefficients in the block in the zig-zag scan order (located in the zig-zag scan The 8th to 16th quantized DCT coefficients are medium and high frequency quantized DCT coefficients) arranged to form a one-dimensional vector containing 9 elements, and then the one-dimensional vector is used as the host vector of the current macroblock, denoted as x(k) , x(k)=(x ₁ x ₂ …x ₉ ), where x ₁ , x ₂ and x ₉ correspond to the first element, the second element and 9th element.

①-5. Embed the eigenvector of the previous macroblock of the current macroblock into the host vector of the current macroblock: determine whether the current macroblock is the first macroblock in the I frame, and if so, perform Do not process, directly execute steps ①-7; otherwise, if the coding mode of the previous macroblock of the current macroblock is Intra4×4, use the method of two generalized difference extensions to convert the feature vector of the previous macroblock of the current macroblock w ^Intra4×4 (k-1) is embedded into the host vector x(k) of the current macroblock to obtain the vector embedded with feature information corresponding to the current macroblock then use Replace all the elements in the current macroblock in sequence with the 8th to 16th quantized DCT coefficients in the feature information embedding block, and then perform steps ①-6; if the coding mode of the previous macroblock of the current macroblock is Intra16 ×16, then use a generalized differential extension method to embed the feature vector w ^Intra16×16 (k-1) of the previous macroblock of the current macroblock into the host vector x(k) of the current macroblock to obtain the current macroblock The corresponding vector embedded with feature information then use All elements in replace the 8th to 16th quantized DCT coefficients in the feature information embedding block in the current macroblock in sequence, and then perform step ①-6.

In this specific embodiment, in steps ①-5, two generalized differential extension methods are used to embed the feature vector w ^Intra4×4 (k-1) of the previous macroblock of the current macroblock into the host vector x of the current macroblock The specific process in (k) is:

A1. Carry out positive transformation to x(k) to obtain vector y(k), y(k)=(y ₁ y ₂ ... y ₉ ), wherein, y ₁ , y ₂ and y ₉ correspond to y(k) The 1st element, the 2nd element and the 9th element of the , y ₂ =x ₂ -x ₁ , y _i' = _xi' -x ₁ , y ₉ =x ₉ -x ₁ , 2≤i'≤9, α _i is the weight, and α _i = 1. Symbol is the rounding down symbol.

A2. Embed the first to eighth elements of w ^Intra4×4 (k-1) into y(k) to get a vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with It corresponds to the 1st element, the i'-1th element and the 8th element in w ^Intra4×4 (k-1).

A3, yes Perform an inverse transformation to obtain a vector embedded with part of the feature information in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9.

A4, yes Perform forward transformation to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight.

A5. Embed the 9th to 16th elements in w ^Intra4×4 (k-1) into , get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with Correspondingly represent the 9th element, the i'+7th element and the 16th element in w ^Intra4×4 (k-1).

A6, yes Perform inverse transformation to obtain the vector embedded with feature information corresponding to the current macroblock Finish embedding the feature vector w ^Intra4×4 (k-1) of the previous macroblock of the current macroblock into the host vector x(k) of the current macroblock, in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9.

In this specific embodiment, step ①-5 adopts a generalized differential extension method to embed the feature vector w ^Intra16×16 (k-1) of the previous macroblock of the current macroblock into the host vector x( The specific process in k) is:

B1. Carry out positive transformation on x(k) to obtain vector y(k), y(k)=(y ₁ y ₂ ... y ₉ ), wherein, y ₁ , y ₂ and y ₉ correspond to y(k) The 1st element, the 2nd element and the 9th element of the , y _i' = x _i' -x ₁ , y ₉ = x ₉ -x ₁ , 2≤i'≤9, α _i is the weight value, α _i =1 is taken in specific implementation, the symbol is the rounding down symbol.

B2. Embed w ^Intra16×16 (k-1) into y(k) to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with It corresponds to the 1st element, the i'-1th element and the 8th element in w ^Intra16×16 (k-1).

B3, yes Perform inverse transformation to obtain the vector embedded with feature information corresponding to the current macroblock Finish embedding the feature vector w ^Intra16×16 (k-1) of the previous macroblock of the current macroblock into the host vector x(k) of the current macroblock, in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9.

①-6. Modify the CBP (CodedBlockPattern) of the current macroblock, and then perform step ①-7.

In this specific embodiment, the specific process of modifying the CBP of the current macroblock in step 1.-6 is:

①-6a. If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the upper left corner of the current macroblock, then count the 8×8 at the upper left corner of the current macroblock after embedding the feature information Whether all the quantized DCT coefficients in the block are all 0, if all are 0, set a ₆ in A to 0, a ₃ , a ₄ , a ₅ remain unchanged, if not all 0, set a 6 in A a ₆ is set to 1, and a ₃ , a ₄ , a ₅ remain unchanged.

If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block in the upper right corner of the current macroblock, then count all the 8×8 blocks in the upper right corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₅ in A to 0, a ₃ , a ₄ , a ₆ remain unchanged, if they are not all 0, set a ₅ in A to 0 1, a ₃ , a ₄ , a ₆ remain unchanged.

If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the lower left corner of the current macroblock, then count all the 8×8 blocks at the lower left corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₄ in A to 0, a ₃ , a ₅ , a ₆ remain unchanged, if they are not all 0, set a ₄ in A to 0 1, a ₃ , a ₅ , a ₆ remain unchanged.

If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the lower right corner of the current macroblock, then count all the 8×8 blocks at the lower right corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₃ in A to 0, a ₄ , a ₅ , a ₆ remain unchanged, if they are not all 0, set a ₃ in A to 0 1, a ₄ , a ₅ , a ₆ remain unchanged.

①-6b. Convert the modified A into a decimal number to obtain the modified CBP of the current macroblock.

①-7. Perform entropy coding on the current macroblock; in specific implementation, the existing context-adaptive variable length coding (CAVLC) technology can be used to perform entropy coding on the current macroblock.

①-8. Make k=k+1, use the next pre-encoded macroblock in the I frame as the current macroblock, and return to step ①-2 to continue until all pre-encoded macroblocks in the I frame are processed After completion, the coded code stream of the I frame embedded with feature information is obtained, wherein "=" in k=k+1 is an assignment symbol.

② At the decoding end, feature information is extracted from each macroblock in the I frame embedded with feature information, and the coding mode and brightness prediction mode of each macroblock are determined, and then error recovery is performed on incorrectly decoded blocks, specifically The process is:

②-1. Perform entropy decoding on each macroblock in the I frame embedded with feature information, then determine whether each macroblock after entropy decoding is a correct decoded block, and then determine the first macroblock after entropy-decoding (the first macroblock may be a correctly decoded block or an incorrectly decoded block, and the first macroblock is not processed here), the vector containing feature information of each correctly decoded block, and then decoded from the entropy Extract the feature information from the vector containing feature information of each correctly decoded block except the first macroblock after entropy decoding, and determine the previous macro of each correctly decoded block except the first macroblock after entropy decoding The encoding mode and brightness prediction mode of the block, assuming that the k'th macroblock after entropy decoding is the correct decoding block, then

The specific process of determining the vector containing feature information of the macroblock is: calculating the sum of the absolute values of all the quantized DCT coefficients of each 4×4 block in the macroblock that are dequantized as AC DCT coefficients, that is, calculating The sum of the absolute values of the partially quantized DCT coefficients of each 4×4 block in the macroblock, where each quantized DCT coefficient in the partially quantized DCT coefficients is dequantized into an AC DCT coefficient, and the 4 with the largest sum value ×4 block as the feature information extraction block, and then scan all the quantized DCT coefficients in the feature information extraction block in zig-zag mode, and then scan the 8th to 16th coefficients in the feature information extraction block in the order of zig-zag scanning The quantized DCT coefficients (the 8th to 16th quantized DCT coefficients in the zig-zag scan are medium and high frequency quantized DCT coefficients) are arranged to form a one-dimensional vector containing 9 elements, and then the one-dimensional vector is used as the macroblock A vector containing feature information, denoted as in, with Corresponding representation The 1st element, the 2nd element and the 9th element in .

The vector containing feature information from the macroblock The feature information is extracted from the macroblock, and the encoding mode and brightness prediction mode of the previous macroblock of the macroblock are determined.

In this specific embodiment, in step ②-1, from the vector containing feature information of the macroblock The specific process of extracting feature information from the macroblock and determining the encoding mode and brightness prediction mode of the previous macroblock of the macroblock is as follows:

②-1a, yes Perform forward transformation to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight value, and α _i =1 is taken in specific implementation, the symbol is the rounding down symbol.

②-1b, from The feature information consisting of 8 feature information bits is extracted from the feature information, which is denoted as in, with Corresponding representation The 1st element, the 2nd element and the 8th element in the , 1≤j≤8, the symbol "||" is the absolute value operation symbol.

②-1c. Construct a new vector z(k'), z(k')=(z ₁ z ₂ ... z ₉ ), where z ₁ , z ₂ and z ₉ correspond to the z(k') 1st element, 2nd element and 9th element, 2≤i'≤9.

②-1d. Perform inverse transformation on z(k') to obtain vector f(k'), f(k')=(f ₁ f ₂ …f ₉ ), where f ₁ , f ₂ and f ₉ correspond to represent The 1st element, the 2nd element and the 9th element in f(k'), f ₂ =z ₂ +f ₁ , f _i' =zi _' +f ₁ , f ₉ =z ₉ +f ₁ , 2≤i'≤9.

②-1e, will be made by The first element in 2nd element 3rd element and the 4th element The composed binary string is converted into a decimal value. If the decimal value corresponds to the number 9, it means that the coding mode of the previous macroblock of this macroblock is Intra16×16, and then replace the macro with all elements in f(k') in order Extract the 8th to 16th quantized DCT coefficients in the block from the feature information in the previous macroblock of the block, that is, restore the original quantized DCT coefficients, and then determine the brightness prediction mode of the previous macroblock of the macroblock, the macro The numerical identification of the luma prediction mode of the previous macroblock of the block is given by The 5th element in 6th element 7th element and the 8th element The formed binary string is converted into a decimal value, and the extraction of feature information and the determination of the encoding mode and the brightness prediction mode are completed.

If the decimal value does not correspond to the number 9, it means that the coding mode of the previous macroblock of this macroblock is Intra4×4, and then step ②-1f is performed.

②-1f, perform positive transformation on f(k') to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight value, and α _i =1 is taken in specific implementation, the symbol is the rounding down symbol.

②-1g, from The feature information consisting of 8 feature information bits is extracted from the feature information, which is denoted as in, with Corresponding representation The 1st element, the 2nd element and the 8th element in the , 1≤j≤8, the symbol "||" is the absolute value operation symbol.

②-1h. Construct a new vector z'(k'), z'(k')=(z ₁ 'z ₂ '...z ₉ '), where z ₁ ', z ₂ ' and z ₉ ' correspond to Indicates the 1st element, the 2nd element and the 9th element in z'(k'), 2≤i'≤9.

②-1i. Perform inverse transformation on z'(k') to obtain vector x(k'), x(k')=(x ₁ x ₂ ... x ₉ ), where x ₁ , x ₂ and x ₉ correspond to Represents the 1st element, the 2nd element and the 9th element in x(k'), x ₂ =z ₂ '+x ₁ , x _i' =zi _' '+x ₁ , x ₉ =z ₉ '+x ₁ , 2≤i'≤9.

②-1j. Replace the 8th to 16th quantized DCT coefficients in the feature information extraction block in the previous macroblock of the macroblock in sequence with all elements in x(k'), that is, restore the original quantized DCT coefficient.

②-1k, construct a vector w(k') containing 16 elements, the first 8 elements of w(k') are The 8 elements of w(k'), the last 8 elements of w(k') are of 8 elements.

②-11. Determine the luminance prediction mode of the 4×4 block numbered 0 in the previous macroblock of the macroblock, and the number of the luminance prediction mode of the 4×4 block is identified by the first in w(k') The decimal value converted from the binary string composed of the first element, the second element, the third element and the fourth element.

Determine the luminance prediction mode of the 4×4 block numbered 4 in the previous macroblock of the macroblock, and the number of the luminance prediction mode of the 4×4 block is identified by the fifth element in w(k'), the first The decimal value converted from the binary string composed of 6 elements, the 7th element and the 8th element.

Determine the luminance prediction mode of the 4×4 block numbered 8 in the previous macroblock of the macroblock, the number of the luminance prediction mode of the 4×4 block is identified by the 9th element in w(k'), the 9th element The decimal value converted from the binary string consisting of 10 elements, the 11th element, and the 12th element.

Determine the luminance prediction mode of the 4×4 block numbered 12 in the previous macroblock of the macroblock, the number of the luminance prediction mode of the 4×4 block is identified by the 13th element in w(k'), the th The decimal value converted from the binary string consisting of 14 elements, the 15th element, and the 16th element.

End feature information extraction and determination of encoding mode and brightness prediction mode.

As mentioned above, 2≤k'≤K.

Existing CAVLC entropy decoding technology can be used in specific implementation to carry out entropy decoding to each macroblock in the I frame that is embedded with feature information; Determine whether each macroblock after entropy decoding is a correct decoding block directly adopts existing technology .

②-2. Define the kth macroblock in the decoded I frame as the current macroblock, where 1≤k≤K, and the initial value of k is 1.

②-3, judge whether the current macroblock is the last macroblock in the decoded I frame, if the current macroblock is not the last macroblock, then perform step ②-4, if the current macroblock is the last macroblock, then Execute steps ②-5.

②-4. Determine whether the current macroblock is a correctly decoded block or an incorrectly decoded block. If the current macroblock is a correctly decoded block, then do not process the current macroblock, and then perform step ②-6.

If the current macroblock is an incorrectly decoded block, then determine whether the next macroblock of the current macroblock is a correctly decoded block or an incorrectly decoded block, and if the next macroblock of the current macroblock is a correctly decoded block, use the current macroblock The coding mode and brightness prediction mode restore the current macroblock, and then perform step ②-6; if the next macroblock of the current macroblock is an incorrectly decoded block, use the existing bilinear interpolation method to restore the current macroblock.

In this specific embodiment, the specific process of recovering the current macroblock using the encoding mode and brightness prediction mode of the current macroblock in step ②-4 is:

②-4a. If the encoding mode of the current macroblock is Intra16×16, use the brightness prediction mode of the current macroblock to predict the pixel value of each pixel in the current macroblock, and then use The predicted value of the pixel is used as the final restored pixel value of the corresponding pixel to complete the restoration of the current macroblock.

②-4b. If the encoding mode of the current macroblock is Intra4×4, use the brightness prediction mode of the 4×4 block numbered 0 in the current macroblock to each of the 8×8 blocks where the 4×4 block is located The pixel value of each pixel in the 4×4 block is predicted, and then the predicted value of each pixel in each 4×4 block is used as the final restored pixel corresponding to each pixel in the 4×4 block value.

Use the brightness prediction mode of the 4×4 block numbered 4 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then The predicted value of each pixel in each 4×4 block is used as the final restored pixel value of each pixel in the corresponding 4×4 block.

Use the brightness prediction mode of the 4×4 block numbered 8 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then The predicted value of each pixel in each 4×4 block is used as the final restored pixel value of each pixel in the corresponding 4×4 block.

Use the brightness prediction mode of the 4×4 block numbered 12 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then The predicted value of each pixel in each 4×4 block is used as the final restored pixel value of each pixel in the corresponding 4×4 block.

②-5. Determine whether the current macroblock is a correctly decoded block or an incorrectly decoded block. If the current macroblock is a correctly decoded block, the current macroblock is not processed. So far, all incorrectly decoded blocks in the decoded I frame are completed recovery; if the current macroblock is an incorrectly decoded block, the existing bilinear interpolation method is used to restore the current macroblock, and the recovery of all incorrectly decoded blocks in the decoded I frame is completed so far.

②-6, make k=k+1, use the next macroblock to be processed in the decoded I frame as the current macroblock, then return to step ②-3 to continue execution, wherein, " in k=k+1 =" is an assignment symbol.

The method of the present invention chooses to carry out the simulation experiment on the test model JM-12.0 of H.264/AVC. In the simulation, the standard QCIF format (176×144) video sequences Foreman, Carphone, Container, Akyio, Silent and Mobile are encoded and tested using H.264/AVC basic profile. In order to verify the effectiveness of the method of the present invention for existing scene changes, a mixed sequence composed of Akiyo, Bridge-close, and Carphone is used to simulate the presence of scene changes. Two groups of test experiments respectively when the loss rate is 10% and 20%, the method of the present invention is combined with the intra-frame recovery method (JM method) carried by the existing test model JM-12.0 and the existing method based on Chung et al. The error recovery algorithm (RH method) of histogram shifting reversible information hiding is compared with the experiment. Table 1 shows some basic encoding parameters set.

Table 1 Encoding parameter setting

grade basic class coding structure IPPPP Encoded frames 150 Frame rate (frame/second) 15 entropy coding CAVLC

In order to verify the error recovery effect of the method of the present invention, especially the quality of video error recovery when there are scene changes, two groups of experiments are done. One group is to encode the standard video test sequences Foreman, Carphone, Container, Akyio, Silent and Mobile, test experiments when the packet loss rate is 10% and 20% respectively, compare the method of the present invention with the existing JM method and The subjective and objective quality of the I-frame image processed by the RH method; the other group is to simulate the situation of scene change, that is, to use the synthesized sequence for encoding test, and to test the experiment when the packet loss rate is 10% and 20%, respectively. The subjective and objective quality of the I-frame image processed by the method of the present invention and the existing JM method and the RH method is compared.

Fig. 3 a to Fig. 3 f have provided coding quantization parameter QP=28 respectively, and the I frame of Foreman, Carphone, Container, Akyio, Silent and Mobile standard test sequence is under different macroblock loss rates, adopts the method of the present invention and existing Schematic diagram comparing objective quality of error recovery after processing by JM method and RH method. It can be seen from Fig. 3a to Fig. 3f that when the loss rate is 10% and 20%, the objective quality of error recovery using the method of the present invention is higher than the objective quality of error recovery using the existing JM method. 3.24dB, 2.79dB.

Fig. 4 a to Fig. 4 f have provided respectively when coding quantization parameter QP=38, and the I frame of Foreman, Carphone, Container, Akyio, Silent and Mobile standard test sequence is under different macroblock loss rates, adopts the method of the present invention and existing Schematic diagram comparing objective quality of error recovery after processing by JM method and RH method.

Fig. 5 a has provided the original picture of the 20th frame (I frame) of Carphone standard test sequence, Fig. 5 b has provided the figure after Fig. 5 a loses block (block loss rate 20%), Fig. 5 c has provided and adopted existing Figure 5b is restored by the JM method, Figure 5d shows the restored image of Figure 5b using the existing RH method, and Figure 5e shows the restored image of Figure 5b using the method of the present invention. It can be seen from Fig. 5c to Fig. 5e that the weighted interpolation method used by the JM platform is easy to make the lost macroblock image blurred, and the image restored by the RH method is easy to produce block effects. In contrast, the method of the present invention can be compared It can well restore the edge and texture of the image, avoid block effects, and the subjective quality is significantly better than the existing JM method and RH method.

Figure 6a and Figure 6b respectively show the encoding and quantization parameters QP=28 and QP=38, the first 150 frames of the synthetic sequence synthesized by the three standard test sequences of Akiyo, Carphone, and Bridge-close at intervals of 30 frames in different macros Under the block loss rate, a schematic diagram of comparing the objective quality of error recovery after processing by the method of the present invention and the existing JM method and RH method. It can be seen from Fig. 6a and Fig. 6b that when there is a scene change, the objective restoration quality of the method of the present invention is obviously higher than that of the existing JM method and RH method. When QP=28, macroblock loss rate is 10%, the objective restoration quality adopting the method of the present invention is higher than adopting the objective restoration quality of existing JM method by 2.14dB; When QP=28, macroblock loss rate is 20% , the objective restoration quality using the method of the present invention is 0.93dB higher than the objective restoration quality using the existing JM method. Compared with the existing RH method, the objective recovery quality of the method of the present invention is on average higher by about 1dB, because the RH method is characterized by the motion vectors of two I-frame macroblocks, so when there is a scene change, the RH The method will mistakenly introduce the scene image block of the previous I frame into the current frame, which will inevitably cause the PSNR value to decrease and affect the reconstruction quality.

Figure 7a shows the original picture of the 29th frame of the synthetic sequence, Figure 7b shows the original picture of the 30th frame (I frame) of the synthetic sequence, and Figure 7c shows the block loss of the 30th frame of the synthetic sequence Fig. 7 (block loss rate 20%), Fig. 7d shows the subjective quality map after using the existing JM method to restore Fig. 7c, and Fig. 7e shows the restoration of Fig. 7c using the existing RH method As for the subjective quality map, Fig. 7f shows the subjective quality map after restoration of Fig. 7c by the method of the present invention. It is not difficult to find from the subjective quality diagrams shown in Figures 7d to 7f that the method of the present invention can ensure that the reconstruction quality decreases in a moderate manner when there is a scene change, avoiding block effects and image blurring. In summary, the analysis of the simulation results found that: when there is a scene change, the objective quality and subjective quality of the video reconstructed by the method of the present invention are obviously better than the objective quality and subjective quality of the video reconstructed by the existing JM method and the RH method; When there is no scene change, the method of the present invention also achieves good results.

Claims (1)

1. a kind of H.264/AVC video I frame error recovery method based on reversible data hiding, it is characterized in that comprising the following steps: ① At the encoding end, embed feature information into each macroblock except the first macroblock in the I frame to obtain an I frame embedded with feature information. The specific process is as follows: ①-1. Define the kth macroblock that is currently pre-encoded in the I frame as the current macroblock, where 1≤k≤K, where K represents the total number of macroblocks contained in the I frame, and k The initial value is 1; ①-2. Convert the decimal value of the CBP of the current macroblock into a binary string composed of 6-bit binary values, and record it as A, A=a ₁ a ₂ a ₃ a ₄ a ₅ a ₆ , where a ₁ is The highest binary value of A, a ₆ is the lowest binary value of A; ①-3. Extract the feature vector of the current macroblock: if the encoding mode of the current macroblock is Intra4×4, then extract the respective luminance predictions of the four 4×4 blocks numbered 0, 4, 8, and 12 in the current macroblock The digital identification of the mode, and then quantize the digital identification of the respective luma prediction modes of the four 4×4 blocks into 4 bits, and then the four 4×4 blocks are numbered in the order of the current macroblock. The luminance prediction mode of the 4×4 block is arranged with the digital identification represented by bits to form a one-dimensional vector containing 16 elements, and then the one-dimensional vector is used as the feature vector of the current macroblock, which is recorded as w ^Intra4×4 (k), in, with Correspondingly represent the first element, the second element and the 16th element in the feature vector w ^Intra4×4 (k) of the current macroblock; If the encoding mode of the current macroblock is Intra16×16, the number 9 is used to identify the encoding mode of the current macroblock, and then the digital identification of the encoding mode of the current macroblock is quantized to 4 bits, and the brightness prediction mode of the current macroblock is The digital identification of the current macroblock is quantized into 4 bits, and then the current macroblock coding mode and the current macroblock's luminance prediction mode are arranged in order to form a one-dimensional matrix containing 8 elements. vector, and then use the one-dimensional vector as the feature vector of the current macroblock, denoted as w ^Intra16×16 (k), in, with Correspondingly represent the first element, the second element and the eighth element in the feature vector w ^Intra16×16 (k) of the current macroblock; ①-4. Determine the host vector of the current macroblock: calculate the sum of the absolute values of all AC DCT coefficients of each 4×4 block in the current macroblock, and embed the 4×4 block with the largest sum value as the feature information into the block, Then scan all the quantized DCT coefficients in the feature information embedding block in zig-zag mode, and then arrange the 8th to 16th quantized DCT coefficients in the feature information embedding block in the order of zig-zag scanning to form a set containing 9 The one-dimensional vector of the element, and then use the one-dimensional vector as the host vector of the current macroblock, which is recorded as x(k), x(k)=(x ₁ x ₂ ... x ₉ ), where x ₁ , x ₂ and x ₉ corresponds to the first element, the second element and the ninth element in the host vector x(k) of the current macroblock; ①-5. Embed the eigenvector of the previous macroblock of the current macroblock into the host vector of the current macroblock: determine whether the current macroblock is the first macroblock in the I frame, and if so, perform Do not process, directly execute steps ①-7; otherwise, if the coding mode of the previous macroblock of the current macroblock is Intra4×4, use the method of two generalized difference extensions to convert the feature vector of the previous macroblock of the current macroblock w ^Intra4×4 (k-1) is embedded into the host vector x(k) of the current macroblock to obtain the vector embedded with feature information corresponding to the current macroblock then use Replace all the elements in the current macroblock in sequence with the 8th to 16th quantized DCT coefficients in the feature information embedding block, and then perform steps ①-6; if the coding mode of the previous macroblock of the current macroblock is Intra16 ×16, then use a generalized differential extension method to embed the feature vector w ^Intra16×16 (k-1) of the previous macroblock of the current macroblock into the host vector x(k) of the current macroblock to obtain the current macroblock The corresponding vector embedded with feature information then use All elements in replace the 8th to 16th quantized DCT coefficients in the feature information embedding block in the current macroblock in order, and then perform steps ①-6; In the step ①-5, the method of two generalized differential extensions is used to embed the feature vector w ^Intra4×4 (k-1) of the previous macroblock of the current macroblock into the host vector x(k) of the current macroblock The specific process is: A1. Carry out positive transformation to x(k) to obtain vector y(k), y(k)=(y ₁ y ₂ ... y ₉ ), wherein, y ₁ , y ₂ and y ₉ correspond to y(k) The 1st element, the 2nd element and the 9th element of the , y ₂ ＝x ₂ -x ₁ , y _i' ＝ _xi' -x ₁ , y ₉ ＝x ₉ -x ₁ , 2≤i'≤9, α _i is weight, symbol is the rounding down symbol; A2. Embed the first to eighth elements of w ^Intra4×4 (k-1) into y(k) to get a vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with Correspondingly represents the first element, the i'-1th element and the 8th element in w ^Intra4×4 (k-1); A3, yes Perform an inverse transformation to obtain a vector embedded with part of the feature information in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9; A4, yes Perform forward transformation to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight; A5. Embed the 9th to 16th elements in w ^Intra4×4 (k-1) into , get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with Correspondingly represent the 9th element, the i'+7th element and the 16th element in w ^Intra4×4 (k-1); A6, yes Perform inverse transformation to obtain the vector embedded with feature information corresponding to the current macroblock in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9; In the step ①-5, the feature vector w ^Intra16×16 (k-1) of the previous macroblock of the current macroblock is embedded into the host vector x(k) of the current macroblock by using a generalized differential extension method The specific process is: B1. Carry out positive transformation on x(k) to obtain vector y(k), y(k)=(y ₁ y ₂ ... y ₉ ), wherein, y ₁ , y ₂ and y ₉ correspond to y(k) The 1st element, the 2nd element and the 9th element of the , y ₂ ＝x ₂ -x ₁ , y _i' ＝ _xi' -x ₁ , y ₉ ＝x ₉ -x ₁ , 2≤i'≤9, α _i is weight, symbol is the rounding down symbol; B2. Embed w ^Intra16×16 (k-1) into y(k) to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, with Correspondingly represents the first element, the i'-1th element and the 8th element in w ^Intra16×16 (k-1); B3, yes Perform inverse transformation to obtain the vector embedded with feature information corresponding to the current macroblock in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9; ①-6. Modify the CBP of the current macroblock, and then perform steps ①-7; The specific process of modifying the CBP of the current macroblock in the described step 1.-6 is: ①-6a. If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the upper left corner of the current macroblock, then count the 8×8 at the upper left corner of the current macroblock after embedding the feature information Whether all the quantized DCT coefficients in the block are all 0, if all are 0, set a ₆ in A to 0, a ₃ , a ₄ , a ₅ remain unchanged, if not all 0, set a 6 in A a ₆ is set to 1, a ₃ , a ₄ , a ₅ remain unchanged; If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block in the upper right corner of the current macroblock, then count all the 8×8 blocks in the upper right corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₅ in A to 0, a ₃ , a ₄ , a ₆ remain unchanged, if they are not all 0, set a ₅ in A to 0 1, a ₃ , a ₄ , a ₆ remain unchanged; If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the lower left corner of the current macroblock, then count all the 8×8 blocks at the lower left corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₄ in A to 0, a ₃ , a ₅ , a ₆ remain unchanged, if they are not all 0, set a ₄ in A to 0 1, a ₃ , a ₅ , a ₆ remain unchanged; If the 8×8 block where the feature information embedded block in the current macroblock is located is the 8×8 block at the lower right corner of the current macroblock, then count all the 8×8 blocks at the lower right corner after the feature information is embedded in the current macroblock Whether the quantized DCT coefficients are all 0, if they are all 0, set a ₃ in A to 0, a ₄ , a ₅ , a ₆ remain unchanged, if they are not all 0, set a ₃ in A to 0 1, a ₄ , a ₅ , a ₆ remain unchanged; ①-6b, converting the modified A into a decimal number to obtain the modified CBP of the current macroblock; ①-7. Entropy encoding is performed on the current macroblock; ①-8. Make k=k+1, use the next pre-encoded macroblock in the I frame as the current macroblock, and return to step ①-2 to continue until all pre-encoded macroblocks in the I frame are processed Complete, obtain the coded stream of the I frame that is embedded with feature information, wherein, "=" in k=k+1 is the assignment symbol; ② At the decoding end, feature information is extracted from each macroblock in the I frame embedded with feature information, and the coding mode and brightness prediction mode of each macroblock are determined, and then error recovery is performed on incorrectly decoded blocks, specifically The process is: ②-1. Perform entropy decoding on each macroblock in the I frame embedded with feature information, then determine whether each macroblock after entropy decoding is a correct decoded block, and then determine the first macroblock after entropy-decoding The vector containing feature information of each correctly decoded block except the first macroblock after entropy decoding extracts the feature information from the vector containing feature information of each correctly decoded block except the first macroblock after entropy decoding, and determines The encoding mode and brightness prediction mode of the previous macroblock of each correctly decoded block outside the first macroblock after entropy decoding, assuming that the k'th macroblock after entropy decoding is a correctly decoded block, then The specific process of determining the vector containing feature information of the macroblock is: calculating the sum of the absolute values of all the quantized DCT coefficients of each 4×4 block in the macroblock that are dequantized as AC DCT coefficients, and the sum The 4×4 block with the largest value is used as the feature information extraction block, and then all quantized DCT coefficients in the feature information extraction block are scanned in zig-zag mode, and then the eighth one in the feature information extraction block is scanned in the order of zig-zag mode The 16th quantized DCT coefficients are arranged to form a one-dimensional vector containing 9 elements, and then the one-dimensional vector is used as the vector containing feature information of the macroblock, which is denoted as in, with Corresponding representation The 1st element, the 2nd element and the 9th element in ; The vector containing feature information from the macroblock Extract feature information from the macroblock, and determine the encoding mode and brightness prediction mode of the previous macroblock of the macroblock; Above, 2≤k'≤K; In the step ②-1, the vector containing feature information from the macroblock The specific process of extracting feature information from the macroblock and determining the encoding mode and brightness prediction mode of the previous macroblock of the macroblock is as follows: ②-1a, yes Perform forward transformation to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight, symbol is the rounding down symbol; ②-1b, from The feature information consisting of 8 feature information bits is extracted from the feature information, which is denoted as in, with Corresponding representation The 1st element, the 2nd element and the 8th element in the , 1≤j≤8, the symbol "||" is the absolute value operation symbol; ②-1c. Construct a new vector z(k'), z(k')=(z ₁ z ₂ ... z ₉ ), where z ₁ , z ₂ and z ₉ correspond to the z(k') 1st element, 2nd element and 9th element, 2≤i'≤9; ②-1d. Perform inverse transformation on z(k') to obtain vector f(k'), f(k')=(f ₁ f ₂ …f ₉ ), where f ₁ , f ₂ and f ₉ correspond to represent The 1st element, the 2nd element and the 9th element in f(k'), f ₂ =z ₂ +f ₁ , f _i' =z _i' +f ₁ , f ₉ =z ₉ +f ₁ , 2≤i'≤9; ②-1e, will be made by The first element in 2nd element 3rd element and the 4th element The composed binary string is converted into a decimal value. If the decimal value corresponds to the number 9, it means that the coding mode of the previous macroblock of this macroblock is Intra16×16, and then replace the macro with all elements in f(k') in order The feature information in the previous macroblock of the block is extracted from the 8th to 16th quantized DCT coefficients in the block, and then the brightness prediction mode of the previous macroblock of the macroblock is determined, and the brightness of the previous macroblock of the macroblock is determined. The numerical identification of the predictive mode is given by The 5th element in 6th element 7th element and the 8th element The decimal value converted from the composed binary string ends feature information extraction and determination of encoding mode and brightness prediction mode; If the decimal value does not correspond to the number 9, it means that the coding mode of the previous macroblock of this macroblock is Intra 4×4, and then perform step ②-1f; ②-1f, perform positive transformation on f(k') to get the vector in, with Corresponding representation The 1st element, the 2nd element and the 9th element in the , 2≤i'≤9, α _i is the weight, symbol is the rounding down symbol; ②-1g, from The feature information consisting of 8 feature information bits is extracted from the feature information, which is denoted as in, with Corresponding representation The 1st element, the 2nd element and the 8th element in the , 1≤j≤8, the symbol "| |" is the absolute value operation symbol; ②-1h. Construct a new vector z'(k'), z'(k')=(z ₁ 'z ₂ '...z ₉ '), where z ₁ ', z ₂ ' and z ₉ ' correspond to Indicates the 1st element, the 2nd element and the 9th element in z'(k'), 2≤i'≤9; ②-1i. Perform inverse transformation on z'(k') to obtain vector x(k'), x(k')=(x ₁ x ₂ ... x ₉ ), where x ₁ , x ₂ and x ₉ correspond to Represents the 1st element, the 2nd element and the 9th element in x(k'), x ₂ =z ₂ '+x ₁ , x _i' =z _i' '+x ₁ , x ₉ =z ₉ '+x ₁ , 2≤i'≤9; ②-1j. Replace the 8th to 16th quantized DCT coefficients in the feature information extraction block in the previous macroblock of the macroblock in sequence with all elements in x(k'); ②-1k, construct a vector w(k') containing 16 elements, the first 8 elements of w(k') are The 8 elements of w(k'), the last 8 elements of w(k') are 8 elements of ②-11. Determine the luminance prediction mode of the 4×4 block numbered 0 in the previous macroblock of the macroblock, and the number of the luminance prediction mode of the 4×4 block is identified by the first in w(k') The decimal value converted from the binary string composed of the first element, the second element, the third element and the fourth element; Determine the luminance prediction mode of the 4×4 block numbered 4 in the previous macroblock of the macroblock, the number of the luminance prediction mode of the 4×4 block is identified by the fifth element in w(k'), the first The decimal value converted from the binary string composed of 6 elements, the 7th element and the 8th element; Determine the luminance prediction mode of the 4×4 block numbered 8 in the previous macroblock of the macroblock, the number of the luminance prediction mode of the 4×4 block is identified by the 9th element in w(k'), the 9th element The decimal value converted from the binary string composed of 10 elements, the 11th element and the 12th element; Determine the luminance prediction mode of the 4×4 block numbered 12 in the previous macroblock of the macroblock, the number of the luminance prediction mode of the 4×4 block is identified by the 13th element in w(k'), the th The decimal value converted from the binary string composed of 14 elements, the 15th element and the 16th element; End feature information extraction and encoding mode and brightness prediction mode determination; ②-2. Define the kth macroblock in the decoded I frame as the current macroblock, wherein, 1≤k≤K, and the initial value of k is 1; ②-3, judge whether the current macroblock is the last macroblock in the decoded I frame, if the current macroblock is not the last macroblock, then perform step ②-4, if the current macroblock is the last macroblock, then Execute steps ②-5; ②-4. Judging whether the current macroblock is a correctly decoded block or an incorrectly decoded block, if the current macroblock is a correctly decoded block, then the current macroblock is not processed, and then step ②-6 is performed; If the current macroblock is an incorrectly decoded block, then determine whether the next macroblock of the current macroblock is a correctly decoded block or an incorrectly decoded block, and if the next macroblock of the current macroblock is a correctly decoded block, use the current macroblock The encoding mode and brightness prediction mode restore the current macroblock, and then perform step ②-6; if the next macroblock of the current macroblock is an incorrectly decoded block, use bilinear interpolation to restore the current macroblock; The specific process of using the encoding mode and brightness prediction mode of the current macroblock in the described step ②-4 to restore the current macroblock is: ②-4a. If the encoding mode of the current macroblock is Intra16×16, use the brightness prediction mode of the current macroblock to predict the pixel value of each pixel in the current macroblock, and then use The predicted value of the pixel is used as the final restored pixel value of the corresponding pixel to complete the restoration of the current macroblock; ②-4b. If the encoding mode of the current macroblock is Intra4×4, use the brightness prediction mode of the 4×4 block numbered 0 in the current macroblock to each of the 8×8 blocks where the 4×4 block is located The pixel value of each pixel in the 4×4 block is predicted, and then the predicted value of each pixel in each 4×4 block is used as the final restored pixel corresponding to each pixel in the 4×4 block value; Use the brightness prediction mode of the 4×4 block numbered 4 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then Taking the predicted value of each pixel in each 4×4 block as the final restored pixel value of each pixel in the corresponding 4×4 block; Use the brightness prediction mode of the 4×4 block numbered 8 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then Taking the predicted value of each pixel in each 4×4 block as the final restored pixel value of each pixel in the corresponding 4×4 block; Use the brightness prediction mode of the 4×4 block numbered 12 in the current macroblock to predict the pixel value of each pixel in each 4×4 block in the 8×8 block where the 4×4 block is located, and then Taking the predicted value of each pixel in each 4×4 block as the final restored pixel value of each pixel in the corresponding 4×4 block; ②-5. Determine whether the current macroblock is a correctly decoded block or an incorrectly decoded block. If the current macroblock is a correctly decoded block, the current macroblock is not processed. So far, all incorrectly decoded blocks in the decoded I frame are completed recovery; if the current macroblock is an incorrectly decoded block, the bilinear interpolation method is used to restore the current macroblock, and the recovery of all incorrectly decoded blocks in the decoded I frame is completed so far; ②-6, make k=k+1, use the next macroblock to be processed in the decoded I frame as the current macroblock, then return to step ②-3 to continue execution, wherein, " in k=k+1 =" is an assignment symbol.