CN101325711A - Method for controlling self-adaption code rate based on space-time shielding effect - Google Patents

Abstract

The invention relates to a control method of self-adapted code rate based on time-space masking effect. The method unifies a macroscopic description that reflects features in some part of a visual sensing system, especially merging the perceiving property of eye about motion information with other sensing features, which includes: preprocessing the original sequence of video code; then calculating frame level complexity and degrees of perception of each macro-block using time-space masking effect; the self-adapted code rate control algorithm distributes frame bit number according to frame complexity; finally determining the quantization parameter of macro-block according to the degree of perception of the macro-block. The method obtains a better object visual quality under the condition of identical or less bit number in comparison with the code rate control algorithm on the JM10.2 edition of H.264 check model.

Description

Method for controlling self-adaption code rate based on space-time shielding effect

Technical field

The present invention relates to a kind of method for controlling self-adaption code rate based on space-time shielding effect, in the Mathematical Modeling with macroscopic description unification to an integral body of reflection visually-perceptible system Partial Feature, particularly the apperceive characteristic of human eye for movable information merged mutually with other apperceive characteristics.Compare with the modified model VM8 rate control algorithm H.264, this method can obtain better subjective visual quality do with bit number still less.

Background technology

Multimedia technology based on computer technology, video, audio frequency and the communication technology makes people's life more rich and varied by the Internet network.The compress technique that adopts in the nowadays various video encoding standards, main information theory based on Shannon, the video quality evaluation criterion of mathematical computations is convenient in employing, removes the statistical redundancy of video information on room and time, thereby obtains the video code flow that is subjected to information entropy constrained.But along with being gradually improved and maturation of various compression coding technologies, compression efficiency has reached higher level, and compression ratio has been difficult to further raising near the limit under the Shannon theory framework.Satisfy more and more higher actual demand in order further to improve compression efficiency, it is extremely urgent to seek the new method and the technology that improve compression efficiency.In recent years, coding method in conjunction with the visually-perceptible model receives great concern, these methods mainly are to excavate and utilize the human visual system (human video system, characteristic HVS) improves compression efficiency, produces better lower compressed bit stream of subjective quality.

Existing method for video coding adopts rate-distortion optimization to remove the statistical redundancy information of video mostly, and the criterion of estimating the video coding distortion in this method is generally mean square error (MSE) and Y-PSNR (PSNR).This interpretational criteria and the human visual system that gets along well (HVS) are in full accord for the evaluation of video, therefore there is the defective that can not remove physiology/psychographic information redundancy in such method for video coding, and human eye is final information receiver, and the frame that signal to noise ratio is high might not have best subjective visual quality do.

Existing bit rate control method based on area-of-interest (ROI) is based on the control method of multi-object, or on the rate distortion equation, make an issue of, they do not consider this final recipient of human eye, can not determine which zone of frame and can use still less bit number, which zone will reach better subjective visual quality do with the more bits number.Or reach the same subjective visual quality do with still less bit number.

Shielding effect is a kind of important HVS characteristic, is meant under the situation that a plurality of visual stimuli exist, and the existence of a visual stimuli makes the visible threshold of another excitation promote the phenomenon of (or reduction of observability).The existence that is to say shielding effect will cause the change of the sensitivity threshold of vision, both can be to suppress, and also can be to strengthen.Occur in time or spatial domain according to shielding effect, can be divided into spatial domain and cover with time-domain and cover.

The time shielding effect is variation discontinuous lifting or the decline that causes the vision threshold value in time owing to brightness.Just explanation is in frame of video for this, and when the target of scene sudden change or rapid movement, the vision threshold value will promote to some extent, and this moment, the perception of human eyes degree will reduce greatly.

And the main forms of space shielding effect is covered for contrast.Discover that in contrast is covered when signal had identical frequency distribution and directional characteristic with the signal that causes shielding effect, the contrast shielding effect was the strongest.According to the space shielding effect, human eye is easier to discover to the distortion ratio of low frequency region, and the distortion of high-frequency region is difficult for discovering.

Given this, the inventive method is from the human visual system, utilize space-time shielding effect to set up a sensor model, by the complexity of this Model Calculation frame of video and the perceptibility of each macro block, utilize complexity and perceptibility to regulate the bit number of frame and macro block, make frame of video change the better subjective visual quality do of acquisition under the little situation at code check and PSNR.

Summary of the invention

The objective of the invention is defective at the prior art existence, a kind of method for controlling self-adaption code rate of imitating based on space-time shielding is provided, utilize space-time shielding effect to calculate the perceptibility of frame level complexity and macro-block level, utilize complexity and perceptibility in GOP, to regulate frame level bit number and macro-block level bit number then.The frame that complexity is high, self-adaption code rate control algolithm will give its more bits number.And for human eye interesting areas in the image, rate control algorithm will give this zone more bits number, promote this regional visual quality; Otherwise, then give less bit number for non-area-of-interest, but people's subjective visual quality do does not descend at this moment.This algorithm can make video coding change the better subjective visual quality do of acquisition under the little situation at code check and PSNR.

For reaching above-mentioned purpose, design of the present invention is:

As shown in Figure 1, at first the original series to video coding carries out preliminary treatment, utilize space-time shielding effect to calculate the perceptibility of frame level complexity and each macro block then, the self-adaption code rate control algolithm is distributed the frame bit number according to the frame complexity again, and the quantization parameter of macro block is then finally determined by the perceptibility of this macro block.

According to the foregoing invention design, the present invention adopts following technical proposals:

A kind of method for controlling self-adaption code rate based on space-time shielding, it is characterized in that particularly the apperceive characteristic of human eye for movable information being merged mutually with other apperceive characteristics in the Mathematical Modeling with macroscopic description unification to an integral body of reflection sensory perceptual system Partial Feature; Its concrete operations step is as follows:

(1) input video frame;

(2) to the frame of video preliminary treatment: uncoded original series is carried out Filtering Processing;

(3) set up sensor model: utilize space-time shielding effect to calculate the frame complexity of current encoded frame and the perceptibility of each macro block,, do not calculate its macro block perceptibility if the I frame then only calculates the frame complexity;

(4) utilize the self-adaption code rate control algolithm to adjust the quantization parameter of frame bit number and macro block;

(5) utilize the quantization parameter revised to the frame of video recompile.

As follows in the above-mentioned steps (2) to the pretreated method of frame of video:

Present frame utilized Gauss's template of 5 * 5 carry out gaussian filtering, template is as follows:

$\left\{\begin{array}{ccccc}2& 4& 5& 4& 2\\ 4& 9& 12& 9& 4\\ 5& 12& 15& 12& 5\\ 4& 9& 12& 9& 4\\ 2& 4& 5& 4& 2\end{array}\right\}$

The step of setting up sensor model in the above-mentioned steps (3) is:

1. ask the border to obtain boundary image f ' (n) to the image of filtering with the Sobel algorithm;

2. ask the accumulation frame poor to the image of filtering, computing formula is as follows, wherein f " (n) the accumulation frame of expression n two field picture is poor, f (n) expression n frame original series image:

$f^{\′\′}\left(n\right)=\underset{i=0}{\overset{4}{\mathrm{\Σ}}}(f(n-i)-f(n-i-1))---\left(8\right)$

3. to boundary image f ' (n) and accumulative total frame difference image f " (n) carry out and computing, just obtained the prospect boundary image, fill then, can obtain foreground image;

4. the smoothness that auto-correlation computation obtains this macro block is carried out in the brightness of the macro block in the original image, wherein Smooth represents the smoothness of current macro, n gray values of pixel points of MBgray (n) expression current macro:

$\mathrm{Smooth}=\underset{n=0}{\overset{256}{\mathrm{\Σ}}}[\mathrm{MBgray}\left(n\right)*\mathrm{MBgray}(n+30)]$

5. the macro block in the original image is carried out the activity Active that the gradient computing obtains this macro block, I in the following formula _{I, j}The brightness value of the capable j row of expression current macro i:

$\mathrm{Active}=\frac{1}{16*15}\{\underset{i=0}{\overset{14}{\mathrm{\Σ}}}\underset{j=0}{\overset{15}{\mathrm{\Σ}}}\sqrt{|I_{i,j}-I_{i+1,j}|}+\underset{i=0}{\overset{15}{\mathrm{\Σ}}}\underset{j=0}{\overset{14}{\mathrm{\Σ}}}\sqrt{|I_{i,j}-I_{i,j+1}|}\}$

6. the macroblock motion vector in the original image is carried out the time masking property that computing obtains this macro block, MVx, MVy represent the motion vector of the level and the vertical direction of current macro in the following formula, and MVz represents that the time of this macro block covers characteristic:

$\mathrm{MVz}=\sqrt{\mathrm{MV}{x}^2+\mathrm{MV}{y}^2}$

7. calculated the perceptibility Focus of this macro block by above Several Parameters, wherein a, b, c are weighting parameters; Forward represents that whether current macro is the macro block in the prospect, if current macro is a foreground blocks, then Forward equals 1, otherwise equals 0:

Focus＝a*MVz+b*Smooth+c*Active+Forward

8. calculate the complexity Complexity of this frame by the perceptibility of all macro blocks in the frame:

$\mathrm{Complexity}=\frac{\underset{n=0}{\overset{395}{\mathrm{\Σ}}}\mathrm{Focu}{s}_n}{396}$

The step of the self-adaption code rate control algolithm in the above-mentioned steps (4) is as follows:

1. according to the bandwidth of Set For Current and the length of frame per second and image sets GOP, current GOP is distributed a target bit $T_r(n_i,0)=\frac{\mathrm{Bit}\_\mathrm{rate}}{\mathrm{Frame}\_\mathrm{rate}}*N_{\mathrm{gop}},$ T wherein _r(n _i, 0) and be the GOP target bit, Bit_rate is a bandwidth, Frame_rate is a frame per second, N _GopBe GOP length;

If 2. present frame is the I frame,, adjust the quantization parameter QP of I frame then according to the complexity of this frame _iα, β, γ are for regulating parameter in the following formula; Complexity is the complexity of present frame, and a, b, c are threshold value:

$Q{P}_i=\left\{\begin{array}{cc}Q{P}_i+\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Complexity}>a)\\ Q{P}_i+\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Complexity}>b)\\ Q{P}_i+\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Complexity}<b)\end{array}\right.$

If 3. present frame is the P frame, then distribute target bit to be:

$f\left(n_{i,j}\right)=\mathrm{\&beta;}*\{\frac{\mathrm{bit}\_\mathrm{rate}}{\mathrm{frame}\_\mathrm{rate}}+\mathrm{\&alpha;}[\mathrm{Tbl}\left(n_{i,j}\right)-B_c\left(n_{i,j}\right)\left]\right\}$

$+(1-\mathrm{\&beta;})*\left\{\frac{W_p\left(n_{i,j-1}\right)*T_r\left(n_{i,j}\right)}{W_p\left(n_{i,j-1}\right)*N_{p,r}(j-1)}\right\}$

F (n in the formula _{I, j}) the preallocated bit number of the current P frame of expression, bit_rate represents bandwidth, frame_rate represents frame per second, Tbl (n _{I, j}) expression i GOP j the preallocated bit number of P frame, B _c(n _{I, j}) j P frame of i GOP of expression preceding actual bit number that occupies of buffering area of encoding, W _p(n _{I, j-1}) complexity of j-1 P frame of i GOP of expression, T _r(n _{I, j}) j P frame of i GOP of expression preceding remaining bit number of GOP of encoding, α and β are constant;

4. according to the complexity of this P frame, adjust its target bit f (n _{I, j}) be:

$f\left(n_{i,j}\right)=\left\{\begin{array}{cc}f\left(n_{i,j}\right)*\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Complexity}>a)\\ f\left(n_{i,j}\right)*\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Complexity}>b)\\ f\left(n_{i,j}\right)*\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Complexity}<b)\end{array}\right.$

Wherein α, β, γ are for regulating parameter; Complexity is the complexity of present frame, and a, b, c are threshold value;

5. according to target bit, calculate the quantization parameter QP of this frame by R-Q secondary rate quantitative model;

6. revise its quantization parameter QP according to the perceptibility of macro block:

$Q{P}_n=\left\{\begin{array}{cc}Q{P}_n+\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Focus}>a)\\ Q{P}_n+\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Focus}>b)\\ Q{P}_n+\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Focus}<b)\end{array}\right.$

QP in the following formula _nThe quantization parameter of representing n macro block, Focus are represented the macro block perceptibility, and α, β, γ are for regulating parameter, and a, b are threshold value.

The step of the adjustment frame bit number in the above-mentioned steps (4) is as follows:

1. the bit number T before the coding of the j frame among i GOP _r(n _i, j)=T _r(n _i, j-1)-A (n _i, j-1), A (n wherein _i, j-1) the actual used bit number of j-1 frame coding of i GOP of expression;

2. the remaining bits number of buffering area is updated to R _r(n _i, j), it represents the remaining bits number of the j frame coding back buffering area of i GOP, A (n _i, j-1) the actual used bit number of j-1 frame coding of i GOP of expression, Bit_rate is a bandwidth, Frame_rate is a frame per second.

$R_r(n_i,j)=R_r(n_i,j-1)+A(n_i,j-1)-\frac{\mathrm{Bit}\_\mathrm{rate}}{\mathrm{Frame}\_\mathrm{rate}}$

The present invention compares with the rate control algorithm on H.264 the verification model JM10.2 version, have following outstanding feature and advantage: the present invention can reasonably adjust the bit number of distributing to GOP in its interframe, the bit number of distributing to coded frame reasonably can be adjusted between human eye area-of-interest and non-area-of-interest, under identical even still less bit number situation, be obtained better subjective visual quality do.

Description of drawings

Fig. 1 is the method for controlling self-adaption code rate FB(flow block) based on space-time shielding effect of the present invention.

Fig. 2 is the structured flowchart of sensor model among Fig. 1.

Fig. 3 is the structured flowchart of self-adaption code rate control algolithm among Fig. 1.

Fig. 4 is the employed bit number of I frame of rate control algorithm coding in the JM10.2 verification model and the comparison diagram of the employed bit number of method for controlling self-adaption code rate of the present invention coding I frame.

Fig. 5 is the contrast of the I two field picture and the method for controlling self-adaption code rate of the present invention coding I frame coded image of rate control algorithm coding in the JM10.2 verification model.

Fig. 6 is the employed bit number of P frame of rate control algorithm coding in the JM10.2 verification model and the comparison diagram of the employed bit number of method for controlling self-adaption code rate of the present invention coding P frame.

Fig. 7 is the contrast of the P two field picture and the method for controlling self-adaption code rate of the present invention coding P frame coded image of rate control algorithm coding in the JM10.2 verification model.

Embodiment

Details are as follows in conjunction with the accompanying drawings for one embodiment of the present of invention:

This method for controlling self-adaption code rate based on space-time shielding effect is by flow chart shown in Figure 1, be that programming realizes that Fig. 5 and Fig. 7 are the contrast of JM10.2 verification model and coded image of the present invention on the PC test platform of Athlon x2 2.0GHz, internal memory 1024M at CPU.

Referring to Fig. 1, this is based on the method for controlling self-adaption code rate of space-time shielding effect, (the I frame only obtains the frame complexity to have obtained the frame complexity of current encoded frame and the perceptibility of each macro block by sensor model, do not calculate its macro block perceptibility), utilize the frame complexity that the bit number of its distribution is adjusted then, utilize the perceptibility of its macro block that the quantization parameter of macro block is adjusted, this invention can utilize identical even bit number still less obtains better subjective visual quality do.

As shown in Figure 2, this at first sets up a sensor model based on the method for controlling self-adaption code rate of space-time shielding effect, and this model is based on the space-time shielding effect basis.Sensor model will reflect in the Mathematical Modeling of macroscopic description unification to an integral body of HVS of visually-perceptible system Partial Feature.Utilize space-time shielding effect, set up a sensor model based on human visual system (HVS).The main effect of sensor model is exactly an importance of analyzing each macro block (MB) in the frame of video, according to the size of its importance to perceptibility of its mark, the complexity of adding up this frame then.The analysis of perceptibility has then utilized space-time shielding effect, and the difference of motion size, activity, texture complexity and front and back scape by calculating MB is taken all factors into consideration.

Perceptibility (Focus) is a key parameter of distinguishing macro block importance in the frame of video.This CALCULATION OF PARAMETERS is that texture complexity, motion size, front and back scape and the activity by macro block comes weighting to differentiate.The calculating of perceptibility is finished by several aspects once:

(1) according to the space shielding effect, the complicated more zone of texture can hide many more noises, and the ability that smooth region is hidden noise comparatively speaking a little less than.Because the histogram information of smooth region is more concentrated, histogram after the translation and former histogram almost are quadratures, and therefore, this algorithm utilizes the brightness histogram information of macro block to calculate the texture complexity of this macro block.

$\mathrm{Smooth}=\underset{n=0}{\overset{256}{\mathrm{\&Sigma;}}}[\mathrm{MBgray}\left(n\right)*\mathrm{MBgray}(n+30)]$

(1)

MBgray in the formula (n) represents the number that this macro block brightness value is the pixel of n, and Smooth represents the texture complexity.

(2) according to human visual system (HVS), human eye is far longer than the attention rate of background image to the attention rate of foreground image in the video.Cutting apart of sport foreground, in order to reduce algorithm complex as much as possible, save the scramble time, what this algorithm adopted is a kind of simple effective method.At first to image sequence do inter-frame difference with every frame difference, then two class difference result are accumulated, the accumulation results cluster of taking to occur simultaneously is obtained the sport foreground profile.But the profile that comes out so still is not very perfect, therefore the boundary graph that this profile is drawn with the Sobel operator again with, after the boundary profile binary conversion treatment that draws at last, scanning filling can obtain the sport foreground in the image sequence.

(3) according to the time shielding effect, when occurrence scene switches or object when bigger motion is arranged, human eye needs the process of an adaptation, this moment, the rate respectively of human eye can descend, so less bit number can be distributed in these zones, and human eye has bigger attention rate among a small circle movement of objects, and more bit number will be distributed in these zones.So the differentiation to the size of moving is a relatively more crucial part.We find in the experiment, in the general video sequence motion amplitude of motion vector greater than the motion of 32 fritter usually enough greatly, when the MB motion amplitude greater than 32 the time, the resolution to this MB of human eye can descend significantly.So we are defined as 32 with the discrimination threshold of motion vector size.The calculating of MB motion vector as shown in Equation 2, MVx, MVy represent the motion vector of the level and the vertical direction of current macro in the formula, MVz represents that the time of this macro block covers characteristic.

$\mathrm{MVz}=\sqrt{\mathrm{MV}{x}^2+\mathrm{MV}{y}^2}---\left(2\right)$

(4) for video image, the activity of MB is big more, just needs many more bit numbers, and on the contrary, if the bit number of the big macroblock allocation of activity has lacked, the well as subjective video quality of image will seriously descend so.Gradient method is adopted in the calculating of activity in this algorithm, as shown in Equation (3), and I _{I, j}The brightness value of the capable j row of expression current macro i.

$\mathrm{Active}=\frac{1}{16*15}\{\underset{i=0}{\overset{14}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{15}{\mathrm{\&Sigma;}}}\sqrt{|I_{i,j}-I_{i+1,j}|}+\underset{i=0}{\overset{15}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{14}{\mathrm{\&Sigma;}}}\sqrt{|I_{i,j}-I_{i,j+1}|}\}$ (3)

By above four steps, the calculating of perceptibility by motion vector MVz, texture complexity Smooth, prospect Forward, these 4 parameter weightings of activity size Active calculate, as formula 4.Wherein a, b, c are weighting parameters; Forward represents that whether current macro is the macro block in the prospect, if current macro is a foreground blocks, then Forward equals 1, otherwise equals 0.

Focus＝a*MVz+b*Smooth+c*Active+Forward (4)

The calculating of frame complexity Complexity is then got by the Focus statistics of all macro blocks, as formula (5).

$\mathrm{Complexity}=\frac{\underset{n=0}{\overset{395}{\mathrm{\&Sigma;}}}\mathrm{Focu}{s}_n}{396}$ (5)

The self-adaption code rate control algolithm is to carry out the distribution of bit number and the adjustment of quantization parameter on the basis of sensor model, and as shown in Figure 3, its committed step is as follows:

The first, the distribution of P frame bit number

The Rate Control model of this algorithm is based on the VM8 model modification, and the distribution of P frame bit number as shown in Equation (6).F (n in the formula _{I, j}) the preallocated bit number of the current P frame of expression, bit_rate represents bandwidth, frame_rate represents frame per second, Tbl (n _{I, j}) expression i GOP j the preallocated bit number of P frame, B _c(n _{I, j}) j P frame of i GOP of expression preceding actual bit number that occupies of buffering area of encoding, W _p(n _{I, j-1}) complexity of j-1 P frame of i GOP of expression, T _r(n _{I, j}) j P frame of i GOP of expression preceding remaining bit number of GOP of encoding, α and β are constant.Sub-distribution then is to carry out according to the complexity of present frame again.

$f\left(n_{i,j}\right)=\mathrm{\&beta;}*\{\frac{\mathrm{bit}\_\mathrm{rate}}{\mathrm{frame}\_\mathrm{rate}}+\mathrm{\&alpha;}[\mathrm{Tbl}\left(n_{i,j}\right)-B_c\left(n_{i,j}\right)\left]\right\}$

(6)

$+(1-\mathrm{\&beta;})\frac{W_p\left(n_{i,j-1}\right)*T_r\left(n_{i,j}\right)}{W_p\left(n_{i,j-1}\right)*N_{p,r}(j-1)}$

P frame bit number depends primarily on that target bit and several two parameters of remaining bits take all factors into consideration as can be seen from formula 6.Complexity W in the formula _p(n _{I, j-1}) be that complexity weighting by quantization parameter, preassignment bit number and former frame gets, with frame complexity in herein the self-adaption code rate algorithm be two notions, the frame complexity in this paper algorithm is calculated by sensor model.Obtaining f (n _{I, j}) after, assigned the first time of P frame bit number.

The second, set up sensor model

Utilize sensor model to calculate the complexity of this frame and the perceptibility of interior each macro block of frame, shown in formula 4 and 5.

The 3rd, the secondary distribution of P frame bit number

Carry out the second time according to frame complexity Complexity and distribute, as formula (7).

$f\left(n_{i,j}\right)=\left\{\begin{array}{cc}f\left(n_{i,j}\right)*\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Complexity}>a)\\ f\left(n_{i,j}\right)*\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Complexity}>b)\\ f\left(n_{i,j}\right)*\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Complexity}<b)\end{array}\right.$ (7)

Wherein α, β, γ are for regulating parameter; Complexity is the complexity of present frame, and a, b, c are threshold value.

The 4th, the correction of macroblock quantization parameter

The distribution of similar and frame bit number, the distribution of macro block bit number also is to count these two parameters according to the macro block actual bit of the complexity of macro block and former frame same position to determine.In conjunction with sensor model, utilize each macro block perceptibility we adjust this and revise quantization parameter.Final quantization parameter is decided by the quantization parameter and the correction quantization parameter of this macro block.

According to above-mentioned theory, the concrete operations step of present embodiment is as follows:

(1) uncoded original series is carried out Filtering Processing, preprocess method is to utilize Gauss's template of 5 * 5 to carry out gaussian filtering to present frame, and template is as follows:

$\left\{\begin{array}{ccccc}2& 4& 5& 4& 2\\ 4& 9& 12& 9& 4\\ 5& 12& 15& 12& 5\\ 4& 9& 12& 9& 4\\ 2& 4& 5& 4& 2\end{array}\right\}$

(2) set up the perceptibility that sensor model calculates frame level complexity and macro block, its process is:

1. ask the border to obtain boundary image f ' (n) to the image of filtering with the Sobel algorithm;

2. ask the accumulation frame poor to the image of filtering, computing formula is as follows, wherein f " (n) the accumulation frame of expression n two field picture is poor, f (n) expression n frame original series image:

$f^{\&prime;\&prime;}\left(n\right)=\underset{i=0}{\overset{4}{\mathrm{\&Sigma;}}}(f(n-i)-f(n-i-1))$ (8)

3. to boundary image f ' (n) and accumulative total frame difference image f " (n) carry out and computing, just obtained the prospect boundary image, fill then, can obtain foreground image;

4. the smoothness that auto-correlation computation obtains this macro block is carried out in the brightness of the macro block in the original image, wherein Smooth represents the smoothness of current macro, n gray values of pixel points of MBgray (n) expression current macro:

$\mathrm{Smooth}=\underset{n=0}{\overset{256}{\mathrm{\&Sigma;}}}[\mathrm{MBgray}\left(n\right)*\mathrm{MBgray}(n+30)]$

5. the macro block in the original image is carried out the activity Active that the gradient computing obtains this macro block, I in the following formula _{I, j}The brightness value of the capable j row of expression current macro i:

$\mathrm{Active}=\frac{1}{16*15}\{\underset{i=0}{\overset{14}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{15}{\mathrm{\&Sigma;}}}\sqrt{|I_{i,j}-I_{i+1,j}|}+\underset{i=0}{\overset{15}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{14}{\mathrm{\&Sigma;}}}\sqrt{|I_{i,j}-I_{i,j+1}|}\}$

6. the macroblock motion vector in the original image is carried out the time masking property that computing obtains this macro block, MVx, MVy represent the motion vector of the level and the vertical direction of current macro in the following formula, and MVz represents that the time of this macro block covers characteristic:

$\mathrm{MVz}=\sqrt{\mathrm{MV}{x}^2+\mathrm{MV}{y}^2}$

7. calculated the perceptibility Focus of this macro block by above Several Parameters, wherein a, b, c are weighting parameters; Forward represents that whether current macro is the macro block in the prospect, if current macro is a foreground blocks, then Forward equals 1, otherwise equals 0:

Focus＝a*MVz+b*Smooth+c*Active+Forward

8. calculate the complexity Complexity of this frame by the perceptibility of all macro blocks in the frame:

$\mathrm{Complexity}=\frac{\underset{n=0}{\overset{395}{\mathrm{\&Sigma;}}}\mathrm{Focu}{s}_n}{396}$

(3) the frame level complexity that two steps calculated more than the utilization and the perceptibility of macro block, the process of self-adaption code rate control algolithm is as follows:

1. according to the bandwidth of Set For Current and the length of frame per second and image sets GOP, current GOP is distributed a target bit $T_r(n_i,0)=\frac{\mathrm{Bit}\_\mathrm{rate}}{\mathrm{Frame}\_\mathrm{rate}}*N_{\mathrm{gop}},$ T wherein _r(n _i, 0) and be the GOP target bit, Bit_rate is a bandwidth, Frame_rate is a frame per second, N _GopBe GOP length;

If 2. present frame is the I frame,, adjust the quantization parameter QP of I frame then according to the complexity of this frame _iα, β, γ are for regulating parameter in the following formula; Complexity is the complexity of present frame, and a, b, c are threshold value:

$Q{P}_i=\left\{\begin{array}{cc}Q{P}_i+\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Complexity}>a)\\ Q{P}_i+\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Complexity}>b)\\ Q{P}_i+\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Complexity}<b)\end{array}\right.$

If 3. present frame is the P frame, then distribute target bit to be:

$f\left(n_{i,j}\right)=\mathrm{\&beta;}*\{\frac{\mathrm{bit}\_\mathrm{rate}}{\mathrm{frame}\_\mathrm{rate}}+\mathrm{\&alpha;}[\mathrm{Tbl}\left(n_{i,j}\right)-B_c\left(n_{i,j}\right)\left]\right\}$

$+(1-\mathrm{\&beta;})*\left\{\frac{W_p\left(n_{i,j-1}\right)*T_r\left(n_{i,j}\right)}{W_p\left(n_{i,j-1}\right)*N_{p,r}(j-1)}\right\}$

F (n in the formula _{I, j}) the preallocated bit number of the current P frame of expression, bit_rate represents bandwidth, frame_rate represents frame per second, Tbl (n _{I, j}) expression i GOP j the preallocated bit number of P frame, B _c(n _{I, j}) j P frame of i GOP of expression preceding actual bit number that occupies of buffering area of encoding, W _p(n _{I, j-1}) complexity of j-1 P frame of i GOP of expression, T _r(n _{I, j}) j P frame of i GOP of expression preceding remaining bit number of GOP of encoding, α and β are constant.

4. according to the complexity of this P frame, adjust its target bit f (n _{I, j}) be:

$f\left(n_{i,j}\right)=\left\{\begin{array}{cc}f\left(n_{i,j}\right)*\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Complexity}>a)\\ f\left(n_{i,j}\right)*\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Complexity}>b)\\ f\left(n_{i,j}\right)*\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Complexity}<b)\end{array}\right.$

Wherein α, β, γ are for regulating parameter; Complexity is the complexity of present frame, and a, b, c are threshold value:

5. according to target bit, calculate the quantization parameter QP of this frame by R-Q secondary rate quantitative model;

6. revise its quantization parameter QP according to the perceptibility of macro block:

$Q{P}_n=\left\{\begin{array}{cc}Q{P}_n+\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Focus}>a)\\ Q{P}_n+\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Focus}>b)\\ Q{P}_n+\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Focus}<b)\end{array}\right.$

QP in the following formula _nThe quantization parameter of representing n macro block, Focus are represented the macro block perceptibility, and α, β, γ are for regulating parameter, and a, b are threshold value.

(4) step of the parameter update of rate control algorithm is as follows:

1. the bit number T before the coding of the j frame among i GOP _r(n _i, j)=T _r(n _i, j-1)-A (n _i, j-1), A (n wherein _i, j-1) the actual used bit number of j-1 frame coding of i GOP of expression;

2. the remaining bits number of buffering area is updated to:

$R_r(n_i,j)=R_r(n_i,j-1)+A(n_i,j-1)-\frac{\mathrm{Bit}\_\mathrm{rate}}{\mathrm{Frame}\_\mathrm{rate}}$

R wherein _r(n _i, j) the remaining bits number of the j frame coding back buffering area of i GOP of expression, A (n _i, j-1) the actual used bit number of j-1 frame coding of i GOP of expression, Bit_rate is a bandwidth, Frame_rate is a frame per second.

Example when below providing the input video form and be 352 * 288 CIF adopts the H.264 encoder of JM10.2 version that standard test sequences is encoded.H.264 the configuration of encoder is as follows: BaselineProfile, and IPPP, per 15 frames insert 1 I frame, 1 reference frame, bandwidth is set to 256kbps, and frame per second is set to 30fps, and the initial quantization parameter is set to 32.

Adopt typical standard test sequences deadline to test as input video, as seen from Figure 4, the used average number of bits of I frame of self-adaption code rate control algolithm is obviously lacked than the average number of bits of I frame in the rate control algorithm of verification model.Subjective visual quality do as shown in Figure 5, upper left for the 0th two field picture of verification Model Reconstruction, it is first I frame, used bit number is 71080, upper right is the image of corresponding adaptive model, and used bit number is 58712, and both differ 12368 bits, but subjective visual quality do does not descend, and human eye can not distinguished the two difference.

The used bit number of P frame more as shown in Figure 6, the last a few two field picture bit numbers of each GOP descend relatively more severely in the verification model, in addition have only a few two field picture bit numbers in front 1/4th less than, make picture quality unbalanced.On the contrary, the bit number of several two field pictures in back promotes to some extent in the self-adaption code rate control algolithm, in order to prevent above-mentioned situation, makes the visual quality for images unanimity exactly.

And the used average number of bits of each GOP is as shown in table 1, and the frame of video complexity in GOP is higher, and when strenuous exercise was arranged, the bit number of distributing to GOP this moment was more, as the 7th GOP.When the GOP complexity is lower, in the time of reaching higher subjective visual quality do with less bit number, the bit number of then distributing to GOP is less.Each GOP of self-adaption code rate control algolithm based on sensor model lacks than the employed bit number of verification model on average, but has reached better subjective visual quality do, as shown in Figure 7.

The comparison of table 1GOP average number of bits

GOP

1 2 3 4 5 6 7 8 Average

Type

Verification model 8,781 8,576 8,476 8,555 8,638 8,533 8,867 8,276 8575

Sensor model 7,624 7,386 7,291 5,875 7,189 9,242 1,138 9,877 58134

Claims (5)

1, a kind of method for controlling self-adaption code rate based on space-time shielding effect, it is characterized in that particularly the apperceive characteristic of human eye for movable information being merged mutually with other apperceive characteristics in the Mathematical Modeling with macroscopic description unification to an integral body of reflection visually-perceptible system Partial Feature; Its concrete operations step is as follows:

A. input video frame;

B. to the frame of video preliminary treatment: uncoded original series is carried out Filtering Processing;

C. set up sensor model: utilize space-time shielding effect to calculate the frame complexity of current encoded frame and the perceptibility of each macro block,, do not calculate its macro block perceptibility if the I frame then only calculates the frame complexity;

D. utilize the self-adaption code rate control algolithm to adjust the quantization parameter of frame bit number and macro block;

E. utilize the quantization parameter of having revised to the frame of video recompile.

2, the self-adaption code rate control algolithm based on space-time shielding effect according to claim 1, it is characterized in that in the described step (2) being to utilize Gauss's template of 5 * 5 to carry out gaussian filtering to present frame to the pretreated method of frame of video, template is as follows:

$\left\{\begin{array}{ccccc}2& 4& 5& 4& 2\\ 4& 9& 12& 9& 4\\ 5& 12& 15& 12& 5\\ 4& 9& 12& 9& 4\\ 2& 4& 5& 4& 2\end{array}\right\}.$

3, the self-adaption code rate control algolithm based on space-time shielding effect according to claim 1 is characterized in that the step of the perceptibility of the calculating frame complexity of setting up sensor model in the described step (3) and macro block is:

1. ask the border to obtain boundary image f ' (n) to the image of filtering with the Sobel algorithm;

2. ask the accumulation frame poor to the image of filtering, computing formula is as follows, wherein f " (n) the accumulation frame of expression n two field picture is poor, f (n) expression n frame original series image:

$f^{\&prime;\&prime;}\left(n\right)=\underset{i=0}{\overset{4}{\mathrm{\&Sigma;}}}(f(n-i)-f(n-i-1))---\left(8\right)$

3. to boundary image f ' (n) and accumulative total frame difference image f " (n) carry out and computing, just obtained the prospect boundary image, fill then, can obtain foreground image;

4. the smoothness that auto-correlation computation obtains this macro block is carried out in the brightness of the macro block in the original image, wherein Smooth represents the smoothness of current macro, n gray values of pixel points of MBgray (n) expression current macro:

$\mathrm{Smooth}=\underset{n=0}{\overset{256}{\mathrm{\&Sigma;}}}[\mathrm{MBgray}\left(n\right)*\mathrm{MBgray}(n+30)]$

5. the macro block in the original image is carried out the activity Active that the gradient computing obtains this macro block, I in the following formula _{I, j}The brightness value of the capable j row of expression current macro i:

$\mathrm{Active}=\frac{1}{16*15}\{\underset{i=0}{\overset{14}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{15}{\mathrm{\&Sigma;}}}\sqrt{|I_{i,j}-I_{i+1,j}|}+\underset{i=0}{\overset{15}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{14}{\mathrm{\&Sigma;}}}\sqrt{|I_{i,j}-I_{i,j+1}|}\}$

6. the macroblock motion vector in the original image is carried out the time masking property that computing obtains this macro block, MVx, MVy represent the motion vector of the level and the vertical direction of current macro in the following formula, and MVz represents that the time of this macro block covers characteristic:

$\mathrm{MVz}=\sqrt{{\mathrm{MVx}}^2+{\mathrm{MVy}}^2}$

7. calculated the perceptibility Focus of this macro block by above Several Parameters, wherein a, b, c are weighting parameters; Forward represents that whether current macro is the macro block in the prospect, if current macro is a foreground blocks, then Forward equals 1, otherwise equals 0:

Focus＝a*MVz+b*Smooth+c*Active+Forward

8. calculate the complexity Complexity of this frame by the perceptibility of all macro blocks in the frame:

$\mathrm{Complexity}=\frac{\underset{n=0}{\overset{395}{\mathrm{\&Sigma;}}}{\mathrm{Focus}}_n}{396}.$

4, the self-adaption code rate control algolithm based on space-time shielding effect according to claim 1 is characterized in that the step of the self-adaption code rate control algolithm in the described step (4) is as follows:

1. according to the bandwidth of Set For Current and the length of frame per second and image sets GOP, current GOP is distributed a target bit $T_r(n_i,0)=\frac{\mathrm{Bit}\_\mathrm{rate}}{\mathrm{Frame}\_\mathrm{rate}}*N_{\mathrm{gop}},$ T wherein _r(n _i, 0) and be the GOP target bit, Bit_rate is a bandwidth, Frame_rate is a frame per second, N _GopBe GOP length;

If 2. present frame is the I frame,, adjust the quantization parameter QP of I frame then according to the complexity of this frame _iα, β, γ are for regulating parameter in the following formula; Complexity is the complexity of present frame, and a, b, c are threshold value:

${\mathrm{QP}}_i=\left\{\begin{array}{cc}{\mathrm{QP}}_i+\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Complexity}>a)\\ {\mathrm{QP}}_i+\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Complexity}>b)\\ {\mathrm{QP}}_i+\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Complexity}<b)\end{array}\right.$

If 3. present frame is the P frame, then distribute target bit to be:

$f\left(n_{i,j}\right)=\mathrm{\&beta;}*\{\frac{\mathrm{bit}\_\mathrm{rate}}{\mathrm{frame}\_\mathrm{rate}}+\mathrm{\&alpha;}[\mathrm{Tbl}\left(n_{i,j}\right)-B_c\left(n_{i,j}\right)\left]\right\}$

$+(1-\mathrm{\&beta;})*\left\{\frac{W_p\left(n_{i,j-1}\right)*T_r\left(n_{i,j}\right)}{W_p\left(n_{i,j-1}\right)*N_{p,r}(j-1)}\right\}$

F (n in the formula _{I, j}) the preallocated bit number of the current P frame of expression, bit_rate represents bandwidth, frame_rate represents frame per second, Tb1 (n _{I, j}) expression i GOP j the preallocated bit number of P frame, B _c(n _{I, j}) j P frame of i GOP of expression preceding actual bit number that occupies of buffering area of encoding, W _p(n _{I, j-1}) complexity of j-1 P frame of i GOP of expression, T _r(n _{I, j}) j P frame of i GOP of expression preceding remaining bit number of GOP of encoding, α and β are constant;

4. according to the complexity of this P frame, adjust its target bit f (n _{I, j}) be:

$f\left(n_{i,j}\right)=\left\{\begin{array}{cc}f\left(n_{i,j}\right)*\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Complexity}>\mathrm{a\&alpha;})\\ f\left(n_{i,j}\right)*\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Complexity}>b)\\ f\left(n_{i,j}\right)*\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Complexity}<b)\end{array}\right.$

Wherein α, β, γ are for regulating parameter; Complexity is the complexity of present frame, and a, b, c are threshold value;

5. according to target bit, calculate the quantization parameter QP of this frame by R-Q secondary rate quantitative model;

6. revise its quantization parameter QP according to the perceptibility of macro block:

${\mathrm{QP}}_n=\left\{\begin{array}{cc}{\mathrm{QP}}_n+\mathrm{\&alpha;}& \mathrm{if}(\mathrm{Focus}>a)\\ {\mathrm{QP}}_n+\mathrm{\&beta;}& \mathrm{if}(a>\mathrm{Focus}>b)\\ {\mathrm{QP}}_n+\mathrm{\&lambda;}& \mathrm{if}(\mathrm{Focus}<b)\end{array}\right.$

QP in the following formula _nThe quantization parameter of representing n macro block, Focus are represented the macro block perceptibility, and α, β, γ are for regulating parameter, and a, b are threshold value.

5, the self-adaption code rate control algolithm based on space-time shielding effect according to claim 1 is characterized in that the step of the adjustment frame bit number in the described step (4) is as follows:

1. the bit number T before the coding of the j frame among i GOP _r(n _i, j)=Tr (n _i, j-1)-A (n _i, j-1), A (n wherein _i, j-1) the actual used bit number of j-1 frame coding of i GOP of expression;

2. the remaining bits number of buffering area is updated to:

$R_r(n_i,j)=R_r(n_i,j-1)+A(n_i,j-1)-\frac{\mathrm{Bit}\_\mathrm{rate}}{\mathrm{Frame}\_\mathrm{rate}}$

R wherein _r(n _i, j) the remaining bits number of the j frame coding back buffering area of i GOP of expression, A (n _i, j-1) the actual used bit number of j-1 frame coding of i GOP of expression, Bit_rate is a bandwidth, Frame_rate is a frame per second.

Images