CN106604031A - Region of interest-based H. 265 video quality improvement method - Google Patents

Abstract

The present invention discloses a region of interest-based H. 265 video quality improvement method. The objective of the invention is to improve subjective visual quality and maintain bit rate control accuracy. According to the method of the invention, a new graph-based saliency model is put forward; the region of interest of a current frame is estimated through a time-space domain saliency graph, so that the region of interest (ROI) can be determined, and frame-level and largest coded macroblock (LCU)-level bit allocation are guided; and for the problem of video quality smoothing, the adjustment of lambda values and QP values in the space domain and the time domain is restricted respectively. As indicated by experimental results, the algorithm can make an output bit rate reach a target value, and improve the peak area signal-to-noise ratio (PSNR) of the region of interest (ROI) of a video frame, and therefore, the overall subjective visual quality of a coded sequence is greatly increased compared with that obtained by using a bit rate control algorithm in the HM10.0 standards.

Description

A kind of H.265 video quality improvements method based on interest region

Technical field

The invention belongs to coding and decoding video field, is related to a kind of H.265 video quality improvements method based on interest region.

Background technology

H.265 video encoding and decoding standard is the motion pictures expert of ITU-T Video Coding Experts Groups (VCEG) and ISO/IEC The newest video encoding and decoding standard (JCT-VC) that standardization body's joint of group (MPEG) is released.H.265 standard remains base In the hybrid video coding concept of traditional block-based motion compensation, but substantial amounts of improvement and innovation are algorithmically carried out, Relative to former standard, supermacro block, the coding structure of quaternary tree and the more perfect coding post processing mould of 64x64 are such as introduced Block, such as SAO.H.265 code efficiency is significantly improved, compared with 264, Video coding effect is in identical image quality (PSNR) In the case of, code check saves 40～50%, but algorithm complex improves hundred times.

Rate Control is vital module in Video coding, and the reality of encoder output is adjusted according to target bit rate Border bit rate, it is to avoid the output of Video coding frame is excessive or too small, has influence on design and the network transmission of decoding.Various code rate is controlled Algorithm is suggested in video encoding standard evolution, such as MPEG-2 test models (TM5) and MPEG-4 checking models (VM) 8, H.264/AVC in JVT-N046.H.265 some new rate control algorithms are it is also proposed in.For example based on secondary Unified rate quantization (URQ) model based on R-Q models and referred to as two sub-pixels, this is also the code check control that HM6 recommends Algorithm processed.A kind of linear R- λ models are proposed in HM10.0, based on rate control algorithm λ models.It can be divided into two portions Point：Part I is bit allocation, and Part II is adjusting coding parameter according to allocated position.Part II is related to R- λ moulds Type

λ=α R^β (1)

Wherein, α and β are the parameters related to video source, and R is that target sets code check.λ is that Lagrangian is excellent in rate distortion Change and play very important effect during (RDO).Quantization parameter (QP) follows from the acquisition in equation：

QP=4.2005ln λ+13.7122 (2)

Because based on its excellent compression performance of the rate control algorithm of linear R- λ models, this rate control algorithm is current The rate control algorithm that H.265 reference software (HM) is recommended is become.From the code that video and human visual system, HM10.0 are recommended Rate control algolithm also have following 2 points it is not enough, there is improved space：

1) it does not account for human-eye visual characteristic, and the bit distribution for causing LCU layers is not optimum.Its LCU levels bit point With being to belong to the LCU mean absolute differences (MAD) of same position based on former coded frame.However, the incomparable people of these methods The characteristics of class vision system, because region has very high mean absolute difference, but this region can't be subject to too many The concern of people.

2) there are two kinds of situations in video sequence：Object is being moved, background geo-stationary；Background is in motion, and object is relatively quiet Only.This two kinds of situations, based on the rate control algorithm of linear R- λ models, can not all obtain optimal subjective visual experience.

The content of the invention

The purpose of the present invention is to overcome the deficiencies in the prior art, is added in JCTVC-K0103R- λ model rate control process Enter one kind and be based on area-of-interest H.265 Rate Control improved method.

Technical scheme is as follows：

The invention discloses a kind of H.265 video quality improvements method based on interest region, comprises the steps：

1) the related parameter of code check is initialized：R- λ pattern Rate Control initial quantization parameters QP equations are as follows：

λ=α R^β (1)

QP=4.2005ln λ+13.7122 (2)

Wherein α and β are the parameters related to video source, and R is target bit rate value, and λ is rate distortion Lagrangian, formula (2) it is exactly to obtain initial quantization parameters；

2) spatial domain notable figure component S is merged based on relevant specification fusion method_pWith time domain notable figure component S_m, when obtaining Empty notable figure S_F, equation is as follows：

S_F=θ₁S_m+θ₂S_p+θ₃S_mS_p (3)

Wherein：U and v are the horizontal and vertical components of the grand motion vector of macro block；S_pBy based on figure Markov chain balanced distribution obtaining, θ₁,θ₂,θ₃It is weighted factor；

By space-time remarkable figure S_FFrame of video area-of-interest and regions of non-interest are judged；

3) the Rate Control initialization procedure of region of interest is distinguished

It is consistent with HM10.0 that frame target position is set, then according to quality factor K by the target of frame before bit allocation procedures Bit number is divided into two parts, and quality factor K is bit number ratio needed for ROI and NROI, ROI and NROI target bits are bases Below equation determines

T=T_ROI+T_NROI (5)

T_ROI=K × T_NROI (6)

Wherein T, T_ROI, and T_NROIPresent frame, the respective target bits of ROI and NROI are represented, in order to ensure the matter in time domain Amount is stable, and parameter lambda and QP need to be limited to a less scope. λ spans：

QP spans：

QP_XlastPic-10≤QP_XcurrPic≤QP_XlastPic+10 (8)

Wherein x is probably ROI or NROI；CurrPic, lastPic represent respectively present frame relevant parameter, previous frame coding Parameter；

4) considering relation pair λ and QP values of current LCU and periphery LCU carries out following scope control：

If current LCU and its left adjacent LCU, belongs to same type, λ the and QP values amendment formula of its ROI or NROI is such as Under：

QP_lastLCU-1≤QP_currLCU≤QP_lastLCU+1 (10)

Wherein lastLCU means the parameter related to previous coding LCU；If current LCU and its left adjacent LCU category In different types, itself λ and QP values amendment formula is as follows：

QP_lastLCU-10≤QP_currLCU≤QP_lastLCU+10 (12)

5) the bit distribution of video frame level

ω_iIt is the weights of each frame of video in current video frame group GOP group, Coded_GOPBe present image group GOP group Use up position；

6) judge whether that also new images group GOP or image sets GOP have new frame of video, if any modification sequential parameter i pair Next image is encoded, and skips to step 2), coding is terminated if not.

It is described by space-time remarkable figure S as preferred_FFrame of video area-of-interest and regions of non-interest are carried out Judge, process is as follows：

The first step：Each pixel space-time remarkable figure accumulated value in each macro block LCU is calculated by formula (4)：

Wherein：M and N are the wide and height of current macro, S_F(i, m, n) is i-th LCU, the pixel in (m, n) coordinate Space-time remarkable figure；

Second step：Space-time remarkable figure ws (i) value of all of macro block in current encoded frame is obtained, is entered by the order for arriving little greatly Row arrangement, sets threshold value T, more than threshold value it is assumed that for be current video frame ROI region, otherwise as NROI regions.

As preferred, the weighted factor θ₁、θ₂、θ₃It is respectively set to 0.5,0.3,0.5.

The present invention is then right respectively by the way that region of interest and the differentiation of non-region of interest are carried out to frame of video based on the method for figure Region of interest and non-region of interest carry out differentiated rate control algorithm, prevent regions of non-interest from excessively taking region of interest The data bit in domain, it is ensured that the video quality improvements of area-of-interest；Area-of-interest bit bit allocation is more, non-region of interest Domain bit bit allocation is a little less, the final subjective quality for realizing lifting video.Test result indicate that, the algorithm compares can output While special rate reaches desired value, the interest region Y-PSNR of frame of video is effectively improved, therefore coded sequence is whole Body subjective visual quality is compared the rate control algorithm carried in HM10.0 standards and is greatly improved.

The present invention is constrained to ensure smooth visual quality by adjusting λ and QP.Except making actual coding ratio The close target bit rate of special rate, compares H.265 existing rate control algorithm, set forth herein bit rate control method have preferably Area-of-interest (ROI) information, is greatly improved well as subjective video quality.

Description of the drawings

Fig. 1：Rate control process block diagram；

Fig. 2:Reconstructing video contrasts PSNR curve maps.

Specific embodiment

Below in conjunction with accompanying drawing, embodiments of the present invention is described in detail.The present embodiment Jing the technology of the present invention method with Premise is implemented, and gives detailed embodiment and concrete operations, but protection scope of the present invention is not limited to following enforcements Example.

As shown in figure 1, the H.265 video quality improvements method based on interest region, step is as follows：

1) the related parameter of code check is initialized：R- λ pattern Rate Control initial quantization parameters QP equations are as follows：

λ=α R^β (1)

QP=4.2005ln λ+13.7122 (2)

Wherein α and β are the parameters related to video source, and R is target bit rate value, and λ is rate distortion Lagrangian, formula (2) it is exactly to obtain initial quantization parameters；

2) spatial domain notable figure component S is merged based on relevant specification fusion method_pWith time domain notable figure component S_m, when obtaining Empty notable figure S_F, equation is as follows：

S_F=θ₁S_m+θ₂S_p+θ₃S_mS_p (3)

Wherein：U and v are the horizontal and vertical components of the grand motion vector of macro block；S_pBy based on figure Markov chain balanced distribution obtaining, θ₁,θ₂,θ₃It is weighted factor；

A) the notable figure in spatial domain

For the final notable figure in the spatial domain needed for producing, the famous conspicuousness model based on figure (GB) is borrowed. It is utilized and Itti model identical feature extracting methods, and the algorithm characteristics channel generally adopts color, intensity, direction and contrast Degree.The weighted factor of all passages of improved method all adopts these parameters.Assume that a Feature Mapping is produced, its expression formula is： M:[n]²→R.Next step operates in activating area or notable figure on M domains.One is introduced based on the conspicuousness model of figure (GB) Individual CTM Approach this process.Two nodes P and Q in for Feature Mapping, their otherness can be defined as：

This otherness is most directly defined：The ratio of two quantity measured on simple logarithmic scale.A points one are completely The digraph G of connection_ACan be obtained with every other n-1 nodes by each node in connection M domains.Digraph G_AEdge section Point is to node q and q weighted expression formula w_A(p, q) is as follows：

Wherein (p_x,p_y),(q_x,q_y) be node P and Q coordinate, σ is a free parameter (usually to ten/10ths / 5th picture width).Fringe node in contrary direction also has equal weight value.In G_AMarkov chain can be with An equivalent definition between node and state, and side right value and transfer are drawn by the weight on the side of each departure of node 1 of specification Probability.The balanced distribution of this chain, reflects in the long run time component of the random walk under each node or state, compares Surroundings nodes, meeting nature has higher accumulation on this node, because be transferred to such subgraph being possible to, it is impossible to turn Move on to on similar M value nodes, this result is one from measurement relatively more useful two-by-two.

After all activated figure is obtained, they should be normalized before additive combination.By another horse It is as follows that Er Kefu algorithms carry out process of normalization：Hypothesis has an activation figure：A:[n]²→R.One digraph G being fully connected_N Can be by being connected to and every other each joint structure of N-1 node.For two nodes p and q in A domains, its side adds Weight function expression formula is as follows：

Wherein A (p) is the activation value of node p, by the Markov chain balanced distribution for calculating, obtains normalized activation Figure.

B) the notable figure in time domain

Under normal circumstances, mobile object is the great point of interest of spectators.Motion vector (MV) is one in time domain notable figure Important characteristic parameter, because it can represent the exercise intensity of object.It is extracted based on Block- matching, in frame above The best matching blocks of search.Due to the relatively low computation complexity of mean absolute difference (MAD), pretend as algorithm and self adaptation ten The searching algorithm of word pattern search (ARPS).Block size is set to 16x16 and hunting zone 7, and amplitude M of MV can represent its block Exercise intensity, it is calculated as follows：

Wherein u and v are horizontal and vertical movement components.There are two kinds of situations in video sequence：Object is being moved, and background is relative It is static；Background is in motion, object geo-stationary.In the latter case, the size of motion vector is can not to represent its block Significant properties.

3) notable figure and area-of-interest combination decision

In order to obtain space-time remarkable figure S_F, notable figure spatially and temporally is combined based on relevant specification fusion method, such as Under：

S_F=θ₁S_m+θ₂S_p+θ₃S_mS_p (7)

Wherein θ₁,θ₂,θ₃It is weighted factor, is rule of thumb respectively set to 0.5,0.3,0.5. and selects to time domain notable figure Larger weighted factor, because the eyes of mobile object always more attractive people.Eq.9 the first two parameters be by when Spatial domain notable figure is lifting pixel independent attribute.On the other hand, the parameters of eq.9 the 3rd be by empty or domain notable figure come pair when Domain notable figure is weighted, and vice versa.Therefore, it is a term mutually promoted, and it strengthens those spatially and temporally On all significant pixel.

Area-of-interest can be determined according to notable figure.The LCU significant properties of each frame can be weighed by following formula：

Wherein ws (i) is considered the weighted value of i-th LCU notable figure, S_FThe value of (i, m, n) and coordinate (m, n) The notable figure of i-th LCU in corresponding frame, m and n is respectively the width and height of the block.In by calculating a frame of video All of ws (i), then being ranked up from big to small, selectes threshold value T, and ws (i) values of LCU will be set to more than T Area-of-interest (ROI), will be considered as regions of non-interest (NROI) less than T.

4) select for target bit rate setting and λ under ROI/NROI

The purpose of Rate Control is to make actually used bit number reach desired value.In this innovatory algorithm, for ROI It is independently to go decision-making with NROI data bit allocations.In other words, ROI and NROI have themselves target position.Should be noted , it is consistent with HM10.0 methods that frame target position is set before bit allocation procedures.Then according to quality factor K by the target of frame Bit number is divided into two parts, and quality factor K is bit number ratio needed for ROI and NROI.ROI and NROI target bits are bases Below equation determines

T=T_ROI+T_NROI (9)

T_ROI=K × T_NROI (10)

Wherein T, T_ROI, and T_NROIRepresent present frame, the respective target bits of ROI and NROI. in order to ensure the matter in time domain Amount is stable, and parameter lambda and QP need to be limited to a less scope. λ spans：

QP spans：

QP_XlastPic-10≤QP_XcurrPic≤QP_XlastPic+10 (14)

Wherein x is probably that ROI or NROI.currPic, lastPic represent respectively present frame relevant parameter, previous frame coding Parameter.

In the Rate Control of LCU levels, the target position setting of each LCU should be in direct ratio with the notable angle value of current LCU, Its expression formula is as follows：

WhereinWithRepresent that ROI and NROI residues digit .R (p) is p-th LCU interesting region estimating respectively Digit order number, M_leftIt is without the LCU numbers of coding in ROI region.R (q) is that p-th LCU regions of non-interest estimates position, N_left It is without the LCU numbers of coding in NROI regions.Estimated by the number of targets word bit to current LCU, the initial value of λ and QP Obtained by Eq.1 and Eq.2. in view of the Video quality smoothing attribute of every frame, λ and QP values should be controlled in following scope：

QP_XcurrPic-2≤QP_currLCU≤QP_XcurrPic+2 (17)

Wherein X can depend on current LCU to classify with ROI or NROI.Currlcu is meant to be current LCU to join accordingly Number.λ the and QP values of LCU also should be by those neighbouring LCU in spatial domain smoothness constraint.If current LCU and its a left side are adjacent LCU, belongs to same type, such as last coding LCU units, λ the and QP values amendment formula of its ROI or NROI is as follows：

QP_lastLCU-1≤QP_currLCU≤QP_lastLCU+1 (19)

Wherein lastLCU means the parameter related to previous coding LCU.If current LCU and its left adjacent LCU category In different types, itself λ and QP values amendment formula is as follows：

QP_lastLCU-10≤QP_currLCU≤QP_lastLCU+10 (21)

5) the bit distribution of video frame level

ω_iIt is the weights of each frame of video in current video frame group GOP group, Coded_GOPBe present image group GOP group Use up position；

6) judge whether that also new images group GOP or image sets GOP have new frame of video, if any modification sequential parameter i pair Next image is encoded, and skips to step 2), coding is terminated if not.

Fig. 2 is that HM10.0 fixes QP, the master under HM10.0 Rate Controls and the inventive method (improvement bit rate control method) See the comparative result of visual test.It is the average MOS values of ten objective examinations that DMOS is represented.Relatively low value represents more preferable vision matter Amount.First, the two sequences are that fixed QP is respectively provided with：32,38,40,42 and 46.Their output bit rate is provided as In the test target bit rate of hm10.0 rate control algorithms.Method proposed by the present invention can be obtained more in the case of colleague's code check Good visual quality.The inventive method main thought is that area-of-interest distributes more bits, regions of non-interest distribution A little less bit, particularly in the case of low bit- rate, can greatly lift the subjective visual quality of whole video.In high code check In the case of, three kinds of methods can provide very good subjective quality, and at this moment the eyes of the mankind are difficult substantially to distinguish them it Between difference.

Table 1 shows that different sequences have the experimental result of different target bit rate.

The test result of table 1 (actual bit rate, ROI and NROI PSNR) is contrasted

As can be seen that comparing with HM10, the present invention is in the case of bit rate precision fluctuation is in the range of 0～1%.This In area-of-interest (ROI) 1.15～1.5db of video quality improvements, the cost paid is regions of non-interest (NROI) to bright method Video quality has 0.76～1.49db to lose.

Claims (3)

1. a kind of H.265 video quality improvements method based on interest region, it is characterised in that comprise the steps：

1) the related parameter of code check is initialized：R- λ pattern Rate Control initial quantization parameters QP equations are as follows：

λ=α R^β (1)

QP=4.2005ln λ+13.7122 (2)

Wherein α and β are the parameters related to video source, and R is target bit rate value, and λ is rate distortion Lagrangian, and formula (2) is just It is to obtain initial quantization parameters；

2) spatial domain notable figure component S is merged based on relevant specification fusion method_pWith time domain notable figure component S_m, obtain space-time and show Write figure S_F, equation is as follows：

S_F=θ₁S_m+θ₂S_p+θ₃S_mS_p (3)

Wherein：U and v are the horizontal and vertical components of the grand motion vector of macro block；S_pBy the Ma Er based on figure Can husband's chain balanced distribution obtaining, θ₁,θ₂,θ₃It is weighted factor；

By space-time remarkable figure S_FFrame of video area-of-interest and regions of non-interest are judged；

3) the Rate Control initialization procedure of region of interest is distinguished

It is consistent with HM10.0 that frame target position is set, then according to quality factor K by the target bits of frame before bit allocation procedures Number is divided into two parts, and quality factor K is bit number ratio needed for ROI and NROI, ROI and NROI target bits are according to following Formula determines

T=T_ROI+T_NROI (5)

T_ROI=K × T_NROI (6)

Wherein T, T_ROI, and T_NROIPresent frame, the respective target bits of ROI and NROI are represented, it is steady in order to ensure the quality in time domain Determine, parameter lambda and QP need to be limited to a less scope. λ spans：

${\mathrm{\λ}}_{XlastPic}\·2^{-\frac{10.0}{3.0}}\≤{\mathrm{\λ}}_{XcurrPic}\≤{\mathrm{\λ}}_{XlastPic}\·2^{\frac{10.0}{3.0}}---\left(7\right)$

QP spans：

QP_XlastPic-10≤QP_XcurrPic≤QP_XlastPic+10 (8)

Wherein x is probably ROI or NROI；CurrPic, lastPic represent respectively present frame relevant parameter, previous frame coding ginseng Number；

4) considering relation pair λ and QP values of current LCU and periphery LCU carries out following scope control：

If current LCU and its left adjacent LCU, belongs to same type, λ the and QP values amendment formula of its ROI or NROI is as follows：

${\mathrm{\λ}}_{lastLCU}\·2^{-\frac{1.0}{3.0}}\≤{\mathrm{\λ}}_{currLCU}\≤{\mathrm{\λ}}_{lastLCU}\·2^{\frac{1.0}{3.0}}---\left(9\right)$

QP_lastLCU-1≤QP_currLCU≤QP_lastLCU+1 (10)

Wherein lastLCU means the parameter related to previous coding LCU；If current LCU and its left adjacent LCU belongs to not Same type, itself λ and QP values amendment formula is as follows：

${\mathrm{\λ}}_{lastLCU}\·2^{-\frac{10.0}{3.0}}\≤{\mathrm{\λ}}_{currLCU}\≤{\mathrm{\λ}}_{lastLCU}\·2^{\frac{10.0}{3.0}}---\left(11\right)$

QP_lastLCU-10≤QP_currLCU≤QP_lastLCU+10 (12)

5) the bit distribution of video frame level

$T_{CurrPic}=\frac{T_{GOP}-{\mathrm{Coded}}_{GOP}}{\underset{NotCodedPictures}{\mathrm{\Σ}}{\mathrm{\ω}}_i}\·{\mathrm{\ω}}_{CurrPic}---\left(13\right)$

ω_iIt is the weights of each frame of video in current video frame group GOP group, Coded_GOPIt is that present image group GOP group has been used up Position, T_GOPIt is frame of video group GOP setting code check value.

6) judge whether that also new images group GOP or image sets GOP have new frame of video, if any, video sequence i Jia 1, Next two field picture is encoded, step 2 is skipped to), coding is terminated if not.

2. a kind of H.265 video quality improvements method based on interest region according to claim 1, it is characterised in that institute State by space-time remarkable figure S_FProcess is as follows to be judged to frame of video area-of-interest and regions of non-interest：

The first step：Each pixel space-time remarkable figure accumulated value in each macro block LCU is calculated by formula (4)：

$ws\left(i\right)={\mathrm{\Σ}}_{m=1}^M{\mathrm{\Σ}}_{n=1}^N{S}_F(i,m,n)---\left(4\right)$

Wherein：M and N are the wide and height of current macro, S_F(i, m, n) is i-th LCU, the space-time of the pixel in (m, n) coordinate Notable figure；

Second step：Space-time remarkable figure ws (i) value of all of macro block in current encoded frame is obtained, is arranged by the order for arriving little greatly Row, set threshold value T, more than threshold value it is assumed that for be current video frame ROI region.

3. a kind of H.265 video quality improvements method based on interest region according to claim 1, it is characterised in that institute State weighted factor θ₁、θ₂、θ₃It is respectively set to 0.5,0.3,0.5.