CN105791980A - Resolution improvement based film and TV works renovation method - Google Patents

Abstract

The present invention provides a resolution improvement based film and TV works renovation method for film and TV works with low resolution and low definition. The method includes the specific steps of firstly, acquiring a resolution of an original video and a target resolution, and calculating a zoom ratio; then, segmenting an input video into an image frame set according to a certain segmentation manner; then, performing conversion according to a prestored mapping relation, so as to obtain high resolution video frames; and finally, combing the high resolution video frames to obtain a high resolution video. The prestored mapping relation is obtained by learning a hybrid expert model, and the process is finished offline in a computer. The method has the advantages of good adaptability, fast speed, good effect, expandability and so on.

Description

Films and television programs based on increase resolution renovate method

Technical field

The invention belongs to computer vision and image processing field, relate to films and television programs renovation method, be specifically related to a kind of films and television programs based on increase resolution and renovate method and system.

Background technology

Along with video acquisition, transmission, storage, Display Technique development, films and television programs constantly towards high-resolution develop.The taste that people appreciate video is also more and more higher, constantly pursues the films and television programs of high-resolution, fine definition.Meanwhile, the appearance (such as 4K, 5K TV and display) of high-resolution display device, make again the universal of high-resolution films and television programs be possibly realized.

But on the other hand, many classical films and television programs of the remote past limit due to technological means, still have relatively low resolution, relatively low definition and poor visual effect.Simultaneously because of the remote past, the film holding time is longer, adds the destruction of the extraneous factor such as natural disaster, war, various quality degradation occurs, such as breakage, flicker, noise, shake etc..People want to review classical films and television programs on the one hand, on the one hand the quality of film have been had again new requirement.Reviewing classical films and television programs to meet people simultaneously and pursue the demand of high-quality video, films and television programs reconditioning technology arises at the historic moment.It is application image/video processing technique that films and television programs renovate its essence, and original video is processed, and eliminates various quality degradation, to improve the visual effect of original video.

Films and television programs are the same with books, are the important culture carriers of human society, some classical films and television programs, although of the remote past, but its cultureal value cannot replace.Therefore, the films and television programs that these are of the remote past are renovated, replay has just shown very important meaning.Specifically, the meaning of films and television programs renovation includes the following aspects:

1. some classical films and television programs, such as documentary film etc., are precious historical summaries.These films and television programs are renovated, can better preserve, propagate these historical summaries.

2. pair classical films and television programs renovate, and allow more modern appreciate, and are by the important form of culture and arts succession.

3. pair classical films and television programs renovate, and allow classical artistic work taking on a new look again, are to artistical maximum respect and souvenir.

The method of existing raising films and television programs visual effect is concentrated mainly in video source modeling means, and as removed noise, removal is fuzzy, remove staggered, enhancing contrast ratio, enhancing color etc..Former video can be played the effect that visual effect strengthens by these methods, but video is not carried out increase resolution, so there is no the demand inherently meeting people to classical films and television programs renovation.

So-called increase resolution, refers to the video (or frame of video) of low resolution, by certain method, generates a high-resolution video fast and effectively.Its difficult point is in that how to break through the restriction of original low-resolution video pixel quantity, fill original non-existent pixel, should keep the structure of former low-resolution video, texture, more naturally reasonable at human eye again.

Traditional increase resolution method, mainly includes based on interpolation, based on rebuilding and based on the method learnt.Method based on interpolation is by existing pixel is carried out linear combination, as the pixel of disappearance.Interpolation algorithm is simply rapid, but " mosaic " effect easily occurs or cross smooth phenomenon；Utilizing the similarity of multiple image to carry out registration reconstruction based on the algorithm rebuild, but this kind of algorithm is often simple combination multiple image, effect is unsatisfactory；Based on the algorithm learnt mainly by a number of training data, obtaining the low-resolution video mapping relations to high-resolution video according to special algorithm training, this type of algorithm is higher to the requirement of model, easy over-fitting or poor fitting, and operand is big, speed slow, practicality is not high.It can be said that above-mentioned video resolution Upgrade Problem annoyings users always.

Summary of the invention

The technology of the present invention solves problem: the present invention provides a kind of films and television programs renovation method and system promoted based on video resolution, by the films and television programs (being usually less than 720P) of low resolution through resolution enhancement technology, be converted to high-resolution video (such as 1080P, 4K etc.) and realize films and television programs renovation.

The technical solution of the present invention is: the concrete scheme of the present invention is: first, obtains resolution and the target resolution of original video, calculates scaling；Secondly, input video is divided into picture frame by certain partitioning scheme；Then, the mapping relations according to prestoring convert, and obtain high-resolution video frame；Finally, high-resolution video frame is combined into high-resolution video.Wherein, the mapping relations model prestored obtains based on Mixture of expert model learning, and this model training process off-line in a computer completes.Concrete steps bag is as follows:

Study mapping relations model:

(1) pretreatment training video

(1.1) choose high-resolution video as training sample, and be split as high-resolution video frame；

(1.2) the high-resolution video frame of step (1.1) gained is used gaussian kernel convolution

(1.3) calculate amplification according to original low-resolution video and target high-resolution video, carry out partiting row sampling according to the multiple of gained, obtain the low resolution video frame of correspondence；

(1.4) low resolution video frame of high-resolution video frame and sampling gained is split in bulk respectively, as training data.

(2) mapping relations model is obtained based on Mixture of expert model

(2.1) a Mixture of expert model is initialized.Mixture of expert model includes expert and gate function two parts, and its structure is tree-like, as shown in Figure 2.In figure, the leafy node in tree structure is called expert, is responsible for data are carried out mapping transformation；Root node is called gate function, is responsible for data and selects suitable expert.The present invention uses linear function as expert's function:

Y=Wx

Wherein W is expert's function parameter, and x and y represents low resolution video frame block and corresponding high-resolution video frame block respectively.

Gate function is responsible for determining to select which expert that data are converted, and in the present invention, i-th gate function is expressed as:

$g_i(x,v_i)=\exp {(x-v_i)}^2{\left({\mathrm{\Σ}}_{j=1}^K{\left(x-v_j\right)}^2\right)}^{-1}$

Wherein, x and y represents low resolution video frame block and corresponding high-resolution video frame block, v respectively_iRepresent i-th gate function parameter, v_jRepresenting jth gate function parameter, K is the number of expert in Mixture of expert model, i.e. the number of leaf node in tree structure.The initialization of Mixture of expert model specifically includes following steps:

(2.1.1) the quantity K of expert is specified；

(2.1.2) the probability distribution Gaussian distributed of each expert is supposed: p (y | x, W_i)=N (y (x, W_i), σ), wherein W_iRepresenting the parameter of i-th expert, σ is the standard deviation of Gauss distribution.Assuming that parameter W_iDistribution also Gaussian distributed: p (W_i)=N (0, μ), wherein μ represents the average of Gauss distribution.

(2.1.3) k-mean algorithm is adopted to be clustered according to the quantity K of expert by training data, the initial value W of the parameter of each expert_i ⁽⁰⁾It is appointed as in class slope, the initial value v of each gate function parameter_i ⁽⁰⁾It is appointed as cluster centre；

(2.1.4) initial value of each gate function is calculated:

${g_i}^{\left(0\right)}(x,{v_i}^{\left(0\right)})=\exp {(x-{v_i}^{\left(0\right)})}^2{\left({\mathrm{\Σ}}_{j=1}^K{\left(x-{v_j}^{\left(0\right)}\right)}^2\right)}^{-1}$

Wherein x represents low resolution video frame block, v_i ⁽⁰⁾Representing the initial value of i-th gate function parameter, K is the number of expert in Mixture of expert model, i.e. the number of leaf node in tree structure.

(2.2) training data using step (1.4) to obtain, is iterated Mixture of expert model optimizing, until iterative process convergence, the model parameter finally given is mapping relations model.Mapping relations model includes gate function parameter and expert parameter.

(2.2.1) allowable error ε during iteration ends is specified；

(2.2.2) posterior probability of each gate function in epicycle iteration is calculated:

$h_i^{(k+1)}\left(x\right)=\frac{{g_i}^{\left(k\right)}(x,{v_i}^{\left(k\right)})p_i\left(y\right|x,{W_i}^{\left(k\right)})}{{\mathrm{\Σ}}_{j=1}^K{g_i}^{\left(k\right)}(x,{v_i}^{\left(k\right)})p_j\left(y\right|x,{W_j}^{\left(k\right)})}$

Wherein k is iterative steps, p_i(y | x, W_i ^(k)) and p_j(y | x, W_j ^(k)) represent expert probability distribution, g_i ^(k)(x, v_i ^(k)) represent that the kth of i-th gate function walks iterative value.

(2.2.3) each expert parameter is updated:

$W_i^{(k+1)}=\frac{{{\mathrm{YH}}_i}^{(k+1)}{X}^T{\mathrm{\μ}}^2}{{{\mathrm{XH}}_i}^{(k+1)}{X}^T{\mathrm{\μ}}^2+{\mathrm{\σ}}^{2}I}$

Wherein k is iterative steps, and X is the vector of all low resolution block x composition in training data, and Y is the vector of all high-resolution block y composition, X in training data^TRepresent the transposition of X, I representation unit matrix, H_i ^(k+1)Represent in kth+1 step the vector of the posterior probability composition of all low resolution block x corresponding to i-th expert.

(2.2.4) each gate function parameter is updated:

$v_i^{(k+1)}=v_i^{\left(k\right)}-(x^{\left(t\right)}-v_i^{\left(k\right)})\underset{t=1}{\overset{N}{\mathrm{\Σ}}}\left(h_i^{\left(k\right)}\right(x^{\left(t\right)})-g_i(x^{\left(t\right)},v_i^{\left(k\right)}\left)\right)$

WhereinRepresent i-th gate function parameter in kth step iteration,The posterior probability of i-th gate function, x in iteration is walked for kth^(t)Represent the t low resolution block.

(2.2.5) output of each gate function in epicycle iteration is calculated:

${g_i}^{(k+1)}(x,{v_i}^{(k+1)})=\exp {(x-v_i^{(k+1)})}^2{\left({\mathrm{\Σ}}_{j=1}^K{\left(x-v_j^{(k+1)}\right)}^2\right)}^{-1}$

(2.2.6) likelihood probability in epicycle iteration is calculated:

$Q^{(k+1)}=\underset{t=1}{\overset{N}{\mathrm{\Σ}}}\underset{i=1}{\overset{K}{\mathrm{\Σ}}}{h}_i^{(k+1)}\left(x\right)\left(\log {g_i}^{(k+1)}\right(x,{v_i}^{(k+1)})+\log p_i(y|x,{W_i}^{(k+1)}\left)p\right({W_i}^{(k+1)}\left)\right)$

Wherein, p_i(y | x, W_i ^(k+1)) represent expert probability distribution, p (W_i ^(k+1)) represent expert parameter probability distribution.

(2.2.7) judge whether iteration restrains.During allowable error ε when the absolute value of difference of likelihood probability of likelihood probability and last round of iteration of epicycle iteration is less than iteration ends, finishing iteration.Otherwise repeated execution of steps (2.2.2)～(2.2.7).

The gate function parameter v obtained when iteration terminates_i, together with expert quantity K, expert parameter W_i, the standard deviation sigma of probability distribution of expert and expert parameter mean of a probability distribution μ, be stored in disk together as final mapping relations model.

After having learnt mapping relations model and having stored, use the mapping relations model of storage that video is carried out increase resolution:

(3) low-resolution video that pretreatment is pending

(3.1) low-resolution video is split as low resolution video frame；

(3.2) low resolution video frame of step (3.1) gained is divided into low resolution video frame block；

(4) the mapping relations model that low-resolution video obtains according to step (2) is promoted to high-resolution video, including:

(4.1) low resolution video frame block step (3) obtained is as the input of gate function, and the gate function parameter in the mapping relations model that use step (2) obtains calculates the output of each gate function:

$g_i(x,v_i)=\exp {(x-v_i)}^2{\left({\mathrm{\Σ}}_{j=1}^K{\left(x-v_j\right)}^2\right)}^{-1}$

Wherein, x is the low resolution video frame block of input.K is the number of expert, v in Mixture of expert model_iRepresent i-th gate function parameter, obtain each through step (2.2).

(4.2) parameter of the expert's function corresponding to gate function that use output valve is maximum calculates corresponding high-resolution block, and wherein the parameter of expert's function is obtained by step (2)；

(4.2.1) calculating obtains the sequence number of maximum output valve gate function: i=argmax (g_i)

Wherein, g_iFor the output of i-th gate function, obtained by step (4.1).

(4.2.2) i-th expert's function is used to calculate high-resolution video frame block: y=W_ix

Wherein, W_iFor the parameter of i-th expert's function, y is the high-resolution video frame block corresponding to low resolution video frame block x of input.

(4.3) each low resolution block is carried out increase resolution according to the step of (4.1) and (4.2), obtain the high-resolution block of correspondence, all of high-resolution block is spliced into according to position in low resolution video frame of the low resolution block of its correspondence the high-resolution video frame of correspondence；

(4.4), after obtaining the high-resolution video frame that all low resolution video frame are corresponding, it is combined into high-resolution video.

In described step (4), all without dependence between frame of video block, between frame of video, therefore can using GPU processor, parallel acceleration processes this step.

The described films and television programs based on increase resolution renovate method, it is possible to use with the form of computer software player, it is also possible to be integrated in hardware platform (such as Set Top Box, intelligent television etc.) and use.

The described films and television programs based on increase resolution renovate method, it is possible to coordinate other screen Enhancement Method as pretreatment or post processing means, it is possible to promote visual effect further.

Present invention advantage compared with prior art is in that: visual effect, and the high-resolution video details obtained by enforcement the present invention program is complete, edge clear, Acacia crassicarpaA are good, and fast and stable.Specifically, the feature of the present invention includes:

1. self adaptation.The present invention program adaptive polo placement scaling multiple, is suitable for different increase resolution demands.

2. speed is fast.Owing to being absent from dependence between sequence of frames of video, it is possible to improve processing speed by parallel processing.It addition, the processing procedure of algorithm is that sequence of frames of video is carried out Linear Mapping conversion, mapping parameters used can pre-save in memory, it is possible to improves processing speed further.

3. effective.The mapping parameters that the present invention carries out using in increase resolution process is based on Mixture of expert model learning and obtains, and has broken and has divided the drawback learning to separate with submodel in the tradition increase resolution algorithm based on study.Simultaneously, combine statistical robustness advantage and the accuracy advantage based on learning algorithm, avoid the drawback that in the past can not utilize mass data information based on learning algorithm, simultaneously also achieve the precision higher than simple statistical method, also can obtain good effect and processing speed faster even for the video that resolution own is higher.

4. expansible.Owing to being absent from dependence between sequence of frames of video, it is possible to realize parallel processing raising processing speed by applying the technological means such as GPU acceleration.It addition, algorithm proposed by the invention may be directly applied to image resolution ratio promotes field.

Accompanying drawing explanation

Fig. 1 is the flow chart that the films and television programs based on increase resolution of the present invention renovate method.

Fig. 2 is Mixture of expert model structure schematic diagram of the present invention.

Frame of video of the present invention is divided into frame of video block schematic diagram by Fig. 3.

Detailed description of the invention

The method of the invention is illustrated by detailed description below with an example.

Renovate method according to the films and television programs based on increase resolution of the present invention, the process that the video that original resolution is 768*432 is promoted to 3072*1728 comprised the following steps:

(1) pretreatment training video

(1.1) high-resolution films and television programs are selected, Video processing software is utilized to read in the video flowing of films and television programs, each frame in video flowing is saved as frame of video, in the present embodiment, films and television programs length is 1200 seconds, frame rate is 25 frames/second, and gained frame of video adds up to: 1200*25=15000；

(1.2) frame of video of step (1.1) gained being used average is 0, and standard deviation is the gaussian kernel convolution of 1；

(1.3) obtain original low-resolution video resolution and target resolution, calculate amplification according to both.Original resolution is 768*432, and target resolution is 3072*1728, and amplification is: 3072/768=4.Accordingly, the frame of video after convolution is down sampled to the 1/4 of original size, obtains the low resolution video frame of correspondence.

(1.4) each width low resolution video frame of gained by existing segmentation standard, it is divided into being sized to the not overlapping fritter of 10 × 10 pixels, as it is shown on figure 3, and therefrom choose 1,000,000 piece as training data.

(2) mapping relations model is obtained based on Mixture of expert model

(2.1) Mixture of expert model is initialized

(2.1.1) the quantity K of expert is specified.In the present embodiment, take K=100；

(2.1.2) specify parameter σ and the μ of the probability distribution of expert and the probability distribution of expert parameter, in the present embodiment, take σ=0.32, μ=0.58；

(2.1.3) k-mean algorithm is adopted to be clustered according to the quantity K of expert by training data, the W of each expert_i ⁽⁰⁾Parameter initialization is slope in class, gate function parameter v_i ⁽⁰⁾It is initialized as cluster centre；

(2.1.4) initial value of each gate function is calculated according to following formula:

${g_i}^{\left(0\right)}(x,{v_i}^{\left(0\right)})=\exp {(x-{v_i}^{\left(0\right)})}^2{\left({\mathrm{\Σ}}_{j=1}^K{\left(x-{v_j}^{\left(0\right)}\right)}^2\right)}^{-1}$

Wherein x represents low resolution video frame block, v_i ⁽⁰⁾Representing the initial value of i-th gate function parameter, K is the number of expert in Mixture of expert model, i.e. the number of leaf node in tree structure.

(2.2) using the training data that step (1.4) obtains, the Mixture of expert model that step (2.1) is obtained is iterated optimizing:

(2.2.1) allowable error ε during iteration ends is specified.In the present embodiment, error ε=0.005 that delivery type iteration ends allows.

(2.2.2) posterior probability of each gate function in epicycle iteration is calculated:

$h_i^{(k+1)}\left(x\right)=\frac{{g_i}^{\left(k\right)}(x,{v_i}^{\left(k\right)})p_i\left(y\right|x,{W_i}^{\left(k\right)})}{{\mathrm{\Σ}}_{j=1}^K{g_i}^{\left(k\right)}(x,{v_i}^{\left(k\right)})p_j\left(y\right|x,{W_j}^{\left(k\right)})}$

Wherein k is iterative steps, p_i(y | x, W_i ^(k)) and p_j(y | x, W_j ^(k)) represent expert probability distribution, g_i ^(k)(x, v_i ^(k)) represent that the kth of i-th gate function walks iterative value.

(2.2.3) each expert parameter is updated:

${W_i}^{(k+1)}=\frac{{{\mathrm{YH}}_i}^{(k+1)}{X}^T{\mathrm{\μ}}^2}{{{\mathrm{XH}}_i}^{(k+1)}{X}^T{\mathrm{\μ}}^2+{\mathrm{\σ}}^{2}I}$

Wherein k is iterative steps, and X is the vector of all low resolution block x composition in training data, and Y is the vector of all high-resolution block y composition, X in training data^TRepresent the transposition of X, I representation unit matrix, H_i ^(k+1)Represent in kth+1 step the vector of the posterior probability composition of all low resolution block x corresponding to i-th expert.

(2.2.4) each gate function parameter is updated:

$v_i^{(k+1)}=v_i^{\left(k\right)}-(x^{\left(t\right)}-v_i^{\left(k\right)})\underset{t=1}{\overset{N}{\mathrm{\Σ}}}\left(h_i^{\left(k\right)}\right(x^{\left(t\right)})-g_i(x^{\left(t\right)},v_i^{\left(k\right)}\left)\right)$

WhereinRepresent i-th gate function parameter in kth step iteration,The posterior probability of i-th gate function, x in iteration is walked for kth^(t)Represent the t low resolution block.

(2.2.5) output of each gate function in epicycle iteration is calculated:

${g_i}^{(k+1)}(x,{v_i}^{(k+1)})=\exp {(x-v_i^{(k+1)})}^2{\left({\mathrm{\Σ}}_{j=1}^K{\left(x-v_j^{(k+1)}\right)}^2\right)}^{-1}$

(2.2.6) likelihood probability in epicycle iteration is calculated:

$Q^{(k+1)}=\underset{t=1}{\overset{N}{\mathrm{\Σ}}}\underset{i=1}{\overset{K}{\mathrm{\Σ}}}{h}_i^{(k+1)}\left(x\right)\left(\log {g_i}^{(k+1)}\right(x,{v_i}^{(k+1)})+\log p_i(y|x,{W_i}^{(k+1)}\left)p\right({W_i}^{(k+1)}\left)\right)$

Wherein, p_i(y | x, W_i ^(k+1)) represent expert probability distribution, p (W_i ^(k+1)) represent expert parameter probability distribution.

(2.2.7) judge whether iteration restrains.During allowable error ε when the absolute value of difference of likelihood probability of likelihood probability and last round of iteration of epicycle iteration is less than iteration ends, finishing iteration.Otherwise repeated execution of steps (2.2.2)～(2.2.7).

The gate function parameter v obtained when iteration terminates_i, together with expert quantity K, expert parameter W_i, the standard deviation sigma of probability distribution of expert and expert parameter mean of a probability distribution μ, be stored in disk together as final mapping relations model.Wherein, v_i, K, σ, μ be called the gate function parameter of mapping relations model, W_iIt is called the expert parameter of mapping relations model.

(3) low-resolution video that pretreatment is pending

(3.1) pending low-resolution video being split as low resolution video frame, in the present embodiment, films and television programs length is 2000 seconds, and frame rate is 25 frames/second, and gained frame of video adds up to: 2000*25=50000；

(3.2) low resolution video frame of step (3.1) gained is divided into the frame of video block of 10 × 10, as shown in Figure 3；

(4) low-resolution video is mapped as high-resolution video, including:

(4.1) low resolution video frame block step (3) obtained is as the input of gate function, and the gate function parameter in the mapping relations model that use step (2) obtains calculates the output of each gate function:

$g_i(x,v_i)=\exp {(x-v_i)}^2{\left({\mathrm{\Σ}}_{j=1}^K{\left(x-v_j\right)}^2\right)}^{-1}$

Wherein, x is the low resolution video frame block of input.K is the number of expert, v in Mixture of expert model_iRepresent i-th gate function parameter, obtain each through step (2.2).

(4.2) the expert's function parameter corresponding to gate function using output valve maximum calculates corresponding high-resolution block；

(4.2.1) calculating obtains the sequence number of maximum output valve gate function: i=argmax (g_i).Wherein, g_iFor the output of i-th gate function, obtained by step (4.1).

(4.2.2) i-th expert's function is used to calculate high-resolution video frame block:

Y=W_ix

Wherein, W_iFor i-th expert's function parameter, obtained by step (2).X is the low resolution video frame block of input, and it is sized to 10 × 10, and y is the high-resolution video frame block after increase resolution, is sized to 40 × 40.

(4.3) each low resolution block is carried out increase resolution according to the step of (4.1) and (4.2), obtain the high-resolution block of correspondence, all of high-resolution block is spliced into according to position in low resolution video frame of the low resolution block of its correspondence the high-resolution video frame of correspondence；

(4.4), after obtaining the high-resolution video frame that all low resolution video frame are corresponding, it is combined into high-resolution video.

Claims (5)

1. the films and television programs based on increase resolution renovate method, it is characterised in that: include study mapping relations model and carry out increase resolution two parts according to mapping relations model；

Wherein, study mapping relations model includes following two step:

(1) pretreatment training video, including:

(1.1) choose high-resolution video as training sample, and be split as high-resolution video frame；

(1.2) the high-resolution video frame of step (1.1) gained is used gaussian kernel convolution

(1.3) calculate amplification according to original low-resolution video and target high-resolution video, carry out partiting row sampling according to the multiple of gained, obtain the low resolution video frame of correspondence；

(1.4) low resolution video frame of high-resolution video frame and sampling gained is split in bulk respectively, as training data.

(2) mapping relations model is obtained based on Mixture of expert model, including:

(2.1) a Mixture of expert model is initialized；

(2.2) training data using step (1.4) to obtain, is iterated Mixture of expert model optimizing, until iterative process convergence, the model parameter finally given is mapping relations model.Mapping relations model includes gate function parameter and expert parameter.

Carry out increase resolution according to mapping relations model and include following two step:

(3) low-resolution video that pretreatment is pending, including:

(3.1) pending low-resolution video is split as low resolution video frame；

(3.2) low resolution video frame is split in bulk.

(4) the mapping relations model that low-resolution video obtains according to step (2) is promoted to high-resolution video, including:

(4.1) low resolution video frame block step (3) obtained is as the input of Mixture of expert model gate function, and the gate function parameter in the mapping relations model that use step (2) obtains calculates the output of each gate function；

(4.2) parameter of the expert's function corresponding to gate function that use output valve is maximum calculates corresponding high-resolution block, and wherein the parameter of expert's function is obtained by step (2)；

(4.3) each low resolution block is carried out increase resolution according to the step of (4.1) and (4.2), obtain the high-resolution block of correspondence, all of high-resolution block is spliced into according to position in low resolution video frame of the low resolution block of its correspondence the high-resolution video frame of correspondence；

(4.4), after obtaining the high-resolution video frame that all low resolution video frame are corresponding, it is combined into high-resolution video.

2. the films and television programs based on increase resolution according to claim 1 renovate method, it is characterised in that the Mixture of expert model described in step (2.1) includes expert and gate function two parts.

Expert is responsible for data are carried out mapping transformation, and the mapping transformation in the present invention uses linear function as expert's function:

Y=Wx

Wherein W is expert parameter, and x and y represents low resolution video frame block and corresponding high-resolution video frame block respectively；

Gate function is responsible for determining to select which expert that data are converted, and in the present invention, i-th gate function is expressed as:

Wherein, x and y represents low resolution video frame block and corresponding high-resolution video frame block, v respectively_iRepresent i-th gate function parameter, v_jRepresenting jth gate function parameter, K is the number of expert in Mixture of expert model.

3. the films and television programs based on increase resolution according to claim 1 renovate method, it is characterised in that: the described initialization Mixture of expert model of step (2.1) comprises the following steps:

1. the quantity K of expert is specified；

2. the probability distribution Gaussian distributed of each expert is supposed: p (y | x, W_i)=N (y (x, W_i), σ), wherein W_iRepresenting the parameter of i-th expert, σ is the standard deviation of Gauss distribution.Assuming that parameter W_iDistribution also Gaussian distributed: p (W_i)=N (0, μ), wherein μ represents the average of Gauss distribution.

3. k-mean algorithm is adopted to be clustered according to the quantity K of expert by training data, the initial value W of the parameter of each expert_i ⁽⁰⁾It is appointed as in class slope, the initial value v of each gate function parameter_i ⁽⁰⁾It is appointed as cluster centre；

4. the initial value of each gate function is calculated:

Wherein x represents low resolution video frame block, v_i ⁽⁰⁾Representing the initial value of i-th gate function parameter, K is the number of expert in Mixture of expert model, i.e. the number of leaf node in tree structure.

4. the films and television programs based on increase resolution according to claim 1 renovate method, it is characterised in that: step (2.2) is described to be iterated optimization to model and comprises the following steps:

1. allowable error ε during iteration ends is specified；

2. the posterior probability of each gate function in epicycle iteration is calculated:

Wherein k is iterative steps, p_i(y | x, W_i ^(k)) and p_j(y | x, W_j ^(k)) represent expert probability distribution, g_i ^(k)(x, v_i ^(k)) represent that the kth of i-th gate function walks iterative value.

3. each expert parameter is updated:

Wherein k is iterative steps, and X is the vector of all low resolution block x composition in training data, and Y is the vector of all high-resolution block y composition, X in training data^TRepresent the transposition of X, I representation unit matrix, H_i ^(k+1)Represent in kth+1 step the vector of the posterior probability composition of all low resolution block x corresponding to i-th expert.

4. each gate function parameter is updated:

WhereinRepresent i-th gate function parameter in kth step iteration,The posterior probability of i-th gate function, x in iteration is walked for kth^(t)Represent the t low resolution block.

5. the output of each gate function in epicycle iteration is calculated:

6. the likelihood probability in epicycle iteration is calculated:

Wherein, p_i(y | x, W_i ^(k+1)) represent expert probability distribution, p (W_i ^(k+1)) represent expert parameter probability distribution.

7. judge whether iteration restrains.During allowable error ε when the absolute value of difference of likelihood probability of likelihood probability and last round of iteration of epicycle iteration is less than iteration ends, finishing iteration.Otherwise repeated execution of steps 2.～7..

The gate function parameter v obtained when iteration terminates_i, together with expert quantity K, expert parameter W_i, the standard deviation sigma of probability distribution of expert and expert parameter mean of a probability distribution μ, be stored in disk together as final mapping relations model.

5. the films and television programs based on increase resolution according to claim 1 renovate method, it is characterised in that: the parameter of the expert's function corresponding to gate function that step (4.2) described use output valve is maximum calculates corresponding high-resolution block and comprises the following steps:

1. calculating obtains the sequence number of maximum output valve gate function: i=argmax (g_i)

Wherein, g_iFor the output of i-th gate function, obtained by step (4.1).

2. i-th expert's function is used to calculate high-resolution video frame block: y=W_ix

Wherein, W_iFor the parameter of i-th expert's function, y is the high-resolution video frame block corresponding to low resolution video frame block x of input.