CN103985114A - Surveillance video person foreground segmentation and classification method - Google Patents

Abstract

The invention discloses a surveillance video person foreground segmentation and classification method. The method comprises the following steps: 1) the person foreground of a surveillance video is extracted, the foreground and the background are segmented by adopting a Gaussian mixture model method, and the foreground person is represented by a bounding box to form a foreground person short video; 2) foreground features are extracted, key frames are extracted for the short video, frames having a moderate foreground occupied area ratio and complete foreground shape and color are selected as the key frames, and a plurality of features are extracted for the key frames; 3) and feature fusion and classification are performed, the non-person foregrounds, such as a car, etc., are classified according to the features, subspaces are studies for person features by adopting the canonical correlation coefficient feature fusion method, the features are projected to the subspace having better class features, and different clustering methods are adopted for the projected features so as to classify foreground persons having similar shape and color into a same class.

Description

A kind of method of monitor video personage foreground segmentation and classification

Technical field

A kind of method that the present invention relates to monitor video personage foreground segmentation and classification, belongs to the fields such as computer video, machine learning techniques.

Background technology

Bringing people, the while has also brought some potential safety hazards to the modern life easily, in order to eliminate these hidden danger, taked many kinds of measures, the monitor video in each corner is exactly a kind of in many measures, but when there is unsafe incidents, in the face of the huge monitor video of quantity, supervisory personnel often needs to spend long time and searches dangerous target, affected the efficiency of eliminating unsafe incidents, there are now some means to shorten monitor video from time and two, space angle, shorten the proportion of the nonsensical shared video of part, although do like this, the time of browsing nonsensical video can effectively be reduced, but or need in a plurality of monitor videos, pick out target, and at present much to the research of foreground classification mainly around be that the kind of prospect is classified, and various hidden dangers are caused by people under normal circumstances, such as prospect being divided into various vehicles and personage, be divided into plant, animal and building etc., seldom there is pair research that personage's prospect is classified, what the research of prospect kind classification mainly adopted is measure of supervision, after conversion scene, conventionally also needing to gather the training of new scene data expends larger, and unsupervised personage's foreground classification will shorten hunting zone effectively, the time that greatly reduces checking monitoring video increases work efficiency, the sort research of monitor video foreground people becomes relevant studying a question.

Traditional monitor video foreground classification mainly adopts the method for supervised learning to divide the kind of prospect, during the monitoring scene that the scope that is suitable for is larger is applied, but the monitoring scene classification that is mainly people for prospect among a small circle seldom has correlative study, and supervised learning need to carry out a large amount of study preparation work larger, adopted in the present invention a plurality of features of cutting apart monitor video prospect and background and extracting key frame and supervised canonical correlation coefficient fusion feature by nothing, adopt lowest rank clustering method, provide the classification results of video.

Summary of the invention

Goal of the invention: technical matters to be solved by this invention is the deficiency for existing research, provides a kind of classification and dividing method of monitor video personage prospect, thereby improves the efficiency of inspection monitor video.

Technical scheme: the invention discloses a kind of method of monitor video personage foreground segmentation and classification, the method is characterized in that the video that can browse in the short time in the different scenes of same personage, specifically comprise the following steps:

1, the prospect of monitoring of separation video and background: generally speaking the prospect of monitor video refers to movable things be people or car conventionally, and the background of monitor video refer to scenery static in video.Adopt mixed Gauss model by the prospect of monitor video and background separation out, and each prospect is surrounded with the minimum bounding box that can surround prospect completely, form independently prospect small video.The detailed step of step 1 prospect and background segment is as follows:

Step 1-1, initialization Gauss model: read the first two field picture of video, for each pixel in image builds a mixed Gauss model that contains K Gauss model, K span 3～5, pixel j is at moment t value x _jprobability P (x _j) can be represented by this K Gauss model: wherein represent the constantly weight of i gaussian component in the mixed Gauss model of pixel j of t, satisfied: with represent t average and the covariance of i the gaussian component of pixel j constantly, represent Gaussian probability-density function, be expressed as follows:

$N(x_j,u_{j,t}^i,{\mathrm{\Σ}}_{j,t}^i)=\frac{1}{{2\mathrm{\π}}^{\frac{d}{2}}{\left|{\mathrm{\Σ}}_{j,t}^i\right|}^{\frac{1}{2}}}\exp [-\frac{1}{2}{(x_j-u_{j,t}^i)}^T{\left({\mathrm{\Σ}}_{j,t}^i\right)}^{-1}(x_j-u_{j,t}^i)],$

Wherein d is x _jdimension, for RGB color space, each pixel has 3 passages, x _jfor tri-vector, covariance matrix wherein wherein be illustrated in t i the Gauss model variance of pixel j constantly, during initialization value is i represents unit matrix, initial phase, the weight of each Gaussian distribution ${\mathrm{\σ}}_{\mathrm{init}}^2=900;$

Step 1-2, upgrade Gauss model: continue to read monitor video, often read a two field picture of monitor video and just mixed Gauss model is done and upgraded, by each gauss component in mixed Gauss model according to descending sequence, the pixel value x of the current new frame reading _{j, t+1}if meet following formula with i Gauss model in mixed Gauss model:

$|x_{j,t+1}-u_{j,t}^i|\≤\mathrm{\δ}*{\mathrm{\σ}}_{j,t}^i$

Upgrade i gauss component, all the other gauss components remain unchanged, and judge pixel x _{j, t+1}in present frame, be background parts pixel, parameter δ is matching threshold, δ span 1～2, and the computing method of upgrading i gauss component are as follows:

${\mathrm{\ω}}_{j,t+1}^i=(1-\mathrm{\α}){\mathrm{\ω}}_{j,t}^i+\mathrm{\α}$

$u_{j,t+1}^i=(1-\mathrm{\ρ})u_{j,t}^i+{\mathrm{\ρx}}_j$

${\left({\mathrm{\σ}}_{j,t+1}^i\right)}^2=(1-\mathrm{\ρ}){\left({\mathrm{\σ}}_{j,t}^i\right)}^2+\mathrm{\ρ}{(x_j-u_{j,t}^i)}^T(x_j-u_{j,t}^i)$

$\mathrm{\ρ}=\frac{\mathrm{\α}}{{\mathrm{\ω}}_{j,t}^i}$

Wherein α is the learning rate of gauss hybrid models, span 0～1, the learning rate that ρ is parameter alpha; If pixel x _{j, t+1}all unmatched with K gauss component, judge that this pixel is the foreground pixel of present frame, to construct new gauss component and replace the gauss component after sequence is leaned on, the average of new gauss component is made as pixel x _{j, t+1}value, standard deviation and weight are set to respectively σ _initand ω _init, average and the variance of the gauss component of reservation remain unchanged, and weight is upgraded according to the following formula:

${\mathrm{\ω}}_{j,t+1}^i=(1-\mathrm{\α}){\mathrm{\ω}}_{j,t}^i$

Step 1-3, completes video foreground and background segment work: pixel x _{j, t+1}k gauss component undated parameter after, weight to K gauss component is normalized, and repeats preceding step 1-1 and 1-2, retains the foreground pixel in every two field picture, until monitor video reads end, obtain the video of non-display background with the display foreground of the same resolution of original monitor video;

Step 1-4, extract the minimum bounding box that surrounds foreground people video: the monitor video obtaining in read step 1-3, every two field picture is first carried out to dilation and erosion operation, thereby the noise in removal of images, progressive scanning picture again, in document image, pixel value is not long l and the wide w of the rectangle that forms of 0 pixel, because the video background pixel value that preceding step 1-2 obtains is 0, so this pixel that represents that pixel value is non-zero is prospect, bounding box for same each frame of personage's prospect has long l and wide w, select l and w the longest in all frames, as the bounding box of this personage's prospect, obtain thus surrounding the small video of personage's prospect video.

2, the feature of extraction prospect small video: prospect small video is extracted to one group of key frame, considering that monitor video moves mainly comprises people and vehicle in prospect, and fundamental purpose of the present invention is in order to carry out personage's classification, thereby each prospect is recorded to area and two features of translational speed, before carrying out human classification first by automobile foreground classification out, personage's profile is very important for distinguishing different personages with colouring information, so after carrying out dilation and erosion operation, to each personage's prospect key-frame extraction color histogram feature, the feature of local two value tags and these three relevant CFs of word bag.Step 2 foreground people feature extraction detailed step is as follows:

Step 2-1, the key frame of extraction personage prospect: fixedly choose the middle F frame of personage's video f ₁, f ₂..., f _fimage is as key frame, and F is desirable 20～40, and in the middle of choosing, F frame is because a framing is than start frame and the more complete profile that has represented personage and the color of end frame in the middle of personage's small video, and personage's prospect to account for the size of video moderate;

Step 2-2, extracts color histogram information: to F frame f ₁, f ₂..., f _fcolor characteristic histogram is extracted in personage subregion, establishes the total m of column subregion of color histogram _cindividual, computed image f _ithe column subregion id that tri-Color Channel RGB values of middle pixel p are corresponding, i gets 1～F, and R represents red color channel value, and G represents green channel value, and B represents blue channel value, formula is as follows:

$\mathrm{id}=\frac{R}{256}+\frac{G}{256}{m_c}^{\frac{2}{3}}+\frac{B}{256}{m_c}^{\frac{1}{3}},$

The number of adding up pixel in each column subregion id obtains f _icolor histogram, it is m that color histogram is finally expressed as length _cvectorial υ _c, all key frames are repeated to this step and obtain m _cthe matrix M of * F ₁;

Step 2-3, extracting local two-value is Local Binary Pattern feature, is called for short LBP feature: calculate F frame f ₁, f ₂..., f _npart two value tags of image, first by image f _igray processing, the radius of the two-value LBP of the portion operator of setting a trap is r, and r gets 3 or 4 or 5, with the window of r*r, in image, moves, and location of pixels of every movement just calculates window center pixel p one time _centerlBP value, computing method are as follows: will with center pixel p _centeran adjacent r*r pixel respectively with center pixel p _centervalue compare, adjacent pixel values is greater than center pixel p _centerthe position of this pixel is marked as 1, otherwise is marked as 0, obtains thus r*r-1 bit, final window has obtained the local two-value LBP feature of whole image while moving to last center pixel position, then by the local two-value LBP feature histogram graph representation of image; The two-value LBP of the portion feature of setting a trap histogram subregion has m _lindividual, the height value of each component of histogram is together in series and obtains final part two value tags: length is m _lvectorial υ ₁, all key frames are repeated to this step, until obtain m _lthe matrix M of * F ₂;

Step 2-4, extracting word bag is bag of words feature, is called for short BOW feature: first calculate F frame f ₁, f ₂..., f _fdimension rotation unchangeability sift unique point (articles of reference: object recognition from local scale-invariant features), establishing word list length in word bag bag of words model is m _b, adopting K-means clustering method, K-means cluster centre is 64, and the close sift unique point of the meaning of a word is merged and obtains m _bindividual class, class center forms the word list of word bag BOW, with cluster centre, replaces each sift unique point in each frame, adds up the corresponding sift unique point of each vocabulary of all words number, finally obtains image f _ithe frequency of each vocabulary, length is m _bvectorial υ _b.All key frames are repeated to this step, until obtain m _b* F matrix M ₃;

Step 2-5, extracts area and velocity characteristic.Calculate F frame f ₁, f ₂..., f _fthe area s of the prospect of each two field picture in image ₁, s ₂..., s _fwith speed υ ₁, υ ₂..., υ _f-1the area of prospect is also number of pixels non-zero in prospect, get the mean value of F display foreground area as the area value s of this prospect, the displacement of the center that prospect speed is surrounded frame by the rectangle of prospect in original monitor video determines, F two field picture calculates F-1 speed, gets the intermediate value of speed as the speed v of this prospect.

3, Fusion Features and classification: first by the separation vehicle in prospect out, it is automobile and personage that the prospect occurring in general monitor video is divided into two large classes conventionally, on the interlude axle of the prospect video of same shot record, one group of image has good robustness to the transparent effect of camera lens, in the middle of personage's prospect video, personage's foreground area of a framing is far smaller than the area of automobile prospect in the middle framing of automobile prospect video conventionally, the speed of personage's prospect is generally also much smaller than the translational speed of automobile prospect, each foreground area obtaining and speed are sorted out to automobile according to the threshold value of setting, the personage's foreground extraction color histogram feature, local two value tags and the word bag feature that again classification are obtained, adopt again the method for canonical correlation coefficient to carry out nothing supervision Fusion Features, obtain one and distinguish different classes of space T, three eigenmatrixes are projected to space T, color characteristic after projection is done to lowest rank subspace clustering, LBP after projection and BOW feature are done to K-means cluster, according to cluster result, foreground people small video is classified, it is as follows that the foreground people of step 3 monitor video is classifyed step in detail:

Step 3-1; the threshold value of foreground area and speed is set; generally the speed of automobile wants large with corresponding speed and the area numerical value of area numeric ratio personage prospect; and foreground object track under shot record or draw near or from the close-by examples to those far off; it is less that middle image size is generally subject to transparent effect impact, here area threshold area _thresh=800pixel, threshold speed speed _thresh=25pixel/image, pixel represents pixel, image presentation video, the foreground partition that area surpasses area threshold is class of vehicles, when area features does not surpass area threshold, the threshold value if prospect velocity characteristic outpaces, foreground partition is class of vehicles otherwise is divided into personage's classification;

Step 3-2, uniform data dimension: the color histogram matrix m for F image that step 2 is obtained _c* F, LBP eigenmatrix m _l* F and BOW eigenmatrix m _b* F, call principal component analysis (PCA) Principal Component Analysis(articles of reference: On Lines and Planes of Closest Fit to Systems of Points in Space), be reduced to unified dimension m, all eigenvectors matrixs become m * F;

Step 3-3, Fusion Features: establishing and having matrix T dimension is m * n, n is determined by matrix A below, three eigenvectors matrix M ₁, M ₂, M ₃project in space, matrix T place, can present the projector distance of similar space vector in the T of space very near, and the far characteristic of the projector distance of inhomogeneous space vector in the T of space, initialization T is vector of unit length, the content of the renewal matrix T of iteration, concrete iterative process is as follows:

3-3-1. is to matrix M ₁, M ₂, M ₃carry out the positive triangle decomposition of matrix and upgrade matrix $M_i:T^T{M}_i={\mathrm{\φ\Δ}}_i,M_i^{\′}=M_i{\mathrm{\Δ}}_i^{-1},$ I gets 1～3;

3-3-2. is to every a pair of M ' _i, M ' _jcarry out Singular Value Decomposition Using:

3-3-3. solution matrix T, compute matrix $A={\mathrm{\Σ}}_{k_1=1}^3{\mathrm{\Σ}}_{k_2=1}^3(M_{k_1}^{\′}{Q}_{k_1{k}_2}-M_{k_2}^{\′}{Q}_{k_2})(M_{k_1}^{\′}{Q}_{k_1{k}_2}-$ ${M_{k_2}^{\′}{Q}_{k_2})}^T$ The proper vector of compute matrix A λ is the proper vector t of matrix A _ieigenwert, by t _iaccording to order sequence formation matrix T, i.e. T={t from big to small ₁, t ₂..., t _n, the different characteristic of matrix A vector number has been determined the size of n here;

Repeating step 3-3-1～3-3-3 is until T convergence repeats above-mentioned steps 3-5 submatrix T and can restrain, and wherein i span 1～3, T ^tthe transposed matrix of representing matrix T, M ' _irepresent M _iinverse matrix, represent M _ireverse put, the orthogonal matrix after the positive triangle decomposition of φ representing matrix, Δ _ithe upper triangular matrix after the positive triangle decomposition of matrix, represent Δ _iinverse matrix, Q _ijthe unitary matrix of representing matrix svd;

Step 3-4, prospect visual classification: by eigenvectors matrix M ₁, M ₂, M ₃project in the space at T place, i.e. M _i=T ^tm _i, i gets 1～3, obtains new eigenvectors matrix M ₁, M ₂, M ₃;

Step 3-5, color histogram feature clustering: color matrix M ₁adopt lowest rank Subspace clustering method, the color histogram of different prospects is often presented on different data dimensions and distance in K-means method (articles of reference A K-means Clustering Algorithm) generally adopts Euclidean distance, be not suitable for the distance of color space, so adopt Subspace clustering method can realize preferably category division.Utilize lowest rank method to calculate the similarity w between every two field picture, structural map image is using all foreground images as node, similarity w between image is as weight, adopt again spectral clustering Ncut method (articles of reference: Normalized Cuts and Image Segmentation) figure image is cut apart, thereby complete the classification to image, the computing method of similarity w are as follows:

3-5-1, initiation parameter λ ₀, correlation matrix Z, the equivalent matrice J=0 of correlation matrix Z, Z=J, noise is corrected matrix E=0, Lagrangian matrix Y ₁=0, Y ₂=0, Lagrange punishment parameter μ=10 ^-6, maximum Lagrange punishment parameter m ax _μ=10 ¹⁰, Lagrange punishment parameter multiple ρ ₀=1.1, constant ε=10 ^-8;

3-5-2, calculates M ₁the correlation matrix equivalent matrice J of every column data: fix other matrix update matrix J, $J=\arg \min \frac{1}{\mathrm{\μ}}{\left|\right|J\left|\right|}_{*}+\frac{1}{2}{\left|\right|J-(Z+Y_2/\mathrm{\μ})\left|\right|}_F^2;$

3-5-3, calculates M ₁the correlation matrix Z of every column data: fix other matrix update matrixes Z, Z=(I+M ₁ ^tm ₁) ^-1(M ₁ ^tm ₁-M ₁ ^te+J+ (M ₁ ^ty ₁-Y ₂)/μ);

3-5-4, calculating noise is corrected matrix E: fix other matrix update matrixes E, $E+\arg \min \frac{{\mathrm{\λ}}_0}{\mathrm{\μ}}{\left|\right|E\left|\right|}_{\mathrm{2,1}}+$ $\frac{1}{2}{\left|\right|E-(M_1-M_{1}Z+Y_1/\mathrm{\μ})\left|\right|}_F^2;$

3-5-5, calculates Lagrangian matrix Y ₁, Y ₂: Y ₁, Y ₂, Y ₁=Y ₁+ μ (M ₁-M ₁z-E), Y ₂=Y ₂+ μ (Z-J);

3-5-6, upgrades Lagrange punishment parameter μ: μ=min (ρ ₀μ, max _μ);

3-5-7, judges whether iteration finishes: check || M ₁-M ₁z-E|| _∞< ε, || Z-J|| _∞whether < ε sets up, and finishes, otherwise continue iteration if set up iteration;

Wherein || || _*represent nuclear norm, || || _frepresent this norm of Fu Luo Benny crow, || || _∞represent maximum norm, smaller value in A and B is returned in min (A, B) representative, and above-mentioned iterative process obtains matrix Z, the element Z in matrix Z _i,j, Z _j,isimilar value between sum representative image i, j, build non-directed graph image, the node of image i representative graph image, similarity between image i, j represents the weight between node i and node j, thereby adopts spectral clustering Ncut method that figure image is cut apart and realized the classification between a plurality of prospect key frames.

Step 3-6, LBP and BOW feature clustering: LBP and the difference of BOW characteristic on Spatial Dimension are very little, directly adopt K-means method just can obtain good result, to matrix M ₂, M ₃adopt K-means method to carry out cluster, the personage's prospect in this method is generally 2-3 class;

Step 3-7, carries out integrated study to result: by step 3-5,3-6,3-7 obtains the classification C under each two field picture under three features _i, utilize three classification information ballots to determine the classification of each two field picture, as C ₁, C ₂, C ₃for image f _iclassification be respectively classification that 0,0,1 occurrence number is the highest 0 for image f _iclassification, thereby all key frame f _ican determine its classification information.Calculate personage's prospect video υ _iwith υ _jbetween similarity for prospect video υ _iif, its same υ _jbetween similarity than high with other video similarity, υ _isame υ _jbe divided into same class.

Accompanying drawing explanation

Fig. 1 is the basic flow sheet of the inventive method.

Fig. 2 is original monitor video 1 part picture.

Fig. 3 is original monitor video 2 part pictures.

Fig. 4 is original monitor video 3 part pictures.

Fig. 5 is monitor video 1 part prospect.

Fig. 6 is monitor video 2 part prospects.

Fig. 7 is monitor video 3 part prospects.

Fig. 8 is the extraction schematic diagram of LBP feature.

Fig. 9 is the schematic diagram of BOW clustering method.

Figure 10 is the subspace schematic diagram at two object prospect places in monitor video 1.

The principle schematic of Figure 11 Canonical correlation Fusion Features.

In Figure 12 this method, human classification precision is with the personage's clustering precision comparison after not merging.

Embodiment

Below in conjunction with the drawings and specific embodiments, the present invention is done further and illustrated.

The process flow diagram of this method as shown in Figure 1, is divided into three large processes: be first that employing mixed Gauss model is cut apart prospect to each monitor video and background obtains prospect video; Next be prospect video personage is separated and to one group of key frame of each personage's video extraction for video feature extraction; Key-frame extraction color histogram to each foreground people again, local two value tags, word bag feature, then these three features are carried out to Fusion Features with canonical correlation coefficient, construct one new can the different classes of space of better distinguish, again these three eigenmatrixes are projected in this space with better distinguish and carry out cluster, color matrix after projection is adopted to lowest rank subspace clustering, local two-value feature and word bag feature are adopted to K-means cluster, finally adopt integrated learning approach to obtain the similarity between prospect.

Specifically, as shown in Figure 1, the invention discloses a kind of monitor video foreground people and cut apart and sorting technique, mainly comprise following step:

Step 1, the prospect of monitor video and background segment: generally speaking the prospect of monitor video refers to movable things, be people or car conventionally, and the background of monitor video refers to scenery static in video.Adopt prospect and the background of mixed Gauss model monitoring of separation video, and each foreground people is surrounded with the minimum bounding box that can surround foreground people completely, form independently foreground people small video;

Step 2, extract the feature of foreground people small video: foreground people small video is extracted to one group of key frame, considering that monitor video moves mainly comprises people and vehicle in prospect, and fundamental purpose of the present invention is personage's classification, so each prospect is recorded to two features of Area and Speed, before carrying out human classification first by automobile foreground classification out, personage's profile is very important for distinguishing different personages with colouring information, so after carrying out dilation and erosion operation, to each personage's key-frame extraction color histogram feature, the feature of local two value tags and these three relevant CFs of word bag,

Step 3, Fusion Features and classification: first by the separation vehicle in prospect out, it is automobile and personage that the prospect occurring in general monitor video is generally two large classes, on the interlude axle of the prospect video of same shot record, one group of image has good robustness to the transparent effect of camera lens, in the middle of personage's prospect, personage's area of a framing is far smaller than the automobile area of the middle framing of automobile prospect conventionally, the speed of personage's prospect is generally also much smaller than the translational speed of automobile prospect, to each foreground area and the speed that obtain, according to the threshold value of setting, sort out automobile, the personage's foreground extraction color histogram feature, local two value tags and the word bag feature that again classification are obtained, adopt again the method for canonical correlation coefficient to carry out nothing supervision Fusion Features, obtain one and distinguish different classes of space T, three Projection Characters are arrived to space T, color characteristic after projection is done to lowest rank subspace clustering, LBP after projection and BOW feature are done to K-means cluster, according to cluster result, foreground people small video is classified,

Step 1, the detailed step of prospect and background segment is as follows:

Step 1-1, initialization Gauss model: the first two field picture that reads video is that in image, each one of pixel structure contains K, in invention, K is the mixed Gauss model of 3, with K Gauss model, represents in monitor video in every two field picture that each pixel j is at the value x of moment t _j, pixel j is at moment t value x _jprobability P (x _j) can be represented by this K Gauss model: $P\left(x_j\right)={\mathrm{\&Sigma;}}_{i=1}^K{\mathrm{\&omega;}}_{j,t}^i*N(x_j,u_{j,t}^i,{\mathrm{\&Sigma;}}_{j,t}^i),$ Wherein represent the constantly weight of i gaussian component in the mixed Gauss model of pixel j of t, satisfied: with represent t average and the covariance of i the gaussian component of pixel j constantly, represent Gaussian probability-density function, be expressed as follows:

$N(x_j,u_{j,t}^i,{\mathrm{\&Sigma;}}_{j,t}^i)=\frac{1}{{2\mathrm{\&pi;}}^{\frac{d}{2}}{\left|{\mathrm{\&Sigma;}}_{j,t}^i\right|}^{\frac{1}{2}}}\exp [-\frac{1}{2}{(x_j-u_{j,t}^i)}^T{\left({\mathrm{\&Sigma;}}_{j,t}^i\right)}^{-1}(x_j-u_{j,t}^i)],$

Wherein d is x _jdimension, for RGB color space, each pixel has 3 passage x _jfor tri-vector, covariance matrix wherein wherein represent t i the Gauss model variance of pixel j constantly, during initialization value be i represents unit matrix.Initial phase, the variance of each Gaussian distribution the weight of each Gaussian distribution is got ω _init=1/K, ω in invention _initvalue is 0.3;

Step 1-2, upgrades Gauss model: continue to read monitor video, often read a two field picture of monitor video and just mixed Gauss model is done and upgraded; By each gauss component in mixed Gauss model according to descending sequence, the pixel value x of the current new frame reading _{j, t+1}if meet following formula with i Gauss model in mixed Gauss model:

$|x_{j,t+1}-u_{j,t}^i|\&le;\mathrm{\&delta;}*{\mathrm{\&sigma;}}_{j,t}^i$

Upgrade i gauss component, all the other gauss components remain unchanged, judgement pixel x _{j, t+1}in present frame, be background parts pixel, parameter δ is matching threshold, δ span 1～2, and in invention, δ gets 1.5, and the computing method of upgrading i gauss component are as follows:

${\mathrm{\&omega;}}_{j,t+1}^i=(1-\mathrm{\&alpha;}){\mathrm{\&omega;}}_{j,t}^i+\mathrm{\&alpha;}$

$u_{j,t+1}^i=(1-\mathrm{\&rho;})u_{j,t}^i+{\mathrm{\&rho;x}}_j$

${\left({\mathrm{\&sigma;}}_{j,t+1}^i\right)}^2=(1-\mathrm{\&rho;}){\left({\mathrm{\&sigma;}}_{j,t}^i\right)}^2+\mathrm{\&rho;}{(x_j-u_{j,t}^i)}^T(x_j-u_{j,t}^i)$

$\mathrm{\&rho;}=\frac{\mathrm{\&alpha;}}{{\mathrm{\&omega;}}_{j,t}^i}$

Wherein α is the learning rate of mixed Gauss model, and α span 0～1 is got the learning rate that 1, ρ is parameter in α invention; If pixel x _{j, t+1}all unmatched with K gauss component, judge that this pixel is the foreground pixel of present frame, to construct new gauss component and replace the gauss component after sequence is leaned on, the average of new gauss component is made as x _{j, t+1}value, standard deviation and weight are set to respectively σ _initand ω _init, average and the variance of the gauss component of reservation remain unchanged, and weight is upgraded according to the following formula:

${\mathrm{\&omega;}}_{j,t+1}^i=(1-\mathrm{\&alpha;}){\mathrm{\&omega;}}_{j,t}^i$

Step 1-3, completes prospect and background segment work: the pixel x of video _{j, t+1}k gauss component undated parameter after, weight to K gauss component is normalized, repeat preceding step 1-1 and 1-2 and retain the foreground pixel in every two field picture, until monitor video reads end, obtain the video of non-display background with the display foreground of the same resolution of original monitor video;

1-4, extract the minimum bounding box that surrounds foreground people video: the monitor video obtaining in read step 1-3, every two field picture is first carried out to dilation and erosion operation, thereby the noise in removal of images, progressive scanning picture again, in document image, pixel value is not long l and the wide w of the rectangle that forms of 0 pixel, because the video background pixel value that step 1-2 obtains is 0, so this pixel that represents that pixel value is non-zero is prospect, bounding box for same each frame of personage's prospect has long l and wide w, select the longest l in all frames, w obtains surrounding the small video of personage's prospect video thus as the bounding box of this personage's prospect, Fig. 2～4th, original monitor video, Fig. 5～7th, corresponding to the monitor video after the employing mixed Gauss model extraction prospect of Fig. 2～4.

Step 2, foreground people feature extraction detailed step is as follows:

Step 2-1, the key frame of extraction personage prospect: fixedly choose the middle F frame of personage's video f ₁, f ₂..., f _fimage is as key frame, and F is desirable 20～40, and in invention, F gets 20, and in the middle of choosing, F frame is because a framing is than start frame and the more complete profile that has represented personage and the color of end frame in the middle of personage's small video, and personage's prospect to account for the size of video moderate;

Step 2-2, extracts color histogram information: to F frame f ₁, f ₂..., f _fcolor characteristic histogram is extracted in personage subregion, establishes the total m of column subregion of color histogram _cindividual, m in invention _cget 64, computed image f _ithe column subregion id that tri-Color Channel rgb values of middle pixel p are corresponding, i gets 1～F, and R represents red color channel value, and G represents green channel value, and B represents blue channel value, formula is as follows:

$\mathrm{id}=\frac{R}{256}+\frac{G}{256}{m_c}^{\frac{2}{3}}+\frac{B}{256}{m_c}^{\frac{1}{3}}$

The number of adding up pixel in each column subregion id obtains f _icolor histogram, it is m that color histogram is finally expressed as length _cvectorial υ _c; To all these steps of key frame repetitive operation, until obtain m _cthe matrix M of * F ₁;

Step 2-3, extracting local two-value is Local Binary Pattern feature, is called for short LBP feature, calculates F frame f ₁, f ₂..., f _npart two value tags of image, first by image f _igray processing, the radius of establishing LBP operator is r, and in invention, r gets 3, with the window of r*r, in image, moves, and location of pixels of every movement just calculates window center pixel p one time _centerlBP value, computing method are as follows: will with center pixel p _centeran adjacent r*r pixel respectively with center pixel p _centervalue compare, adjacent pixel values is greater than center pixel p _center, the position of this pixel is marked as 1, otherwise is marked as 0, as Fig. 8 represents, obtain thus r*r-1 bit, final window has obtained the LBP feature of whole image while moving to last center pixel position, then by the LBP feature histogram graph representation of image.If LBP histogram subregion has m _lindividual, m in invention _lvalue 64, is together in series the height value of each component of histogram, obtains final part two value tags: length is m _lvectorial υ _l.All key frames are repeated to this step, until obtain m _lthe matrix M of * F ₂;

Step 2-4, extracting word bag is bag of words feature, is called for short BOW feature: first calculate F frame f ₁, f ₂..., f _fdimension rotation unchangeability sift unique point, establishing word list length in BOW model is m _b, m in invention _bget 64, adopt K-means method, in K-means, cluster centre is made as 64, and the close sift unique point of the meaning of a word is merged and obtains m _bindividual class, class center forms the word list of BOW, and in invention, word list length is 64, replaces each the sift unique point in each two field picture, as m in Fig. 9 with the vocabulary in word list _bvalue be after 3, K-means cluster, to obtain 3 cluster centres, more again with the vocabulary in word list, replace each the sift unique point in each two field picture, unique point sift ₁distance-like m ₁recently, m ₁central point representation feature point sift ₁, add up the corresponding sift unique point of each vocabulary of all word lists number, obtain image f _ithe frequency of each vocabulary be that length is m _bvectorial υ _b, the operation that all key frames is repeated to 2-4 obtains m _b* F matrix M ₃;

Step 2-5, extracts area and velocity characteristic.Calculate F frame f ₁, f ₂..., f _fthe area s of the prospect of each two field picture in image ₁, s ₂..., s _fwith speed υ ₁, υ ₂..., υ _f-1the area of prospect is also number of pixels non-zero in prospect, get the mean value of F display foreground area as the area value s of this prospect, the displacement of the center that prospect speed is surrounded frame by the rectangle of prospect in original monitor video determines, F two field picture calculates F-1 speed, gets the intermediate value of speed as the speed v of this prospect.

Step 3, it is as follows that the foreground people of monitor video is classifyed step in detail:

Step 3-1; the threshold value that foreground area and speed are set sorts out automobile; generally the speed of automobile speed and the area numerical value corresponding with area numeric ratio personage prospect wants large; and foreground object track under shot record or draw near or from the close-by examples to those far off; it is less that middle image size is generally subject to transparent effect impact, here area threshold area _thresh=800pixel, threshold speed speed _thresh=25pixel/image, pixel represents pixel, image presentation video; The foreground partition that area surpasses area threshold is class of vehicles, when area features does not surpass area threshold, and the threshold value if prospect velocity characteristic outpaces, foreground partition is class of vehicles otherwise is divided into personage's classification;

Step 3-2, uniform data dimension: the color histogram matrix m for F image that step 2 is obtained _c* F, LBP eigenmatrix m _l* F and BOW eigenmatrix m _b* F, eigenmatrix is 64 * 20, calls principal component analytical method and is reduced to unified dimension m, and in invention, establishing the maximum major component size of reservation is 64, m value 64 in inventing, this all eigenvectors matrix just becomes m * F;

Step 3-3, Fusion Features: establishing and having matrix T dimension is m * n, in invention, T size is 64 * 64, three eigenvectors matrix M ₁, M ₂, M ₃project in space, matrix T place, can present space vector P generic in Figure 11 ₁, P ₂projector distance in the T of space is very near, different classes of space vector P ₁, P ₃the far characteristic initialization T of projector distance in the T of space is vector of unit length, the content of the renewal matrix T of iteration, and concrete iterative process is as follows:

3-3-1, to matrix M ₁, M ₂, M ₃carry out the positive triangle decomposition of matrix and upgrade matrix M _i: $T^T{M}_i=$ ${\mathrm{\&phi;\&Delta;}}_i,M_i^{\&prime;}=M_i{\mathrm{\&Delta;}}_i^{-1},i=1~3;$

3-3-2, to every a pair of M ' _i, M ' _jcarry out Singular Value Decomposition Using: i=1～3;

3-3-3, solution matrix T, compute matrix $A={\mathrm{\&Sigma;}}_{k_1=1}^3{\mathrm{\&Sigma;}}_{k_2=1}^3(M_{k_1}^{\&prime;}{Q}_{k_1{k}_2}-M_{k_2}^{\&prime;}{Q}_{k_2})(M_{k_1}^{\&prime;}{Q}_{k_1{k}_2}-$ ${M_{k_2}^{\&prime;}{Q}_{k_2})}^T$ The proper vector of compute matrix A λ is proper vector t in matrix A _ieigenwert, by t _iaccording to order sequence formation matrix T, i.e. T={t from big to small ₁, t ₂..., t _n, the proper vector number that matrix A is different is here determined n size;

Repeating step 3-3-1～3-3-3, until T restrains, repeats above-mentioned steps 3-5 submatrix T and can restrain, wherein T ^tthe transposed matrix of representing matrix T, M ' _irepresent M _iinverse matrix, represent M _ireverse put, the orthogonal matrix after the positive triangle decomposition of φ representing matrix, Δ _ithe upper triangular matrix after the positive triangle decomposition of matrix, represent Δ _iinverse matrix, Q _ijthe unitary matrix of representing matrix svd, the svd of matrix, positive triangle decomposition, inverse of a matrix, transpose of a matrix method is called svd function, qr function, the symbol of inverting under matlab environment ' and ask transposition symbol T;

Step 3-4, prospect visual classification: by eigenvectors matrix M ₁, M ₂, M ₃project in the space at T place, i.e. M _i=T ^tm _i, i gets 1～3, obtains new eigenvectors matrix M ₁, M ₂, M ₃;

Step 3-5, color histogram feature clustering: color matrix M ₁adopt lowest rank Subspace clustering method, the color histogram of different prospects is often presented on different data dimensions, as shown in figure 10, and distance in K-means method generally adopts Euclidean distance, be not suitable for the distance of color space, so adopt Subspace clustering method, can realize preferably category division, utilize lowest rank method to calculate the similarity w between every two field picture, in Figure 10, two data acquisitions belong to different subspaces, by lowest rank method, can distinguish this two different subspaces; Structural map image, using all foreground images as node, the similarity w between image is as weight, then adopts spectral clustering Ncut method to cut apart figure image, thereby completes the classification to image, and the computing method of similarity w are as follows:

3-5-1, initiation parameter λ ₀, correlation matrix Z, the equivalent matrice J=0 of correlation matrix Z, Z=J, noise is corrected matrix E=0, Lagrangian matrix Y ₁=0, Y ₂=0, Lagrange punishment parameter μ=10 ^-6, maximum Lagrange punishment parameter m ax _μ=10 ¹⁰, Lagrange punishment parameter multiple ρ ₀=1.1, constant ε=10 ^-8;

3-5-2, calculates M ₁the correlation matrix equivalent matrice J of every column data: fix other matrix update matrix J, $J=\arg \min =\frac{1}{\mathrm{\&mu;}}{\left|\right|J\left|\right|}_{*}+\frac{1}{2}{\left|\right|J-(Z{+y}_2/\mathrm{\&mu;})\left|\right|}_F^2;$

3-5-3, calculates M ₁the correlation matrix Z of every column data: fix other matrix update matrixes Z, $Z=$ ${(I+{M_1}^t{M}_1)}^{-1}({M_1}^t{M}_1-{M_1}^{t^t}E+J+({M_1}^t{Y}_1-Y_2)/\mathrm{\&mu;});$

3-5-4, calculating noise is corrected matrix E: fix other matrix update matrixes E, $E=\arg \min \frac{{\mathrm{\&lambda;}}_0}{\mathrm{\&mu;}}{\left|\right|E\left|\right|}_{\mathrm{2,1}}+$ $\frac{1}{2}{\left|\right|E-(M_1-M_{1}Z+Y_1/\mathrm{\&mu;})\left|\right|}_F^2;$

3-5-5, calculates Lagrangian matrix Y ₁, Y ₂: Y ₁=Y ₁+ μ (M ₁-M ₁z-E), Y ₂=Y ₂+ μ (Z-J);

3-5-6, upgrades LaGrange parameter μ: μ=min (ρ ₀μ, max _μ);

3-5-7, judges whether iteration finishes: check || M ₁-M ₁z-E|| _∞< ε, || Z-J|| _∞whether < ε sets up, and finishes, otherwise continue iteration if set up iteration;

Wherein || || _*represent nuclear norm, || || _frepresent this norm of Fu Luo Benny crow, || || _∞represent maximum norm, smaller value in A and B is returned in min (A, B) representative, and above-mentioned iterative process obtains matrix Z, the element Z in matrix Z _i,j, Z _j,isimilar value between sum representative image i, j, build non-directed graph image, the node of image i representative graph image, similarity between image i and image j represents the weight between node i and node j, adopt spectral clustering Ncut method that thereby figure image is cut apart and realized the classification between a plurality of prospect key frames, in invention, the center of spectral clustering is set to the number of different foreground people.

Step 3-6, LBP and BOW feature clustering: LBP and the difference of BOW characteristic on Spatial Dimension are very little, directly adopt K-means method just can obtain good result, to matrix M ₂, M ₃adopt K-means method to carry out cluster, K-means cluster centre number is set as the number of foreground people;

Step 3-7, carries out integrated study to result: by step 3-5,3-6,3-7 obtains the classification C under each two field picture under three features _i, utilize three classification information ballots to determine the classification of each two field picture, as C ₁, C ₂, C ₃for image f _iclassification be respectively classification that 0,0,1 occurrence number is the highest 0 for image f _iclassification, thereby all key frame f _ican determine its classification information, thus all key frame f _ican determine its classification information.Calculating prospect video υ _iwith υ _jbetween similarity for prospect video υ _iif, its same υ _jbetween similarity higher than the similarity of video with other, υ _isame υ _jbe divided into same class.The contrast discovery providing from Figure 12, the method in invention relatively directly adopts the method for feature clustering to improve accuracy.

Embodiment

The Experimental Hardware environment of the present embodiment is: Intel-Core2Duo i321003.1GHz, 4G internal memory, programmed environment is visual studio2010, opencv2.3, matlab R2012a, the monitor video of test use mainly comes from the monitor video in Campus Monitoring System.

Adopt Gauss model number K=3 in mixed Gauss model extraction prospect, matching threshold parameter δ=1.5, initial variance σ init ²=30 ², initial weight ω _initget 0.3, learning rate α=1, key frame extraction parameter N=20, area threshold area _thresh=800pixel, threshold speed speed _thresh=25pixel/image, color histogram parameter m in characteristic extraction procedure _cradius r=3 in=64, LBP feature, histogram parameter m _l=64, BOW feature word number m _b=64, length m=64 after feature employing PCA dimensionality reduction, columns n=64 of matrix T in Fusion Features process.

Histogram parameter is set to 64 can reduce computational data amount, and histogram parameter setting is greater than 64 and can causes cluster result to disperse and be greater than 64 to bring numerous and jumbled calculated amount, and histogram parameter setting is less than 64 phenomenons of likely bringing a plurality of classes to merge.So select 64 to be used for doing histogram column subregion number parameter, in experiment, for the personage's prospect in different scenes, in order to reduce the impact of different surround lightings, adopted for the sift feature of illumination robust and processed, improved the accuracy rate of classification.

The present invention has used without supervision typical coefficient merges the method that a plurality of features promote classification, adopt multiple clustering method to divide a class personage's prospect of profile and color similarity, improve the efficiency of consulting monitor video, to have classification accuracy high in the present invention in a word, effective information ratio is high, without the feature of artificial mark.

Claims (4)

1. a method for monitor video personage foreground segmentation and classification, is characterized in that, comprises the following steps:

Step 1, the prospect of monitoring of separation video and background: adopt prospect and the background of mixed Gauss model monitoring of separation video, and each prospect is surrounded with the minimum bounding box that can surround prospect completely, form independently prospect small video;

Step 2, the feature of extraction prospect small video: prospect small video is extracted to one group of key frame, each prospect is recorded to area and two features of translational speed, carry out human classification before first by automobile foreground classification, the key frame of personage's prospect is carried out after dilation and erosion operation, to each key-frame extraction color histogram, local two value tags and word bag feature;

Step 3, Fusion Features and classification: the Area and Speed to each prospect obtaining arranges threshold value, sort out automobile and obtain personage's prospect; To personage's foreground extraction color histogram feature, local two value tags and word bag feature, adopt the method for canonical correlation coefficient to carry out unsupervised Fusion Features to described three features, obtain one and distinguish inhomogeneous space T, three Projection Characters are arrived to space T, color histogram feature after projection is done to lowest rank subspace clustering, LBP after projection and BOW feature are done to K-means cluster, and according to the result of cluster, foreground people small video is classified.

2. the method for a kind of monitor video personage foreground segmentation as claimed in claim 1 and classification, is characterized in that, step 1 comprises the following steps:

Step 1-1, initialization Gauss model: the first two field picture that reads monitor video, for each pixel in image builds a mixed Gauss model that contains K Gauss model, K span 3～5, represents in monitor video in every two field picture that with K Gauss model each pixel j is at the value x of moment t _j, pixel j is at moment t value x _jprobability P (x _j) can be determined by following formula:

Wherein represent the constantly weight of i gaussian component in the mixed Gauss model of pixel j of t, satisfied: with represent respectively t average and the covariance of i the gaussian component of pixel j constantly, represent Gaussian probability-density function, be expressed as follows:

Wherein d is x _jdimension, for RGB color space, each pixel has 3 passages, x _jfor tri-vector, covariance matrix wherein wherein be illustrated in t i the Gauss model variance of pixel j constantly, during initialization value is represent unit matrix, initial phase, the weights omega of each Gaussian distribution _init=1/K,

Step 1-2, upgrades Gauss model: continue to read monitor video, often read a two field picture of monitor video and just mixed Gauss model is done and upgraded; By each gauss component in mixed Gauss model according to descending sequence, the pixel value x of the current new frame reading _{j, t+1}if meet following formula with i Gauss model in mixed Gauss model:

Upgrade i gauss component, all the other gauss components remain unchanged, and pixel x _{j, t+1}in present frame, be considered to background parts pixel, parameter δ is matching threshold, δ span 1～2, and the computing method of upgrading i gauss component are as follows:

Wherein α is the learning rate of mixed Gauss model, span 0～1, the learning rate that ρ is parameter alpha; If pixel x _{j, t+1}all unmatched with K gauss component, judge that this pixel is the foreground pixel of present frame, to construct new gauss component and replace the gauss component after sequence is leaned on, the average of new gauss component is made as pixel x _{j, t+1}value, standard deviation and weight are set to respectively σ _initand ω _init, average and the variance of the gauss component of reservation remain unchanged, and weighted value upgrades according to the following formula:

Step 1-3, completes video foreground and background segment work: pixel x _{j, t+1}k gauss component upgrade after, weight to K gauss component is normalized, the step 1-1 and the 1-2 that repeat above retain the foreground pixel in every two field picture, until monitor video reads end, obtain the video of non-display background with the display foreground of the same resolution of original monitor video;

Step 1-4, extract the minimum bounding box that surrounds foreground people video: the monitor video obtaining in read step 1-3, every two field picture is first carried out to dilation and erosion operation, progressive scanning picture again, in document image, pixel value is not long l and the wide w of the rectangle that forms of 0 pixel, for same personage's prospect, the bounding box of each frame has long l and wide w, select l and w the longest in all frames, as the bounding box of this personage's prospect, obtain thus surrounding the small video of personage's prospect video.

3. the method for a kind of monitor video personage foreground segmentation as claimed in claim 2 and classification, is characterized in that, step 2 comprises the following step:

Step 2-1, the key frame of extraction personage prospect: fixedly choose the middle F frame of personage's video f ₁, f ₂..., f _fimage is as key frame, and F gets 20～40;

Step 2-2, extracts color histogram information: to F frame f ₁, f ₂..., f _fimage personage extracted region color characteristic histogram, the column subregion of establishing color histogram has m _cindividual, computed image f _ithe column subregion id that tri-Color Channel rgb values of middle pixel p are corresponding, i gets 1～F, and R represents red color channel value, and G represents green channel value, and B represents blue channel value, formula is as follows:

The number of adding up pixel in each column subregion id, obtains image f _icolor histogram, it is m that color histogram is finally expressed as length _cvectorial υ _c, all key frames are repeated to this step, until obtain m _cthe matrix M of * F ₁;

Step 2-3, extracts local two value tags: calculate F frame f ₁, f ₂..., f _fpart two value tags of image, first by image f _igray processing, the radius of the portion's two value tag LBP operators of setting a trap is r, and r gets 3 or 4 or 5, with the window of r*r, in image, moves, and location of pixels of every movement just calculates window center pixel p one time _centerlBP value, computing method are as follows: will with center pixel p _centeran adjacent r*r pixel respectively with center pixel p _centervalue compare, adjacent pixel values is greater than center pixel p _center, the position of this pixel is marked as 1, otherwise is marked as 0, obtain thus r*r-1 bit, final window has obtained the LBP feature of whole image while moving to last center pixel position, then by the LBP feature histogram graph representation of image, establishing LBP histogram subregion has m _lindividual, the height value of each component of histogram is together in series, obtain final part two value tags, length is m _lvectorial υ ₁, all key frames are repeated to this step, until obtain m _lthe matrix M of * F ₂;

Step 2-4, extracts word bag feature: first calculate F frame f ₁, f ₂..., f _fthe dimension rotation invariant feature sift unique point of image, establishing word list length in word bag model is m _b, adopt K-means clustering method that the close sift unique point of the meaning of a word is merged and obtains m _bindividual class, class center forms the word list of word bag, more again with the vocabulary in word list, replaces each the yardstick invariant feature conversion sift unique point in each two field picture, and in statistics word list, the corresponding sift unique point of each vocabulary number, finally obtains image f _ithe frequency of each vocabulary, length is m _bvectorial υ _b, all key frames are repeated to this step, until obtain m _b* F matrix M ₃;

Step 2-5, extracts area features and velocity characteristic: calculate F frame f ₁, f ₂..., f _fthe area s of the prospect of each two field picture in image ₁, s ₂..., s _fwith speed υ ₁, υ ₂..., υ _f-1the area of prospect is also number of pixels non-zero in prospect, get the mean value of F display foreground area as the area value s of this prospect, the displacement of the center that prospect speed is surrounded frame by the rectangle of prospect in original monitor video determines, a displacement is determined in the encirclement frame center of every two two field pictures, F two field picture calculates F-1 speed, gets the intermediate value of speed as the speed v of this prospect.

4. the method for a kind of monitor video personage foreground segmentation as claimed in claim 3 and classification, is characterized in that, step 3 comprises the following step:

Step 3-1, arranges the threshold value of foreground area and speed, area threshold area _thersh=800pixel, threshold speed speed _thersh=25pixel/image, pixel represents pixel, image presentation video, the foreground partition that area features surpasses area threshold is class of vehicles, when area features does not surpass area threshold, the threshold value if prospect velocity characteristic outpaces, foreground partition is class of vehicles, otherwise foreground partition is personage's classification;

Step 3-2, uniform data dimension: the color histogram matrix m for F image that step 2 is obtained _c* F, local two-value eigenmatrix m _l* F and word bag eigenmatrix m _b* F, calls principal component analysis (PCA) PCA method, is reduced to unified dimension m, and all eigenvectors matrixs become m * F;

Step 3-3, Fusion Features: establishing and having matrix T dimension is m * n, three eigenvectors matrix M ₁, M ₂, M ₃project in space, matrix T place, similar space vector is near at the projector distance in space, matrix T place, the projector distance far characteristic of inhomogeneous space vector in space, matrix T place, initialization T is vector of unit length matrix, the content of the renewal matrix T of iteration, concrete iterative process is as follows:

3-3-1, to matrix M ₁, M ₂, M ₃carry out the positive triangle decomposition of matrix and upgrade matrix M _i:

3-3-2, to every a pair of matrix M ' _i, M ' _jcarry out Singular Value Decomposition Using:

3-3-3, solution matrix T: compute matrix the proper vector of compute matrix A λ is proper vector t in matrix A _ieigenwert, by t _iaccording to order sequence formation matrix T, i.e. TT{t from big to small ₁, t ₂..., t _n, the different characteristic of matrix A vector number has been determined the size of n here;

Repeating step 3-3-1～3-3-3 is until T restrains, and wherein i span 1～3, T ^tthe transposed matrix of representing matrix T, M ' _irepresent M _iinverse matrix, represent M _icontrary transposed matrix, the orthogonal matrix after the positive triangle decomposition of φ representing matrix, Δ _ithe upper triangular matrix after the positive triangle decomposition of matrix, represent Δ _iinverse matrix, Q _ijthe unitary matrix of representing matrix svd, the row dimension n of T is determined by matrix A;

Step 3-4, prospect visual classification: by eigenvectors matrix M ₁, M ₂, M ₃project in the space at T place, i.e. M _i=T ^tm _i, i gets 1～3, obtains new eigenvectors matrix M ₁, M ₂, M ₃;

Step 3-5, color histogram feature clustering: color matrix M ₁adopt lowest rank Subspace clustering method to calculate the similarity w between every two field picture, structural map image, using all foreground images as node, similarity w between image is as weight, then adopt spectral clustering Ncut method to cut apart figure image, thereby complete the classification to image, the computing method of similarity w are as follows:

3-5-1, initiation parameter λ ₀, correlation matrix Z, the equivalent matrice J=0 of correlation matrix Z, Z=J, noise is corrected matrix E=0, Lagrangian matrix Y ₁=0, Y ₂=0, Lagrange punishment parameter μ=10 ^-6, maximum Lagrange punishment parameter m ax _μ=10 ¹⁰, Lagrange punishment parameter multiple ρ ₀=1.1, constant ε=10 ^-8;

3-5-2, calculates M ₁the correlation matrix equivalent matrice J of every column data: fix other matrix update matrix J,

3-5-3, calculates M ₁the correlation matrix Z of every column data: fix other matrix update matrixes Z, Z=(I+M ₁ ^tm ₁) ^-1(M ₁ ^tm ₁-M ₁ ^te+J+ (M ₁ ^ty ₁-Y ₂)/μ);

3-5-4, calculating noise is corrected matrix E: fix other matrix update matrixes E,

3-5-5, calculates Lagrangian matrix Y ₁, Y ₂: upgrade matrix Y ₁, Y ₂, Y ₁=Y ₁+ μ (M ₁-M ₁z-E), Y ₂=Y ₂+ μ (Z-J);

3-5-6, upgrades Lagrange punishment parameter μ, μ=min (ρ ₀μ, max _μ);

3-5-7, judges whether iteration finishes: check || M ₁-M ₁z-E|| _∞< ε, || Z-J|| _∞whether < ε sets up, and finishes, otherwise continue iteration if set up iteration;

Wherein || || _*represent nuclear norm, || || _frepresent this norm of Fu Luo Benny crow, || || _∞represent maximum norm, smaller value in A and B is returned in min (A, B) representative, and above-mentioned iterative process obtains matrix Z, the element Z in matrix Z _i,j, Z _j,isimilar value between sum representative image i, j, build non-directed graph image, the node of image i representative graph image, similarity between image i, j represents the weight between node i and node j, thereby adopts spectral clustering Ncut method that figure image is cut apart and realized the classification between a plurality of prospect key frames;

Step 3-6, local two value tags and word bag feature clustering: to matrix M ₂, M ₃adopt K-means method to carry out cluster;

Step 3-7, carries out integrated study to result: by step 3-5,3-6,3-7 obtains the classification C under each two field picture under three features _i, utilize three classification information ballots to determine the classification of each two field picture, thus all key frame F _ican determine its classification information, calculate prospect video υ _iwith υ _jbetween similarity

For prospect video υ _iif, its same υ _jbetween similarity higher than the similarity of video with other, υ _isame υ _jbe divided into same class.