CN101377813A - Method for real time tracking individual human face in complicated scene - Google Patents

Abstract

The invention relates to a method for tracing the face of a single person in real time in complex scenes. The method comprises the following steps: (1) pretreating frames: linearly transforming the frames captured by a video and filtrating waves with Gaussian templates; (2) detecting the face by adopting complexion features: detecting the face by adopting the convergence of complexion in YCrCb colored space and morphologic operation of the face; (3) elimination of areas similar to the complexion: elimination of non-face areas similar to complexion in combination with the face area and the face geometrical features to get a perfect face area before the barycenter of the face area is computed with the perfect face area and the initial search area is set around the barycenter; (4) tracing the face: tracing the face by adopting the continuous adaptive mean deflection method with the face detection results as novel back projection drawings and features. In the method, the face is abstracted and tracked in combination with multiple features; the effects are stable and the complexity of computation is low. The method can be widely applied in the fields of video monitoring, man computer interaction interfaces and so on.

Description

The method for real time tracking of individual human face in the complex scene

Technical field

The present invention relates to the mutual and field of video monitoring of man-machine interface, particularly to the real-time tracing of individual human face in the complex scene.Specifically be a kind ofly to take all factors into consideration many features, utilize the continuous average drifting method of novel trans to perspective view.

Background technology

In man-machine interaction and video monitoring etc. are used,, will be directly connected to the overall performance of system to the detection and tracking effect of people's face as one of notable feature of discerning personal identification.Recent decades, existing countless scholars did extensive studies and discussion to the face tracking problem, wherein mainly were divided into: based on the faceform, based on characteristic matching, based on movable information and based on the method for the colour of skin.Yet owing to exist such as head rotation, all multifactor influences such as illumination, shade and shelter cause the face tracking in the actual complex environment always all to be difficult to effective realization.

To consulting as can be known of prior art document, in various trackings, the average drifting algorithm is more satisfactory continuously.It is proposed and is successfully applied in the computer game human-computer interaction interface machine vision to the seizure of people's face motion in the article of a piece that delivered on the Intel technology quarterly in 1998 " Computer Vision Face Trackingfor Use in a Perceptual User Interface " by name by G.R.Bradski..Article is based on the average drifting algorithm, utilizes statistics with histogram in the inherent H color channel in region of search to calculate the similarity degree of follow-up each frame pixel of video sequence and skin color probability, and as feature people's face followed the tracks of.Yet, at the H color channel interference of background pixel in the region of search is carried out being subject in the statistics with histogram process in the region of search, influence the accuracy of calculating pixel class skin color probability, and then can not reach good tracking effect.Simultaneously, the initial search area in the literary composition must manually be set.

Summary of the invention

The objective of the invention is to overcome the deficiency of above-mentioned prior art, the method for real time tracking of individual human face in a kind of complex scene is provided, the binaryzation that detects with people's face is the probability of the pixel class colour of skin in the token image as a result, and as feature people's face is followed the tracks of.Simultaneously, utilize people's face testing result of first two field picture to determine initial search area automatically, algorithm is moved automatically.

For achieving the above object, the present invention adopts following technical proposals:

A kind of method for real time tracking at individual human face in the complex scene, its step comprises:

(1) picture pre-service: for reducing the influence of ambient lighting and picture noise, the present invention at first implements segmentation grey level fractal transform and gaussian filtering to image, and purpose is to reduce the interference of ambient lighting and picture noise;

(2) utilizing features of skin colors to carry out people's face detects: utilize the polymerism of the colour of skin in the YCrCb color space to carry out people's face and detect, and remove detection noise by morphological operation;

(3) elimination of class area of skin color: at the class area of skin color, bonded area thresholding and people's face height, widely further eliminate than restriction condition, determine the central point of facial zone by the space weight of calculating human face region, and be center setting initial search area with it;

(4) face tracking: people's face is detected the binaryzation result make tracked feature, utilize continuous adaptive mean shift method to realize following the tracks of.

Experimental result shows that this method does not need manual intervention to carry out real-time follow-up to the people's face in the picture automatically, and accuracy rate can reach more than 90%.

Below each step is described further:

(1) picture pre-service: comprise grey level fractal transform and the filtering of Gauss's template.Because the influence of complex illumination can make face some high light or dim zone occur, and these zones all can exert an influence to the judgement of the colour of skin in the environment.Therefore the brightness of picture need be carried out linear adjustment, so that it reaches the overall uniform effect.And because the hardware reason of video capture device can noise occur unavoidably in image, this can exert an influence to the judgement of pixel class area of skin color equally.Utilize Gauss's template that image is carried out convolution, can under the prerequisite that keeps the edge, remove noise preferably.

(2) utilizing features of skin colors to carry out people's face detects: in various color spaces, the colour of skin in the Cr among the YCrCb, the Cb passage has good polymerism, and observation data can obtain more satisfactory segmentation threshold by experiment, and then produces good skin color segmentation result.Simultaneously the initial detecting result is carried out morphological operation, eliminate some tiny cavity and detection noise.

(3) elimination of class area of skin color: utilize colour of skin information that face is detected and no doubt have many advantages, yet only rely on this single information always to be subjected to the interference of class area of skin color, and these zones can't rely on shape filtering to eliminate merely.Because some class area of skin color is bigger, needs repeatedly corrosion could eliminate, and repeatedly can cause the facial zone atrophy processing time.This method is further eliminated this class zone with geometric properties in conjunction with people's face is long-pending.At first be when zone during, think that this zone does not belong to people's face less than certain area; Next is that the long and short axial ratio of human face region is limited, and the zone in this scope all will not be eliminated, to reduce the influence of class area of skin color as far as possible.Simultaneously, in detected human face region, calculate centre of gravity place, and be that initial search area is set at the center, algorithm is moved automatically with it.

(4) face tracking: with people's face testing result as novel trans to perspective view, spatial moment in the search window is carried out iteration, make feature in the current search frame trend towards the distribution pattern of target signature, thereby instruct continuous average drifting method to adjust the position of search window and major axis angle automatically to dope the information of target in next frame, make track algorithm continue to carry out, when guaranteeing good tracking effect, also have simple calculated amount concurrently.

The pretreated specific implementation method of above-mentioned picture is as follows:

(1) linear transformation

If the brightness of piece image by f (x, y) expression, (x, the y) locus of representative image pixel, Min[f (x, y)], Max[f (x, y)] be the maximum of linear transformation, minimum thresholding, then pass through brightness of image G after the linear transformation (x y) is:

$G(x,y)=\left\{\begin{array}{cc}0& f(x,y)\&le;\mathrm{Min}\left[f(x,y)\right]\\ \frac{f(x,y)-\mathrm{Min}\left[f(x,y)\right]}{\mathrm{Max}\left[f(x,y)\right]-\mathrm{Min}\left[f(x,y)\right]}\&times;256& \mathrm{Min}\left[f(x,y)\right]<f(x,y)<\mathrm{Max}\left[f(x,y)\right]\\ 255& f(x,y)\&GreaterEqual;\mathrm{Max}\left[f(x,y)\right]\end{array}\right.,$

(2) Gauss's masterplate filtering

Filtering adopts discretize Gauss template that image is carried out convolution, and discretize Gauss's template is as follows:

It is above-mentioned that to utilize features of skin colors to carry out the specific implementation method that people's face detects as follows:

Digital picture is as follows to the conversion formula of YCrCb color space by RGB, wherein R, G, B, Y, C _r, C _bThe pixel value of difference represent pixel in corresponding color channel:

$\left\{\begin{array}{c}Y=0.299\&times;R+0.587\&times;G+0.114\&times;B\\ \mathrm{Cr}=R-Y\\ \mathrm{Cb}=B-Y\end{array}\right.,$

Utilize following formula afterwards, the pixel value that satisfies condition is set to 255, otherwise pixel value is set to 0, obtains people's face Preliminary detection result with this.C wherein _rRepresent the pixel value of digital picture in respective channel, C _RMax, C _RMinC when defining pixel and belonging to human face region _rThe maximal value that is satisfied, minimum value scope; C _b, C _BMax, C _BMinThe representative implication in like manner.(C _rMin<C _r<C _rMax)∩(C _bMin<C _b<C _bMax)，

The elimination specific implementation method of above-mentioned class area of skin color is as follows:

(1) takes all factors into consideration the long-pending and geometric properties elimination class area of skin color of people's face

Each isolated area is asked its area, in digital picture, be the number of pixels in the zone, when region area is just eliminated it less than 100 the time from the Preliminary detection result; Simultaneously, remaining isolated area is calculated long and short axial ratio respectively,, the zone that long and short axial ratio does not change in 1.0～2.3 scopes is further eliminated according to facial characteristics and observation experience.

(2) initial search area determines

Initial search area is determined by following formula:

$x_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_y\mathrm{xI}(x,y)}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{y}I(x,y)}$ $y_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_y\mathrm{yI}(x,y)}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{y}I(x,y)},$

Wherein, (x _o, y _o) be the center of initial search area, (and x, y) representative's face detects the locus of binary image pixel, and (x is that it is at (x, the pixel value of y) locating y) to I; With (x _o, y _o) be the center, the rectangular window of setting one 200 * 100 is as initial search area.

The present invention has following conspicuous outstanding substantive distinguishing features and remarkable advantage compared with prior art:

Face tracking method of the present invention can be when having low calculated amount to complex scene in the situations such as rotation, inclination of people's face successfully realize following the tracks of, after following the tracks of failure, also can carry out again automatically, and to not obvious such as the interference of hand class area of skin color.Simultaneously, the present invention has also realized the automatic setting of initial search area, and whole process can be carried out under unmanned situation of interfering automatically.The present invention extracts people's face in conjunction with multiple characteristics and realizes following the tracks of, and effect stability and computation complexity are low, can be widely used in fields such as video monitoring, human-computer interaction interface.

Description of drawings

Fig. 1 is a FB(flow block) of the present invention.

Fig. 2 is for carrying out the comparison of morphological operation front and back to the initial detecting result among the present invention.

The Cr that Fig. 3 utilizes for the present invention, the colour of skin polymerization property in the Cb color channel are showed.

Fig. 4 is that final tracking effect of the present invention is showed.

Embodiment

A preferred embodiment of the present invention accompanying drawings is as follows: referring to Fig. 1, this is at the method for real time tracking of individual human face in the complex scene, and its step comprises:

(1) picture pre-service: the picture to Video Capture carries out image linear transformation and the filtering of Gauss's template;

(2) utilizing features of skin colors to carry out people's face detects: utilize the colour of skin people's face to be detected in the CrCb passage at the polymerism of YCrCb color space, and utilize the noise after morphological operation is tentatively removed detection;

(3) elimination of class area of skin color: at the class area of skin color in the scene, further eliminate, calculate the human face region barycenter, and be center setting initial search area with it in conjunction with human face region area and geometric properties;

(4) face tracking: with people's face testing result as novel trans to perspective view, utilize continuous adaptive mean shift method to realize face tracking.

The pretreated implementation method of above-mentioned picture is as follows:

Before people's face is detected, need carry out pre-service to picture.Specifically comprise gray scale linear transformation and Gauss's masterplate convolution.Wherein gray scale linear transformation purpose is the overall brightness adjustment of picture even.Adjustment process is calculated according to formula (1):

$G(x,y)=\left\{\begin{array}{cc}0& f(x,y)\&le;\mathrm{Min}\left[f(x,y)\right]\\ \frac{f(x,y)-\mathrm{Min}\left[f(x,y)\right]}{\mathrm{Max}\left[f(x,y)\right]-\mathrm{Min}\left[f(x,y)\right]}\&times;256& \mathrm{Min}\left[f(x,y)\right]<f(x,y)<\mathrm{Max}\left[f(x,y)\right]\\ 255& f(x,y)\&GreaterEqual;\mathrm{Max}\left[f(x,y)\right]\end{array}\right.---\left(1\right)$

Wherein, (x y) represents gray-scale value in the original image to f, and ((x y) is gray-scale value after the linear transformations to G for x, the y) locus of representative image pixel.Min[f (x, y)], Max[f (x, y)] be the maximum, the minimum thresholding that set.

Afterwards, adopt following Gauss's template and original image to carry out convolution, to eliminate the noise that brings by video capture device.

The above-mentioned implementation method of utilizing features of skin colors to carry out the detection of people's face is as follows: in various color spaces, the colour of skin in Cr among the YCrCb, the Cb passage has good polymerism (seeing that Fig. 3 is that colour of skin piece polymerism in Cr, Cb passage is showed), therefore image need be transformed into the YCrCb color space from RGB, wherein R, G, B, Y, Cr, Cb distinguish the pixel value of represent pixel in respective channel, conversion formula such as formula (2):

$\left\{\begin{array}{c}Y=0.299\&times;R+0.587\&times;G+0.114\&times;B\\ \mathrm{Cr}=R-Y\\ \mathrm{Cb}=B-Y\end{array}\right.---\left(2\right)$

As table 1, show according to experimental observation, under different periods, illumination condition, by setting different thresholdings, can utilize formula (3) in Cr, Cb passage, people's face to be detected, and the pixel value that satisfies condition is changed to 255, otherwise pixel value is changed to 0, obtains people's face Preliminary detection result with this

(C _rMin<C _r<C _rMax)∩(C _bMin<C _b<C _bMax) (3)

Table 1

Carry out morphological operation according to table 1 couple initial detecting result afterwards, eliminate some tiny cavities and detection noise (Fig. 2 is seen in morphological operation result displaying).

The implementation method of the elimination of above-mentioned class area of skin color is as follows: after utilizing features of skin colors to finish people's face Preliminary detection, comprise that some pixel values of human face region meeting generation are 255 zone, these class area of skin color to be eliminated: at first each isolated area is asked its area (being the number of pixels in the zone in digital picture), when region area is just eliminated it less than 100 the time from the Preliminary detection result.Secondly, more remaining isolated area is calculated long and short axial ratio respectively.According to facial geometric properties and observation experience, the zone that long and short axial ratio does not change in 1.0～2.3 scopes is further eliminated.Through above-mentioned processing, most class area of skin color can both be eliminated smoothly.

Set initial search area:

For tracing process is moved automatically, utilize formula (4) to calculate the facial zone barycenter and be center automatic setting initial search area with it:

$x_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_y\mathrm{xI}(x,y)}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{y}I(x,y)}$ $y_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_y\mathrm{yI}(x,y)}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{y}I(x,y)}---\left(4\right)$

Wherein, (x _o, y _o) be the center of human face region, (and x, y) representative's face detects the locus of binary image pixel, and (x is that it is at (x, the brightness value of y) locating y) to I.With (x _o, y _o) be the center, the rectangular window of setting one 200 * 100 is as initial search area.

The implementation method of above-mentioned face tracking is as follows:

By above-mentioned processing, face tracking just can move automatically.This is the computation process of an iteration, and concrete computation process is as follows:

(a) in first frame of sequence of frames of video, carry out people's face and detect,, then determine initial search area automatically if detect successfully; Otherwise then continue next frame is detected, till detecting the success of people's face

(b) in the binary picture that people's face detects, utilize 0 ~ 2 rank square of average drifting algorithm, in order to determine to follow the tracks of position of window, major axis deflection angle by one or many iterative computation zone

(c) keep following the tracks of the window size constancy, change the center and the deflection angle of search box, it is moved to definite position by (b)

(d) follow-up every two field picture is repeated (b), (c) step,, realize face tracking up to the window's position convergence.Fig. 4 is that final tracking effect of the present invention is showed, the result shows that the present invention has not only successfully realized aligning the tracking of face, under offside face and the head deflection situation good tracking effect is arranged also, in addition, the interference such as class area of skin color such as hands is also had stronger robustness.

Claims (4)

1. method for real time tracking at individual human face in the complex scene, its step comprises:

A. picture pre-service: the picture to Video Capture carries out image linear transformation and the filtering of Gauss's template;

B. utilizing features of skin colors to carry out people's face detects: utilize the colour of skin people's face to be detected in the CrCb passage at the polymerism of YCrCb color space, and utilize the noise after morphological operation is tentatively removed detection;

C. the elimination of class area of skin color: at the class area of skin color in the scene, further eliminate, calculate the human face region barycenter, and be center setting initial search area with it in conjunction with human face region area and geometric properties;

D. face tracking: with people's face testing result as novel trans to perspective view, utilize continuous adaptive mean shift method to realize face tracking.

2. the method for real time tracking at individual human face in the complex scene according to claim 1 is characterized in that the pretreated specific implementation method of described picture is as follows:

(1) linear transformation

If the brightness of piece image by f (x, y) expression, (x, the y) locus of representative image pixel, Min[f (x, y)], Max[f (x, y)] be the maximum of linear transformation, minimum thresholding, then pass through brightness of image G after the linear transformation (x y) is:

$G(x,y)=\left\{\begin{array}{cc}0& f(x,y)\&le;\mathrm{Min}\left[f(x,y)\right]\\ \frac{f(x,y)-\mathrm{Min}\left[f(x,y)\right]}{\mathrm{Max}\left[f(x,y)\right]-\mathrm{Min}\left[f(x,y)\right]}\&times;256& \mathrm{Min}\left[f(x,y)\right]<f(x,y)<\mathrm{Max}\left[f(x,y)\right]\\ 255& f(x,y)\&GreaterEqual;\mathrm{Max}\left[f(x,y)\right]\end{array}\right.,$

(2) Gauss's masterplate filtering

Filtering adopts discretize Gauss template that image is carried out convolution, and discretize Gauss's template is as follows:

3. the method for real time tracking at individual human face in the complex scene according to claim 1 is characterized in that the described specific implementation method of utilizing features of skin colors to carry out the detection of people's face is as follows:

Digital picture is as follows to the conversion formula of YCrCb color space by RGB, wherein R, G, B, Y, C _r, C _bThe pixel value of difference represent pixel in corresponding color channel:

$\left\{\begin{array}{c}Y=0.299\&times;R+0.587\&times;G+0.114\&times;B\\ \mathrm{Cr}=R-Y\\ \mathrm{Cb}=B-Y\end{array}\right.,$

Utilize following formula afterwards, the pixel value that satisfies condition is set to 255, otherwise pixel value is set to 0, obtains people's face Preliminary detection result with this.C wherein _rRepresent the pixel value of digital picture in respective channel, C _RMax, C _RMinC when defining pixel and belonging to human face region _rThe maximal value that is satisfied, minimum value scope; C _b, C _BMax, C _BMinThe representative implication in like manner.

$(C_{\mathrm{rMin}}<C_r<C_{\mathrm{rMax}})\&cap;(C_{\mathrm{bMin}}<C_b<C_{\mathrm{bMax}}).$

4. the method for real time tracking at individual human face in the complex scene according to claim 1 is characterized in that the specific implementation method of elimination of described class area of skin color is as follows:

(1) takes all factors into consideration the long-pending and geometric properties elimination class area of skin color of people's face

Each isolated area is asked its area, in digital picture, be the number of pixels in the zone, when region area is just eliminated it less than 100 the time from the Preliminary detection result; Simultaneously, remaining isolated area is calculated long and short axial ratio respectively,, the zone that long and short axial ratio does not change in 1.0～2.3 scopes is further eliminated according to facial characteristics and observation experience;

(2) initial search area determines

Initial search area is determined by following formula:

$x_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_y\mathrm{xI}(x,y)}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{y}I(x,y)}$ $y_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_y\mathrm{yI}(x,y)}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{y}I(x,y)},$

Wherein, (x _O, y _O) be the center of initial search area, (and x, y) representative's face detects the locus of binary image pixel, and (x is that it is at (x, the pixel value of y) locating y) to I; With (x _O, y _O) be the center, the rectangular window of setting one 200 * 100 is as initial search area.

Images