CN101377813A - Method for real time tracking individual human face in complicated scene - Google Patents

Abstract

The invention relates to a real-time tracking method for a single human face in a complex scene. The steps of the method are: (1) picture preprocessing: linear transformation and Gaussian template filtering are carried out on the picture captured by the video; (2) face detection is carried out by using the skin color feature: the human face is detected by using the aggregation of the skin color in the YCrCb color space Detect and perform morphological operations; (3) Elimination of skin-like areas: Combine the area and geometric features of the face to eliminate non-face skin-like areas, and after obtaining the ideal face area, use it to calculate the center of gravity of the face area , and use it as the center to set the initial search area; (4) Face tracking: use the face detection result as a new type of back-projection map, and use continuous adaptive mean shift method to realize face tracking based on it. The invention combines multiple features to extract human faces and realize tracking, the effect is stable and the calculation complexity is low, and the invention can be widely used in the fields of video monitoring, human-computer interaction interface and the like.

Description

Real-time Tracking Method for Individual Faces in Complex Scenes

technical field

The invention relates to the fields of man-machine interface interaction and video monitoring, especially the real-time tracking of a single human face in complex scenes. Specifically, it is a continuous mean shift method that comprehensively considers multiple features and utilizes a new type of back-projection graph.

Background technique

In applications such as human-computer interaction and video surveillance, as one of the salient features for identifying personal identities, the detection and tracking of human faces will directly affect the overall performance of the system. For decades, countless scholars have done extensive research and discussion on the problem of face tracking, which are mainly divided into: methods based on face models, based on feature matching, based on motion information and based on skin color. However, due to the influence of many factors such as head rotation, lighting, shadows, and occluders, it has always been difficult to effectively implement face tracking in actual complex environments.

A review of the existing technical literature shows that among various tracking methods, the continuous mean shift algorithm is more satisfactory. It was proposed by G.R. Bradski in an article titled "Computer Vision Face Tracking for Use in a Perceptual User Interface" published in the Intel Technology Quarterly in 1998 and successfully applied to the human-computer interaction interface of computer games. Motion capture. Based on the mean shift algorithm, the article uses the histogram statistics in the H color channel in the search area to calculate the similarity between the pixels of each subsequent frame of the video sequence and the probability of skin color, and uses this as a feature to track the face. However, in the process of performing histogram statistics on the search area by the H color channel, it is susceptible to the interference of background pixels in the search area, which affects the accuracy of calculating the probability of pixel-like skin color, and thus cannot achieve a good tracking effect. At the same time, the initial search area in the text must be set manually.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the above-mentioned prior art, provide a real-time tracking method of a single face in a complex scene, and use the binarization result of face detection to represent the probability of pixel-like skin color in the image, and use this as a feature Track faces. At the same time, the initial search area is automatically determined by using the face detection result of the first frame image, so that the algorithm runs automatically.

To achieve the above object, the present invention adopts the following technical solutions:

A real-time tracking method for a single human face in a complex scene, the steps of which include:

(1) Picture preprocessing: in order to reduce the impact of ambient light and image noise, the present invention first implements segmented grayscale linear transformation and Gaussian filtering to the image, with the aim of reducing the interference of ambient light and image noise;

(2) Face detection using skin color features: use the aggregation of skin color in YCrCb color space to detect faces, and remove detection noise through morphological operations;

(3) Elimination of the skin-like area: For the skin-like area, combined with the area threshold and the face height and width ratio constraints to further eliminate, the center point of the face area is determined by calculating the spatial weight of the face area, and it is used as The center sets the initial search area;

(4) Face tracking: The binarization result of face detection is used as the tracked feature, and the continuous adaptive mean shift method is used to realize the tracking.

The experimental results show that this method can automatically track the faces in the picture in real time without manual intervention, and the accuracy rate can reach more than 90%.

Each step is further explained below:

(1) Screen preprocessing: including grayscale linear transformation and Gaussian template filtering. Due to the influence of complex lighting in the environment, there will be some bright or dark areas on the face, and these areas will affect the judgment of skin color. Therefore, it is necessary to linearly adjust the brightness of the picture to achieve an overall uniform effect. Due to the hardware of the video capture device, it is inevitable that noise will appear in the image, which will also affect the judgment of the pixel-like skin color area. Using a Gaussian template to convolve the image can better remove noise while maintaining the edge.

(2) Face detection using skin color features: in various color spaces, the skin color in the Cr and Cb channels of YCrCb has good aggregation, and an ideal segmentation threshold can be obtained through experimental observation data, and then a good segmentation threshold can be obtained. Skin color segmentation results. At the same time, morphological operations are performed on the initial detection results to eliminate some small holes and detection noise.

(3) Elimination of skin-like areas: There are many advantages in using skin-color information to detect faces, but only relying on this single information will always be interfered by skin-like areas, and these areas cannot be eliminated simply by morphological filtering. Because some types of skin color have large areas, multiple erosions are required to eliminate them, and multiple treatments will cause the facial area to shrink at the same time. This method combines face area and geometric features to further eliminate such areas. Firstly, when the area is smaller than a certain area, it is considered that this area does not belong to the face; secondly, the ratio of the long axis to the short axis of the face area is limited, and the areas that are not within this range will be eliminated to reduce the skin-like area as much as possible. Influence. At the same time, calculate the position of the center of gravity in the detected face area, and set the initial search area with it as the center, so that the algorithm can run automatically.

(4) Face tracking: use the face detection result as a new type of back-projection image, and iterate the spatial moments in the search window, so that the features in the current search box tend to the distribution pattern of the target features, thereby guiding the continuous mean shift method Automatically adjust the position of the search window and the angle of the long axis to predict the information of the target in the next frame, so that the tracking algorithm can continue to perform, while ensuring a good tracking effect, it also has a simple amount of calculation.

The specific implementation method of the above picture preprocessing is as follows:

(1) Linear transformation

Suppose the brightness of an image is represented by f(x, y), (x, y) represents the spatial position of the image pixel, and Min[f(x, y)], Max[f(x, y)] are linear transformation The maximum and minimum threshold values, then the image brightness G(x, y) after linear transformation is:

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>\\ \frac{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 256</span>}& \mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>\\ \mathrm{<span\; class="notranslate">\; 255</span>}& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>\end{array}\right.<span\; class="notranslate">\; ,</span>$

(2) Gaussian template filtering

The filtering process uses a discrete Gaussian template to convolve the image, and the discrete Gaussian template is as follows:

The specific implementation method of the above-mentioned face detection using the skin color feature is as follows:

The conversion formula of a digital image from RGB to YCrCb color space is as follows, where R, G, B, Y, C _r , and C _b represent the pixel values of pixels in the corresponding color channels:

$\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0.299</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.587</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.114</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; B</span>}\\ \mathrm{<span\; class="notranslate">\; Cr</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Y</span>}\\ \mathrm{<span\; class="notranslate">\; Cb</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Y</span>}\end{array}\right.<span\; class="notranslate">\; ,</span>$

Then use the following formula to set the pixel value that satisfies the condition to 255, otherwise set the pixel value to 0, so as to obtain the preliminary detection result of the face. Among them, C _r represents the pixel value of the digital image in the corresponding channel, C _rMax and C _rMin limit the maximum and minimum value range that C _r satisfies when the pixel belongs to the face area; the meanings represented by C _b , C _bMax and C _bMin the same way. (C _rMin < _Cr < _CrMax )∩( _CbMin < _Cb < _CbMax ),

The specific implementation method of the above-mentioned skin-color-like area elimination is as follows:

(1) Comprehensively consider the face area and geometric features to eliminate skin-like areas

Calculate the area of each independent area, which is the number of pixels in the area in the digital image. When the area of the area is less than 100, it will be eliminated from the preliminary detection results; at the same time, the long and short axes of the remaining independent areas are calculated respectively. According to the facial features and observation experience, the areas where the ratio of the major axis and the minor axis do not change within the range of 1.0 to 2.3 are further eliminated.

(2) Determination of the initial search area

The initial search area is determined by the following formula:

$x_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{\mathrm{the\; y}}\mathrm{xI}(x,\mathrm{the\; y})}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{\mathrm{the\; y}}I(x,\mathrm{the\; y})}$ ${\mathrm{the\; y}}_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{\mathrm{the\; y}}i(x,\mathrm{the\; y})}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{\mathrm{the\; y}}I(x,\mathrm{the\; y})},$

Among them, (x _o , y _o ) is the center of the initial search area, (x, y) represents the spatial position of the binarized image pixel for face detection, and I(x, y) is its position at (x, y) Pixel value; with (x _o , y _o ) as the center, set a 200×100 rectangular window as the initial search area.

Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages:

The human face tracking method of the present invention can successfully track the rotation and tilt of the human face in complex scenes while having a low amount of calculation, and can automatically re-start when the tracking fails, and can track skin-colored areas such as hands interference is not obvious. At the same time, the invention also realizes the automatic setting of the initial search area, so that the whole process can be carried out automatically without human intervention. The invention combines multiple features to extract human faces and realize tracking, the effect is stable and the calculation complexity is low, and the invention can be widely used in the fields of video monitoring, human-computer interaction interface and the like.

Description of drawings

Fig. 1 is a flow chart of the present invention.

Fig. 2 is the comparison before and after the morphological operation on the initial detection results in the present invention.

Fig. 3 shows the aggregation characteristics of skin color in the Cr and Cb color channels utilized in the present invention.

Fig. 4 shows the final tracking effect of the present invention.

Detailed ways

A preferred embodiment of the present invention is described as follows in conjunction with accompanying drawing: Referring to Fig. 1, this real-time tracking method for single human face in complex scene, its step comprises:

(1) Picture preprocessing: perform image linear transformation and Gaussian template filtering on the picture captured by the video;

(2) Face detection using skin color features: use the aggregation of skin color in the YCrCb color space to detect faces in the CrCb channel, and use morphological operations to initially remove the detected noise;

(3) Elimination of the skin-like area: for the skin-like area in the scene, combined with the face area and geometric features to further eliminate, calculate the centroid of the face area, and set the initial search area centered on it;

(4) Face tracking: The face detection result is used as a new back projection image, and the continuous adaptive mean shift method is used to realize face tracking.

The implementation method of the above screen preprocessing is as follows:

Before detecting the face, the image needs to be preprocessed. Specifically, it includes grayscale linear transformation and Gaussian template convolution. Among them, the purpose of grayscale linear transformation is to adjust the overall brightness of the picture evenly. The adjustment process is calculated according to formula (1):

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>\\ \frac{\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 256</span>}& \mathrm{<span\; class="notranslate">\; Min</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>\\ \mathrm{<span\; class="notranslate">\; 255</span>}& \mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; Max</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Among them, f(x, y) represents the gray value in the original image, (x, y) represents the spatial position of the image pixel, and G(x, y) is the gray value after linear transformation. Min[f(x, y)] and Max[f(x, y)] are the set maximum and minimum threshold values.

Afterwards, the following Gaussian template is used to convolve with the original image to remove the noise brought by the video capture device.

The above-mentioned implementation method of using skin color features for face detection is as follows: in various color spaces, the skin color in the Cr and Cb channels in YCrCb has good aggregation (see Figure 3 for a skin color block aggregated in the Cr and Cb channels Sexual display), so it is necessary to convert the image from RGB to YCrCb color space, where R, G, B, Y, Cr, and Cb represent the pixel values of the pixels in the corresponding channels, and the conversion formula is as formula (2):

$\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0.299</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.587</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 0.114</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; B</span>}\\ \mathrm{<span\; class="notranslate">\; Cr</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Y</span>}\\ \mathrm{<span\; class="notranslate">\; Cb</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Y</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

As shown in Table 1, according to experimental observations, under different time periods and lighting conditions, by setting different threshold values, the formula (3) can be used to detect faces in the Cr and Cb channels, and the pixel values satisfying the conditions Set it to 255, otherwise set the pixel value to 0, so as to get the preliminary detection result of the face

(C _rMin <C _r <C _rMax )∩(C _bMin <C _b <C _bMax ) (3)

Table 1

Then perform morphological operations on the initial detection results according to Table 1 to eliminate some small holes and detection noise (see Figure 2 for the results of morphological operations).

The implementation method of the elimination of the above-mentioned skin-like areas is as follows: After the preliminary detection of the face is completed using the skin-color features, some areas with a pixel value of 255 will be generated including the face area. To eliminate these skin-like areas: first, each independent Calculate the area of the area (in the digital image, it is the number of pixels in the area), and when the area is less than 100, it will be eliminated from the preliminary detection results. Secondly, calculate the long-axis and short-axis ratios for the remaining independent regions respectively. According to the facial geometric characteristics and observation experience, the areas where the ratio of the major axis to the minor axis did not change within the range of 1.0 to 2.3 were further eliminated. After the above processing, most of the skin-like areas can be successfully eliminated.

Set the initial search area:

In order to make the tracking process run automatically, formula (4) is used to calculate the centroid of the face area and automatically set the initial search area with it as the center:

$x_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{\mathrm{the\; y}}\mathrm{xI}(x,\mathrm{the\; y})}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{\mathrm{the\; y}}I(x,\mathrm{the\; y})}$ ${\mathrm{the\; y}}_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{\mathrm{the\; y}}i(x,\mathrm{the\; y})}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{\mathrm{the\; y}}I(x,\mathrm{the\; y})}---\left(4\right)$

Among them, (x _o , y _o ) is the center of the face area, (x, y) represents the spatial position of the binarized image pixel for face detection, and I(x, y) is its brightness at (x, y) value. With (x _o , y _o ) as the center, set a 200×100 rectangular window as the initial search area.

The implementation method of the above face tracking is as follows:

Through the above-mentioned processing, the face tracking can run automatically. This is an iterative calculation process, and the specific calculation process is as follows:

(a) Face detection is performed in the first frame of the video frame sequence. If the detection is successful, the initial search area is automatically determined; otherwise, continue to detect the next frame until the detection of the face is successful.

(b) In the binary image of face detection, use the mean shift algorithm to calculate the 0~2 order moments of the region through one or more iterations to determine the position of the tracking window and the long axis deflection angle

(c) Keep the tracking window size unchanged, change the center position and deflection angle of the search box, and move it to the position determined by (b)

(d) Repeat steps (b) and (c) for each subsequent frame of image until the window position converges to realize face tracking. Figure 4 shows the final tracking effect of the present invention. The results show that the present invention not only successfully realizes the tracking of the front face, but also has a good tracking effect on the side face and head deflection. The interference in the skin-like area is also robust.

Claims (4)

1. method for real time tracking at individual human face in the complex scene, its step comprises:

A. picture pre-service: the picture to Video Capture carries out image linear transformation and the filtering of Gauss's template;

B. utilizing features of skin colors to carry out people's face detects: utilize the colour of skin people's face to be detected in the CrCb passage at the polymerism of YCrCb color space, and utilize the noise after morphological operation is tentatively removed detection;

C. the elimination of class area of skin color: at the class area of skin color in the scene, further eliminate, calculate the human face region barycenter, and be center setting initial search area with it in conjunction with human face region area and geometric properties;

D. face tracking: with people's face testing result as novel trans to perspective view, utilize continuous adaptive mean shift method to realize face tracking.

2. the method for real time tracking at individual human face in the complex scene according to claim 1 is characterized in that the pretreated specific implementation method of described picture is as follows:

(1) linear transformation

If the brightness of piece image by f (x, y) expression, (x, the y) locus of representative image pixel, Min[f (x, y)], Max[f (x, y)] be the maximum of linear transformation, minimum thresholding, then pass through brightness of image G after the linear transformation (x y) is:

$G(x,y)=\left\{\begin{array}{cc}0& f(x,y)\&le;\mathrm{Min}\left[f(x,y)\right]\\ \frac{f(x,y)-\mathrm{Min}\left[f(x,y)\right]}{\mathrm{Max}\left[f(x,y)\right]-\mathrm{Min}\left[f(x,y)\right]}\&times;256& \mathrm{Min}\left[f(x,y)\right]<f(x,y)<\mathrm{Max}\left[f(x,y)\right]\\ 255& f(x,y)\&GreaterEqual;\mathrm{Max}\left[f(x,y)\right]\end{array}\right.,$

(2) Gauss's masterplate filtering

Filtering adopts discretize Gauss template that image is carried out convolution, and discretize Gauss's template is as follows:

3. the method for real time tracking at individual human face in the complex scene according to claim 1 is characterized in that the described specific implementation method of utilizing features of skin colors to carry out the detection of people's face is as follows:

Digital picture is as follows to the conversion formula of YCrCb color space by RGB, wherein R, G, B, Y, C _r, C _bThe pixel value of difference represent pixel in corresponding color channel:

$\left\{\begin{array}{c}Y=0.299\&times;R+0.587\&times;G+0.114\&times;B\\ \mathrm{Cr}=R-Y\\ \mathrm{Cb}=B-Y\end{array}\right.,$

Utilize following formula afterwards, the pixel value that satisfies condition is set to 255, otherwise pixel value is set to 0, obtains people's face Preliminary detection result with this.C wherein _rRepresent the pixel value of digital picture in respective channel, C _RMax, C _RMinC when defining pixel and belonging to human face region _rThe maximal value that is satisfied, minimum value scope; C _b, C _BMax, C _BMinThe representative implication in like manner.

$(C_{\mathrm{rMin}}<C_r<C_{\mathrm{rMax}})\&cap;(C_{\mathrm{bMin}}<C_b<C_{\mathrm{bMax}}).$

4. the method for real time tracking at individual human face in the complex scene according to claim 1 is characterized in that the specific implementation method of elimination of described class area of skin color is as follows:

(1) takes all factors into consideration the long-pending and geometric properties elimination class area of skin color of people's face

Each isolated area is asked its area, in digital picture, be the number of pixels in the zone, when region area is just eliminated it less than 100 the time from the Preliminary detection result; Simultaneously, remaining isolated area is calculated long and short axial ratio respectively,, the zone that long and short axial ratio does not change in 1.0～2.3 scopes is further eliminated according to facial characteristics and observation experience;

(2) initial search area determines

Initial search area is determined by following formula:

$x_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_y\mathrm{xI}(x,y)}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{y}I(x,y)}$ $y_o=\frac{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_y\mathrm{yI}(x,y)}{{\mathrm{\&Sigma;}}_x{\mathrm{\&Sigma;}}_{y}I(x,y)},$

Wherein, (x _O, y _O) be the center of initial search area, (and x, y) representative's face detects the locus of binary image pixel, and (x is that it is at (x, the pixel value of y) locating y) to I; With (x _O, y _O) be the center, the rectangular window of setting one 200 * 100 is as initial search area.

Images