CN104392223A - Method for recognizing human postures in two-dimensional video images - Google Patents

Abstract

The invention discloses a human body posture recognition method in a two-dimensional video image, which comprises the following steps: Group according to the size of the scale; calculate a sampled image of a specified scale for each group of images, and calculate HOG for the sampled image; use the HOG prediction of a sampled image in each group to calculate the HOG corresponding to other specified scale sampled images in the group; According to the obtained multi-scale HOG, combined with the trained SVM classifier to detect the original video image Human body target areas under different scales; use the trained random forest classifier to classify the pixels of the detected human body target area, and determine the body part area in the human body target area; connect the body parts to form a human body contour, and realize Human gesture recognition. By applying the method of the present invention, the calculation speed of the multi-scale bottom layer features is accelerated without reducing the detection precision, and the gesture recognition speed and precision are improved.

Description

Human Pose Recognition Method in 2D Video Image

technical field

The invention belongs to the technical field of image processing, and in particular relates to a human body gesture recognition method in two-dimensional video images.

Background technique

Human gesture recognition can be applied to human activity analysis, human-computer interaction, and visual surveillance, and it is a hot issue in the field of computer vision in the near future. Human pose recognition refers to detecting the position of each part of the human body from the image and calculating its direction and scale information. The results of pose recognition are divided into two-dimensional and three-dimensional cases, and the estimation methods are divided into model-based and model-free approaches.

A Chinese patent application with a publication number of CN101350064A discloses a method and device for estimating a two-dimensional human pose. This method first detects the human body area in the two-dimensional image and determines the search range of the human body parts in the two-dimensional image. Then according to the search range of human body parts, combined with the torso, head, hands, legs, and feet of human body parts, the template calculates the matching similarity to realize the identification of each part; combined with the constraint relationship between adjacent parts, the two The posture of the human body. The implementation steps are as follows:

Step 1: Use the existing methods such as optical flow method, frame difference method, and background phase difference to detect the human body area in the two-dimensional image.

Step 2: Determine the search range of multiple human body parts in the human body area.

(1) Perform face detection in the human body area, and use the position of the detected face as the search range of the head;

(2) Use the detected skin color features of the face to determine the search range of the left and right hands; and then determine the search range of the human torso, left arm, and right arm.

(3) Determine the remaining part of the human body region as the search range of the left leg, left foot, right leg, and right foot.

Step 3: Calculate the matching similarity within the corresponding body part search range according to each body part template, determine the optimal position of each part of the human body, and combine the constraint relationship between adjacent human body parts to obtain the posture of the two-dimensional human body.

The above-mentioned method for estimating human posture has the following disadvantages:

First, using existing methods such as optical flow method, inter-frame difference method, and background phase difference to detect human body areas in two-dimensional images, there are problems such as illumination changes, background dynamic changes, and optical flow multi-scale calculation speed. , which often leads to a large error in the detected human body area, burying hidden dangers for the subsequent human body part detection algorithm, which will lead to the failure of the overall algorithm;

Second, using the face detection method to locate the head area will have the problem of partial or complete occlusion of the face, resulting in the inability to detect. Moreover, face detection algorithms often only have high detection accuracy for frontal faces, and have high detection accuracy for side faces. The face effect is poor;

Third, the template matching method for human body part recognition and positioning will cause low accuracy problems. The human body parts shown in the video image will be due to factors such as scale changes and different clothing, which will cause the accuracy of the matching recognition algorithm to deteriorate, resulting in human body parts. A wrong part positioning makes the whole algorithm invalid.

Contents of the invention

The purpose of the present invention is to provide a human body posture recognition method in two-dimensional video images with high recognition accuracy and fast recognition speed.

In order to achieve the above-mentioned purpose of the invention, the present invention adopts the following technical solutions to achieve:

A human body gesture recognition method in a two-dimensional video image, said method comprising the steps of:

a. According to the principle of scale space layering, the original video image is Divided into Group, , is the resolution of the original video image;

b. For each group of video images, calculate a scale of The sampled image of , for One of the scales in represents the sampling function, Indicates the first group video images, , is the resolution of the original video image, is a set natural number greater than 1, indicating the number of sampled video images contained in each group of video images, ;

c. For the sampled images in each group Compute the HOG underlying feature descriptors separately ;

d. Based on the HOG underlying feature descriptor of a sampled image in each group obtained in step c, according to the prediction formula Calculate the inner scale of each group as in the rest ( ) The HOG underlying feature descriptor corresponding to the sampled video image of the scale, and Represent the sampled image respectively and the sampled image scale, is the set value;

E, according to the HOG bottom layer feature descriptor of all different scale sampling video images of step c and step d, in combination with the trained SVM, detect the human body target area in the original video image;

f, using the trained random forest classifier to classify the pixels of the human body target area detected in step e, and determine the body part area in the human body target area;

g. Connect the parts of the limbs determined in step f to form a human body contour to realize human body posture recognition.

Preferably, in the step b, using The end scale in Samples each group of video images, and calculates the sampled image corresponding to the end scale .

As mentioned above, the human body gesture recognition method in the two-dimensional video image, the random forest classifier in the step f is preferably trained by the following method:

Obtain artificially synthesized video images including human poses and real video images in the target test scene, and each video image is used as a training sample;

Mark the background area and human body target area in each training sample according to the set body parts;

Use the SURF operator to calculate the pixel features of each marked area, and all marked areas and their pixel feature data constitute a training data set;

Using the training data set and objective function Train a random forest classifier;

in, is a classification node of a decision tree in a random forest, is the weight, is the information entropy calculation function, is the pixel feature of the marked area in the artificially synthesized video image training sample, is the pixel feature of the labeled region in the real video image training sample, is the labeled first in the artificially synthesized video image training sample A statistical descriptor of the pixel features of a limb part, is the statistical descriptor of all pixel features in all labeled regions in the artificially synthesized video image training sample, is the statistical descriptor of all pixel features in all labeled regions in the real video image training sample, for and of distance.

Compared with prior art, advantage and positive effect of the present invention are:

(1) When the HOG multi-scale bottom-level feature extraction method is used to detect human targets from the original video images, only the HOG bottom-level feature descriptors of one sampled image need to be calculated in each group of sampled images after grouping, and the bottom-level feature descriptors of the remaining sampled images The character is obtained through feature prediction and calculation. On the basis of not reducing the detection accuracy, the calculation speed of the multi-scale underlying features is accelerated, and it fundamentally solves the problem of large amount of calculation and real-time performance that restrict the multi-scale human target detection method to practical application. Insufficient thorny problem.

(2) The random forest classifier is used to classify and identify human limbs. When the random forest classifier is trained, a new objective function is used to train the decision tree nodes in the classifier, which can generalize the weak classifier from the training sample space to the test sample space. still have a consistent spatial activation pattern. In this way, the training of the classifier can be completed by combining the human body posture video image samples artificially synthesized by computer graphics with a small amount of marked real human body posture videos to complete the training of the random forest classifier, thereby realizing the human body posture from artificial synthesis. The generalization of samples to real human pose features reduces the requirement for training samples.

Other features and advantages of the present invention will become clearer after reading the detailed description of the present invention in conjunction with the accompanying drawings.

Description of drawings

Fig. 1 is the flow chart of an embodiment of the human body posture recognition method in the two-dimensional video image of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

First of all, a brief description of the general processing ideas of the present invention to realize human gesture recognition:

Recognizing human body posture from two-dimensional video images is divided into two steps. The first step is to detect the human body target area from the original video image. , hands, elbows, shoulders, hips, knees, feet and other joints, and connect the limbs to form the outline of the human body, and then realize the recognition of human body posture. In the present invention, in the first step of detecting the target area of the human body, the HOG multi-scale bottom-level feature extraction method is used to reduce the influence of background, illumination, etc., and maintain scale invariance; and the bottom-level feature extraction method is improved to improve real-time performance. The second step is to use the random forest classification tree to identify human limbs and improve the classification accuracy; and improve the objective function in the random forest classification tree to improve the generalization ability of the classifier and reduce the complexity of the training samples required for classifier training Spend. For more specific implementation methods, please refer to the description below.

Please refer to FIG. 1 , which is a flow chart of an embodiment of the method for recognizing human gestures in two-dimensional video images according to the present invention.

As shown in Figure 1, the process of recognizing the human body posture in this embodiment is implemented by the following steps:

Step 101: Divide the original video image into multiple groups of images according to the spatial layering principle.

According to the principle of scale space layering, the original video image Divided into group, of which, , for raw video image resolution. The principle and method of layering video images according to the scale space is the prior art, and will not be described in detail here.

Step 102: Calculate a sampled image of a specific scale in each group, and calculate the HOG underlying feature descriptor of the sampled image.

Sampling each group of video images and calculating a scale as The sampled image of . scale for a particular scale, specifically, for One of the scales in . preferred, for The end scale in . in, represents the sampling function, Indicates the first group video images, , is the resolution of the original video image, is a set natural number greater than 1, indicating the number of sampled video images contained in each group of video images, . normally, The value of is 5-8, indicating that each group of video images includes 5-8 layers of sampled video images.

Then, calculate the HOG (Histogram of Oriented Gradient, histogram of directional gradient) underlying feature descriptor of the sampled image of the selected scale in each group. The method in the prior art may be used to calculate the HOG bottom layer feature descriptor, which will not be described in detail here.

Step 103: Calculate the HOG bottom-level feature descriptors of other sampled video images of a specific scale in each group through a prediction algorithm.

For each group of video images, a HOG bottom layer feature descriptor of a sampled image is calculated through step 102 . Then, based on the calculated HOG bottom-level feature descriptor, predict and calculate the HOG bottom-level feature descriptor of sampled video images of other specific scales.

Specifically, other specific scales refer to In addition to step 102 has calculated the scale of the HOG bottom feature descriptor ( ) scale. The following formula is used to predict and calculate the HOG underlying feature descriptors of sampled video images of other specific scales:

in, and Represent the sampled image respectively and the sampled image scale, , for the set value, for the sampled image The HOG underlying feature descriptor, for the sampled image The HOG underlying feature descriptor.

in, As a power exponent, it is a set value, which can be determined by fitting according to an empirical verification method. In this example, The preferred value of is 0.0042.

In the above formula, the exponent In order to determine the value, one of the scales and its corresponding HOG bottom-level feature descriptor are calculated in step 102, then, for another specified scale, the HOG bottom-level feature corresponding to the specified another scale can be easily calculated by the above formula Descriptor. By analogy, the HOG bottom-level feature descriptors corresponding to the remaining scales in the group can be easily calculated, so as to calculate the HOG bottom-level feature descriptors of all sampled video images contained in the group.

Step 104: According to the HOG underlying feature descriptors of all different scales of video images sampled, combined with the trained SVM, detect the human body target area in the video image.

By using the HOG underlying feature descriptors of the sampled video images included in all the groups calculated in step 102 and step 103, human body target regions at different scales can be detected. Using the HOG underlying feature descriptor and the trained SVM, the specific method for realizing the detection of the human target area can be realized by using the existing technology, and will not be described in detail here.

Step 105: Use the random forest classifier to classify the pixels of the human body target area to determine the body part area.

In step 104, after the human body target area is determined, the trained random forest classifier is used to classify the pixels of the human body target area, thereby determining the body part area. The input of the random forest classifier is the feature of the pixel, and the parameters of the selected classifier include the number of decision trees in the forest, the number of randomly selected attributes of internal nodes, and the minimum number of samples of the terminal node. The pixel features of the human body target area are used as The input parameters are input into the classifier, and the classifier will output the result of the body part area to which the pixel belongs, so as to determine the body part area. In this embodiment, a SURF (speed up robust features, fast robust gradient feature) operator is selected to calculate pixel features, and each pixel feature can be constructed as a 128-dimensional descriptor. The body parts area includes seven joint parts of the human body, namely: feet, knees, hips, shoulders, elbows, hands, and head.

Step 106: Connecting various body parts to form a human body outline to realize human body posture recognition.

Step 105: After determining the limb parts, connect the limb parts according to the head-shoulder-hip-knee-feet to form the torso, and then connect the elbows and hands on both sides, so that the outline of the human body can be identified, so as to realize the human body-based Human Pose Recognition for Joint Models.

In this embodiment, although the method of HOG low-level feature descriptor is used when detecting the human body target area, only the original video images are grouped, and the number of sampled video images included in each group is determined, that is, each group The number of layers in a group, in each group, only the HOG bottom layer feature descriptor of a sampled image is calculated using the bottom layer feature calculation function, and the HOG bottom layer feature descriptors of sampled images of other scales in the group are calculated using the prediction algorithm in step 103, and the calculation The complexity and amount of calculation are much smaller than the way of using the underlying feature calculation function. Moreover, using the prediction algorithm, there is no need to calculate the sampled video image corresponding to each scale, and the HOG underlying feature descriptor of the sampled video image is directly obtained, which further reduces the amount of calculation. Furthermore, the rapidity and real-time performance of HOG-based human target detection are improved, and the thorny problems of large amount of calculation and insufficient real-time performance that restrict multi-scale human target detection methods to practical applications are fundamentally solved.

In machine learning, a random forest is a classifier consisting of multiple decision trees. The main reason for its use in gesture recognition is the high classification accuracy. In addition, there are four factors. One is that its learning process is very fast; the other is that the complexity of the algorithm can be controlled by the depth of the internal decision tree; the third is When building a forest, it can generate an unbiased estimate of the generalized error internally; fourth, it has a good tolerance for outliers and noise, and is not prone to overfitting. But its main disadvantage is that the training data and test data are required to be similar, that is, both have the same distribution, which limits the generalization ability of the classifier. Therefore, to obtain a high-precision random forest classifier, the training samples are required to cover all possible changes in the future test data. However, in the actual test scene, it is impossible to obtain sufficient training samples due to factors such as changes in viewing angle, twisting of limbs, changes in texture of human clothing, and changes in illumination.

For the above-mentioned shortcomings of the random forest classifier, in the above-mentioned embodiments of the present invention, the objective function of the training decision tree node in the random forest classifier is improved, so that when the weak classifier generalizes from the training sample space to the test sample space Still have a consistent pattern of spatial activation. In this way, only some weakly labeled samples in the target test space are needed when selecting training samples, while other training data can be completed using human body pose video image samples artificially synthesized by computer graphics, thereby reducing the need for training samples. requirements. The specific training process is as follows:

Obtain artificially synthesized video images including human poses and real video images in the target test scene, and each video image is used as a training sample. Moreover, artificially synthesized video images are used as the main body, combined with a small amount of real video images in the target test scene with marked body parts and backgrounds.

The background area and human body target area in each training sample are marked according to the set body parts. Specifically, the target area of the human body is marked into eight parts according to the joint parts of the human body, one part is the background, and the remaining seven parts are: feet, knees, buttocks, shoulders, elbows, hands, and head.

The SURF operator is used to calculate the feature of each pixel in each marked area, and all marked areas and their corresponding pixel feature data constitute the training data set. Specifically, the SURF operator is selected to calculate the features of each pixel in each labeled region in the artificially synthesized video image training samples and the real video image training samples, and each pixel feature is constructed as a 128-dimensional descriptor. The pixel features of the labeled regions in the artificially synthesized video image training samples are denoted as , the pixel features of the labeled region in the real video image training samples are denoted as , and form the training data set, A classification node for a decision tree in a random forest. At the same time, calculate the statistical descriptors of all 128-dimensional SURF descriptors in all labeled regions of the artificially synthesized video image training samples and statistical descriptors of all 128-dimensional SURF descriptors in all labeled regions of real video image training samples .

Finally, using the above training data set and the improved objective function Train a random forest classifier. Among them, the improved objective function The expression is:

In the above formula, is the weight, which is an experimentally measured fixed value, preferably , the classifier has the best recognition effect. It is an information entropy calculation function, and the specific function expression adopts the prior art. is the labeled first in the artificially synthesized video image training samples Statistical descriptors of all pixel features within a limb part, for and of distance.

The objective function in the above expression not only takes into account the training sample entropy ( ), combined with the degree of information difference between the training data and the target test data ( ), the weighted sum of the two is used as the objective function of the training decision tree, thus improving the generalization ability of the trained classifier. When the trained classifier is used to identify body parts, a higher recognition accuracy can be obtained.

The above objective function uses The distance represents the degree of information difference between the training data and the target test data, but it is not limited thereto, and Euclidean distance or other distances may also be used to represent the degree of difference between the two.

The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art can still understand the foregoing embodiments. Modifications are made to the technical solutions described, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions claimed in the present invention.

Claims (3)

1. A method for recognizing human body posture in a two-dimensional video image is characterized by comprising the following steps:

a. dividing original video image according to scale space layering principleIs divided intoThe number of the groups is set to be,，the resolution of the original video image;

b. each set of video image samples is sampled and a scale is calculated asOf the sampled image，Is composed ofOf the number of the first and second dimensions,is shown asThe video images are grouped together to form a video image,，for the resolution of the original video image, in general,a natural number greater than 1 is set, which indicates the number of sampled video images included in each group of video images,；

c. for the sampled image in each groupSeparately computing HOG underlying feature descriptors；

d. C, based on the HOG bottom layer characteristic descriptor of the sampling image in each group obtained in the step c, according to a prediction formulaCalculating the inner dimension of each group asTo (1) rest of () The HOG bottom layer feature descriptors corresponding to the sampling video images of each scale,andrespectively representing sampled imagesAnd sampling the imageThe size of (a) is greater than (b),is a set value, and is used as a starting point,for sampling imagesThe HOG underlying feature descriptors of (a) are,for sampling imagesHOG underlying feature descriptors;

e. detecting human body target areas under different scales in the original video image according to the HOG bottom layer feature descriptors of all the sampled video images in the steps c and d and in combination with the trained SVM;

f. e, classifying the pixels of the human body target area detected in the step e by adopting a trained random forest classifier, and determining a limb part area in the human body target area;

g. and f, connecting the limb parts determined in the step f to form a human body outline, and realizing human body posture recognition.

2. The method for recognizing human body gesture in two-dimensional video image according to claim 1, wherein in step b, the method utilizesSampling each group of video images by the end part scale in the image processing system, and calculating the sampling image corresponding to the end part scale。

3. The method for recognizing human body posture in two-dimensional video image according to claim 1, wherein the random forest classifier in the step f is trained by the following method:

acquiring a manually synthesized video image comprising a human body posture and a real video image in a target test scene, wherein each video image is used as a training sample;

marking a background area and a human body target area in each training sample according to the set limb part;

calculating the pixel characteristics of each labeled region by using a SURF operator, wherein all labeled regions and pixel characteristic data thereof form a training data set;

using the training data set and an objective functionTraining a random forest classifier;

wherein,is a classification node of a decision tree in a random forest,as a weight value, the weight value,a function is calculated for the entropy of the information,is the pixel characteristics of the labeled area in the artificially synthesized video image training sample,is the pixel characteristic of the labeled area in the real video image training sample,is the labeled second in the artificially synthesized video image training sampleStatistical descriptors of pixel characteristics of individual limb portions,is a statistical descriptor of all pixel features in all labeled regions in the artificially synthesized video image training sample,is a statistical descriptor of all pixel features in all labeled regions in the real video image training sample,is composed ofAndis/are as followsDistance.