KR20000039192A - Gesture recognizing system and method by analyzing trace on polar coordinates - Google Patents

Abstract

PURPOSE: A gesture recognizing system and method in accordance with the trace analysis on polar coordinates is provided to pursue the motion of hand by using the natural color information included in the hand and to recognize the obtained information as the dynamic pattern. CONSTITUTION: A gesture recognizing system and method in accordance with the trace analysis on polar coordinates comprises a hand region extractor(11), a key-gesture spotter(12) and a gesture recognizer(13). The hand region extractor(11) separates the hand region from the video image inputted continuously and extracts the dynamic gesture. The key-gesture spotter(12) separates the key-gesture which is the meaning gesture from the dynamic gesture. The gesture recognizer(13) extracts the command is meant by the key-gesture. A gesture identification method in accordance with the track analysis on polar coordinates comprises the steps of;inputting the hand motion gesture image which means the specific command;separating the hand region from the hand motion gesture image and obtaining a track of gesture;separating the key-gesture which is the meaning gesture from the track of gesture;and extracting the command is meant by the key-gesture.

Description

System and method of recognition of dynamic gesture on the polar coordinate space

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a dynamic gesture recognition system and method by trajectory analysis on polar coordinates. Particularly, the dynamic characteristic of a gesture is extracted by projecting the trajectory of the dynamic gesture on polar coordinates, and the dynamic characteristic is hidden by the Markov as a double statistical model. The present invention relates to a dynamic gesture recognition system and method by trajectory analysis on polar coordinates to train and learn from a model.

Conventional gesture recognition system utilizes data gloves or colored gloves that convert hand movements into electrical signals, and handles the dynamic characteristics of gestures by treating the dynamic pattern of gestures as static information without considering temporal information. It was not available.

Accordingly, the present invention has been made to solve the problems of the prior art as described above, polar coordinates to track the movement of the hand using the natural color information of the hand in the natural environment, and to recognize the obtained information suitably in the dynamic pattern The purpose of the present invention is to provide a dynamic gesture recognition system and method by trajectory analysis of images.

In addition, the present invention provides a dynamic gesture by the trajectory analysis on the polar coordinates to increase the affinity of the person and the computer by making the connection technology flexible and intelligent by connecting and controlling the person and the computer or various devices using the gesture of the promised hand. Another object is to provide a recognition system and method.

1 illustrates a dynamic gesture recognition system by trajectory analysis on polar coordinates according to an embodiment of the present invention;

2 is a diagram illustrating a trajectory before and after planarization of a gesture trajectory according to an embodiment of the present invention;

3 is a diagram illustrating a key-gesture separation state of a gesture using a gesture trajectory and step-by-step speed information in time and space;

4 shows a trajectory of projected circle and square gestures on polar coordinates;

5 illustrates a left-right hidden Markov model utilized in a dynamic gesture recognition system according to an embodiment of the present invention;

6 is a diagram illustrating a user interface of an editing system utilized in a dynamic gesture recognition system according to an embodiment of the present invention.

※ Explanation of code about main part of drawing ※

11: Hand Area Extractor 12: Key-Gesture Separator

13: gesture recognizer

In order to achieve the above object, a dynamic gesture recognition system by trajectory analysis on polar coordinates according to the present invention includes a hand region extractor for separating a hand region from a continuously input video image and extracting a dynamic gesture;

A key-gesture separator for separating key-gestures that are meaningful gestures in the dynamic gestures, and

And a gesture recognizer for extracting a command meaning the key-gesture.

In addition, the dynamic gesture recognition method by trajectory analysis on the polar coordinates according to the present invention, the first step of inputting a hand gesture gesture image means a specific command;

A second step of separating a hand region from the hand gesture gesture image and obtaining a trajectory of the gesture;

A third step of separating a key gesture which is a meaningful gesture among the trajectories of the gesture, and

And a fourth step of extracting a command meaning the key-gesture.

In addition, the present invention, in a computer,

A first step of separating a hand region from the hand gesture gesture image and obtaining a trace of the gesture when a hand gesture gesture image representing a specific command is input;

A second step of extracting a key gesture which is a meaningful hand gesture by separating the trajectory of the gesture into a pre-gesture, a key gesture, and a trailing-gesture;

Polar coordinates of the key-gesture (ρ, φ) A third step of projecting onto the parameter space of and applying the clustering algorithm to quantify the characteristics of the key-gesture,

A fourth step of dividing the dynamic feature of the quantized key-gesture and applying it to a hidden Markov model to construct a learning and recognition model to extract a command meaning of the key-gesture; and

A computer-readable recording medium having recorded thereon a program for executing the fifth step of executing the command is provided.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a diagram illustrating a dynamic gesture recognition system by trajectory analysis on polar coordinates according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating a trajectory before and after flattening the gesture trajectory according to an embodiment of the present invention. 3 is a diagram illustrating a key-gesture separation state of a gesture using a gesture trajectory and step-by-step velocity information in space-time, FIG. 4 is a diagram illustrating a trajectory of a circle and a rectangular gesture projected on polar coordinates, and FIG. 5. Is a diagram illustrating a left-right hidden Markov model utilized in a dynamic gesture recognition system according to an embodiment of the present invention, and FIG. 6 is a user of an editing system utilized in a dynamic gesture recognition system according to an embodiment of the present invention. It is a figure which shows an interface.

The dynamic gesture recognition system by trajectory analysis on polar coordinates according to the present invention includes a hand region extractor 11, a key-gesture separator 12, and a gesture recognizer 13 as shown in FIG. 1.

The hand region extractor 11 separates the hand region from the continuously input video image. The research on tracking an object from a video image has been actively conducted in the field of image processing, but it is not generally applicable in all cases, and it is common to apply an algorithm suitable for the research environment.

In order to track a moving object, a moving object is tracked by differential image information between frames of a video image, or a priori knowledge about color, position, size, etc. of attribute information of moving information is obtained. If so, a method of tracking the moving object based on these information, or using the former and the latter simultaneously.

That is, the hand region extractor 11 extracts the hand region continuously by applying the prior knowledge about the color information, the size of the hand, and the movement of the hand, to obtain the trajectory of the gesture, and the key gesture separator 12. Separates key-gestures by state transition information.

In order to extract the hand region, a model of the color of the hand should be obtained. The RGB gesture image inputted to the hand region extractor 11 is converted into YIQ image information, and the IQ information representing the color information is extracted from the model, that is, Create a Look-Up table. In order to extract a hand region from an input gesture image, an area having a color similar to that of a hand model must be found. In this case, if the gesture image is processed in pixel units, the processing cost is high, so that the gesture image is divided into cells having a predetermined size. That is, the size of one cell is 4 x 4 pixels, and the color information of each pixel in the cell is registered in the built-up look-up table, and the degree of similarity to the model color, that is, the similarity (similarity) is determined. It is obtained using Equation 1.

In Equation 1, H ^m is a histogram value of the model, and H _k is a histogram value of the k-th cell of the input image. S _k is the similarity with the hand model of the k-th cell of the input image, which is expressed as a value between 0 and 1, and when the calculated value does not have a certain value or more, that is, when S _k <S _th , Considering (noise) makes the similarity zero and removes it.

After removing the noise, one hand region is extracted by using prior knowledge such as dilation, binarization of the candidate region, and position and size information among the remaining candidate regions.

The extracted hand region information includes information regarding the shape of the hand. In general, it is known that a change in the shape of a hand while the hand is moving has no meaning, and since the vocabulary of the gesture of the present invention is a shape of the trajectory itself, the gesture can be recognized by tracking and analyzing the movement trajectory. have. Therefore, when considering the center point of the hand area as a token, the gesture trajectory is represented as a set of center point tokens.

If this center coordinate is used as it is, when the state of hand region extraction is not stable, unlike the actual trajectory, the output state of the trajectory may show a sudden change on the way. Go through. FIG. 2A is a view showing a state before the flattening step of the extracted trajectory only by tracking the center coordinates, and FIG. 2B is a view showing a state after the flattening step.

The trajectory thus obtained is represented as a set of tokens of the points having the time information in the two-dimensional space. Therefore, the information of the gesture that we can actually obtain is expressed as Equation 3, which means that the gesture is represented as a token set of trajectory coordinate points. Since the gesture information obtained is the information of the trajectory according to the change of time, it contains dynamic information such as the absolute coordinate, the moving speed and the angular velocity, the moving direction, and the moved absolute distance according to the change of time, and the information includes the characteristics of the dynamic gesture. It is used for recognition as an element.

Next, when we look at the key-gesture separator, the gesture in the broadest sense includes all the hand gestures that a person creates, but the gesture we want to deal with refers only to actions that have a narrow meaning. Meaningful gestures make up part of a human's hand movements, so they must be recognized separately. Human gesture recognition mechanism can extract and recognize only meaningful key-gestures very flexibly even when meaningful and meaningless parts are mixed. In other words, meaningless hand gestures are difficult to learn and recognize by defining them as patterns with varying forms and a wide range of changes, and they are bad for recognition results of key-gestures due to meaningless hand gestures even when learning within a limited category is possible. It is important to recognize key-gestures, ie, meaningful hand gestures, separately.

According to the analysis of psychologist Kendon et al., One gesture is generally divided into three stages: preparation, nucleus, peak, or stroke, and retraction.

Also, based on this separation concept, Queck et al. Proposed the following rules. First, the gesture starts to move slowly from the stop state, passes through a state where the speed is gradually increased, and ends at the stop state again. Second, the hand is assumed to be of a particular shape at the stroke stage. Third, low speed is not a motion in the stationary state. Fourth, the gesture made by the hand gesture is formed in a certain spatial region. Fifth, static gestures require a certain amount of time for recognition. Sixth, repetitive movements can also be gestures. There are also some exceptions, but the movement of the finger makes sense when the hand is at rest.

Based on the above theory, the key-gesture separator 12 classifies gestures into three types: pre-gesture, key-gesture, and post-gesture. Then, the hypothesis that there is a short pause action between gestures in each step is set so that the key recognizer 13 recognizes only the key gesture by separating the key gesture.

In other words, the key-gesture separator 12 finds a portion in which the state where there is almost no change in speed between the three gestures above at the beginning and the end of the gesture is found at the beginning and the end of the gesture, and the time points t _pre and It is called t _post . At this time, the key-gesture is determined when the length of the separated part by Equation 5 is greater than or equal to the threshold value.

FIG. 3 is a diagram illustrating a shape in which a position coordinate appears on a time axis by using a circle gesture as an example. The gesture is divided into a pre-gesture step, a key-gesture step, and a post-gesture step. There is a moment in which there is little movement of the gesture between the changing parts, in which the key-gesture is separated.

The key-gesture separator continuously retrieves the velocity information and separates the key-gesture by spattering in the part where the state of the speed change is almost continuous.

Next, finally, the gesture recognizer 13 will be described. The gesture trajectory information analysis extracts and quantifies the dynamic information of the trajectory to learn and recognize on the Hidden Markov model. The obtained trajectory information is tokens of hand position information over time, and includes information such as direction and velocity displacement along with time position information.

However, since the trajectory information is severely displaced in size and shape to the trajectories generated in front of the camera that does not consider the three-dimensional displacement, that is, on the virtual two-dimensional space, normalization capable of absorbing them is necessary. Polar coordinates of gesture position information on a two-dimensional plane (ρ, φ) Polar coordinates after projecting onto the parameter space of (ρ, φ) Quantify features in space. The reason for this is (ρ, φ) This is because the value easily reflects the information of the trajectory change from the center point of the gesture and the quantification of the size change is easy, and thus Equation 6 is used for the conversion.

In the above expression Is the center coordinate of the gesture trajectory, and r _max is the radius of the furthest point from the center point of the position of the gesture trajectory. θ Represents the angle from the baseline. A baseline is placed every π / 2 in the diagonal direction to flexibly handle where the gesture first starts. That is, the angle is calculated based on four reference lines for all sections. This is to improve flexibility by considering the same baseline even when the gesture starting in the same section has a slight difference at the starting point because the starting point of the gesture is different for each person.

4 shows a gesture representing a circle and a square ρ-φ It shows the result of projection in polar space. ρ-φ The tokens of the gesture projected on the polar space are clustered by the LBG algorithm and converted into symbols that are codewords. this ρ-φ The feature space is divided into ^2M pieces and used as input information of the Hidden Markov model shown in FIG. 5, where the type of Hidden Markov model utilized is left-right to allow only forward of information flow for efficient processing. It is used as a model.

The present invention provides a user interface of the editing system as shown in FIG.

The present invention as described above is recorded on a computer-readable recording medium and processed by a computer.

As described above, according to the present invention, since the remote control of the home appliance or the computer or various industrial devices can be controlled by the gesture, the human friendliness of the machine is increased.

Claims (9)

A hand region extractor for separating a hand region and extracting a dynamic gesture from a continuously input video image; A key-gesture separator for separating key-gestures that are meaningful gestures in the dynamic gestures, and And a gesture recognizer for extracting the command meaning of the key-gesture. The method of claim 1, wherein the gesture recognizer, Project the key-gesture onto the parameter space in polar coordinates, ρ-φ A dynamic gesture recognition system using a trajectory analysis on polar coordinates by dividing a feature space and applying it to a hidden Markov model, and extracting a symbol that is a codeword by applying the gestures projected on the polar coordinates to an LBG algorithm. A first step of inputting a gesture gesture image representing a specific command; A second step of separating a hand region from the hand gesture gesture image and obtaining a trajectory of the gesture; A third step of separating a key gesture which is a meaningful gesture among the trajectories of the gesture, and And a fourth step of extracting a command meaning the key-gesture. The method of claim 3, wherein the second step, A first sub step of converting the hand gesture gesture image input as RGB information into YIQ image information; A second sub step of creating a lookup table using I-Q information among the YIQ image information; A third sub step of dividing the hand gesture gesture image by a predetermined cell unit; A fourth sub-step of extracting the center point of the hand region as a token by examining the similarity between the color information of the arbitrary cell stored in the lookup table and the model color, and And a fifth sub-step of obtaining a gesture trajectory in space-time by repeating the second sub-step to the fourth sub-step on the continuous hand gesture gesture image. The method of claim 4, wherein the second step, And a sixth sub-step of flattening the extracted gesture trajectory. The method of claim 3, wherein the third step, A first sub-step of extracting a portion (t _pre , t _post ) in which a state where there is almost no change in speed continues at a threshold or more at the beginning and the end of the trajectory of the gesture, respectively; And a second sub-step of _pre -gesturing a part before t _pre based on the extracted two parts, key-gesture a part between t _pre and t _post , and extracting a part after t _post as a post-gesture. Dynamic gesture recognition method by trajectory analysis on polar coordinates. The method of claim 3, wherein the fourth step, Polar coordinates of the key-gesture (ρ, φ) The first sub-projection onto the parameter space of The polar coordinates (ρ, φ) A second sub-step of quantifying the feature of the key-gesture in space, and And a third sub-step of dividing the dynamic features of the quantified key-gestures and applying them to a hidden Markov model to construct a learning and recognition model. The method of claim 7, wherein the second sub-step, Dynamic gesture recognition method by trajectory analysis on polar coordinates, characterized in that the step of quantifying the characteristics of the key-gesture by the statistical method using the LBG algorithm. On your computer, A first step of separating a hand region from the hand gesture gesture image and obtaining a trace of the gesture when a hand gesture gesture image representing a specific command is input; A second step of extracting a key gesture which is a meaningful hand gesture by separating the trajectory of the gesture into a pre-gesture, a key gesture, and a trailing-gesture; Polar coordinates of the key-gesture (ρ, φ) A third step of projecting onto the parameter space of and applying the clustering algorithm to quantify the characteristics of the key-gesture, A fourth step of dividing the dynamic feature of the quantized key-gesture and applying it to a hidden Markov model to construct a learning and recognition model to extract a command meaning of the key-gesture; and A computer-readable recording medium having recorded thereon a program for executing the fifth step of executing the command.