KR100399530B1 - method for recognition an object in video image - Google Patents

Abstract

The present invention relates to an object recognition method of a video image. According to the present invention, a representative feature of a moving image is extracted through a method of statistical weight recognition using a Poisson non-zero probability distribution for each section and a method combining fuzzy and neural networks. The representative features thus extracted are used for game or sports analysis, crime prevention or human or animal behavior analysis.

Description

Method for recognition an object in video image}

The present invention relates to object recognition of a video image, and more particularly, to an object recognition method of a video image that can recognize a feature of an operation more accurately.

As data storage devices, image processing technologies, and communication data compression technologies, which can store various data without loss, have been highly developed, information processing technology using video has recently been in the spotlight. These video information processing technologies contain huge amounts of spatial and temporal data and provide more information than the amount of data provided by text, graphics and images. The location, distance, and temporal / spatial relationships of objects are implicitly included in the video data, and in areas such as entertainment, visual communication, multimedia, education, medical, motion prediction, scientific research, and sports, image sequences are subject to the use of television and VTR. It is also increasing. The image and video sequences are the main data to be processed in the future computer vision and multimedia environment, which is an indispensable requirement as the computer industry develops.

As the computing environment develops rapidly, computers are being applied to the visual field, which is a unique area of humans and living things. In other words, the computer analyzes and recognizes objects just as humans analyze and recognize objects by giving them visual capabilities. However, the visual ability of such a computer is incomparably low compared to the ability to analyze and recognize objects through the human eye.

On the other hand, the development of semiconductors, computer hardware, software and so on, and the development of society as a whole, requires the development of the computer vision field, but the computer vision field is still in the beginning stage. Currently, research is being conducted in the field of computer vision mainly in universities and research institutes, but few systems are commercialized except for simple data analysis and recognition systems.

On the other hand, the study of moving objects is showing more and more attention in the sports field, and the research is getting wider. M.I.T's Stephen S. Intille and others described the problem of object tracking in complex and dynamic environments. They have presented and analyzed how Cross can also be used to track American football players by providing an analysis method and context feature tracking method. However, they did not understand their motion, but rather the analysis of object motion in American football, which is natural, fast and difficult to predict the direction of motion. Elisabeth Andre also discussed simultaneous automatic speech generation for the appropriate combination of progressive event recognition and language generation through the SOCCER system, a football analysis system. Chueh Wei and Suh Yin Lee described several important methods in the video information system for sports motion analysis. First, we explained how to efficiently calculate the position of moving objects in video sequences, and used them for object tracking and feature extraction. Second, we proposed a matching mechanism to compare the difference of object motion in two video sequences. However, these papers also describe the tracking of moving objects and comparative papers of the tracked objects, but they have strong characteristics of motion analysis, but they do not recognize human motion. Dennis Yow et al. Proposed a technique to analyze video image content to automatically discover soccer highlights, and to represent shot motion by reconstructing selected events into panoramas. However, the method relates to the tracking and retrieval of an object, including the recognition of football features, tracking the ball, and the composition of camera views and the composition of the panoramic view for efficient recognition.

On the other hand, a study on the quantitative qualitative representation of group behaviors in soccer games was also attempted. Toshio Kawashima et al. Proposed a qualitative analysis of group behaviors based on multi-scale domain analysis, and Tsuyoshi Taki et al. Was implemented. The purpose of this system is to quantitatively evaluate teamwork based on the movements of all players in the game, and to analyze group behaviors rather than individual behavior analysis in the football field.

On the other hand, as a result of research on human motion recognition, Koh Kakusho et al. Discussed the method of recognizing the type of feed dancing from real image sequence and sound signal of music. They tracked the strength and sound of the music when they danced and recognized the type of dancing by tracking the mock rotation and the position and direction of the two dancers during the dance. However, this paper deals with the recognition of dancers' steps, positions, and types of music, rather than the recognition of human motion itself. Lee Campbell and Aaron Bobick developed a system that recognizes traditional ballet motion from tracking data on the XYZ axis. In the lower space of the phase space it has the space, the position of the body, and the axis of posture. Also, the axis of the lower space is a subset of the axis of the phase space. They recognized nine traditional ballet moves performed by two dancers. This paper is similar to the motion recognition of moving objects. However, this paper is far from motion recognition of moving objects in that it recognizes fixed object positions, cameras and standardized motion. James W. Davis expresses human moving real-time computer vision approaches based on patterns of motion. This paper proposes a motion history image (MHI) method characterized by local multiple histograms in the direction of motion. This method is a good way to recognize simple motion in real time indoors, but it is not suitable for motion recognition of objects moving over a wide range. Jia Ching Cheng recognized the motion of a moving person in the video footage. They proposed a model-based recognition method in a dynamic environment. They searched for people, marked their positions, expressed human body and walking as modeling elements, and compared moving elements to modeling elements. But this paper only recognizes human walking in a 30-frame video image of walking straight. Yaser Yaccb proposed and tested a parametric model for behavioral modeling and recognition when a large number of transient parameters were extracted from an image sequence. They used a PCA (Principal Component Analysis) linear transformation technique to walk, march, line, walk, and recognize language according to kick and lip changes. Although the background of the data used in this paper is natural, and the recognition range such as walking is quite similar to ours, this paper recognizes the standardized movement in the room and the object movement is predictable and Speed also doesn't show a big difference in motion.

On the other hand, researches on such motion analysis have been conducted in Korea, and Pohang University has performed work on football game analysis. They developed a method for tracking players and balls for automatic soccer analysis, and built player and ball trajectories on field models. Then, they proposed a method of finding the three-dimensional position of a soccer ball from a black and white image sequence of a soccer game. The height transition of the ball was easily calculated using a simple triangular geometric relationship to the given start and end positions of the ball on the ground.

As mentioned above, the recognition of moving objects is of great interest to many researchers interested in gesture recognition. However, the above-mentioned studies recognize a stereotyped movement under a limited background. In the natural real-world environment, there are not many cases of object movement recognition in a form where it is difficult to predict a moving direction of an object. In addition, motion recognition in the freely moving sports area is extremely rare. As a result, image analysis is performed by analyzing the motion of a person through an object recognition technology that is close to the real person's eye. There is an urgent need for technology to secure technology and develop programs in a wide range of fields, and to expand them into applications such as games, sports analysis, crime prevention, and analysis of human or animal morphology.

Accordingly, an object of the present invention is to provide a method for object recognition of a video image, which can solve the above problems.

Another object of the present invention is to provide an object recognition method of a video image which can recognize a feature of an operation more accurately.

In order to achieve the above object, the present invention provides a method for object recognition of a video image, comprising: defining an operating range of a moving video image; Extracting representative features of a video image moving according to the defined motion range; It provides a method for object recognition of a video image, comprising the step of distinguishing the motion using the weighted recognition method through the Poisson non-zero probability distribution for each interval and the method of combining fuzzy and neural networks. .

1 is a conceptual diagram illustrating a method for analyzing a soccer game video according to an embodiment of the present invention.

2 is a graph showing a change in leg angle and a distance between a ball and a player indicating a dribble state.

3 is a graph showing the change in leg angle and the distance between the ball and the player showing the running state.

4 is a graph showing the change in leg angle and the distance between the ball and the player showing the walking state.

5 is a graph showing the change in leg angle and the distance between the ball and the player showing the standing state.

6 is a graph showing a change in leg angle and a distance between a ball and a player indicating a kick state.

7 is a graph showing the change in leg angle and the distance between the ball and the player showing the kick state after dribbling.

8 shows an overall flowchart of motion recognition.

9 shows a recognition process through non-zero probability distribution.

10 to 12 show fuzzy sets using standard Gaussian random variables.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In the present invention, it is difficult to predict the motion of an object in a natural and dynamic environment, and sometimes the motion of the object which is very fast is analyzed and the recognition of the motion is explained and implemented. In addition, the present invention proposes a method for analyzing and recognizing the motion of a moving object in a natural and dynamic situation, and extracts the representative features of the motion by classifying and analyzing the motion to recognize the motion of the moving object. It also represents the correlation with other objects coming around the object. In addition, we propose a motion recognition algorithm using a combination of Poisson non-zero probability distribution, fuzzy sets, and neural networks.

1 is a conceptual diagram illustrating the present invention, illustrating a method of analyzing a soccer game video.

Referring to the figure, the football game analysis (1) is composed of video scene change search and indexing (2), multi-object extraction and representation (3), panorama synthesis (4) process. Soccer games are now one of the most widely known sports on the planet. In particular, Korea is supposed to host the 2002 World Cup football, so video analysis will provide a lot of interest in football games. The process of extracting and representing the moving object (3) includes the recognition of the moving object and the soccer game formation in the soccer game. Moreover, motion recognition and formation recognition are fundamental elements of football game analysis.

In general, the movement of a person in a dynamic environment is not generated at a predetermined time, and the start and end of a motion are not divided at regular time intervals and are variable according to each motion. The dynamic operating region is thus defined as a variable region at the beginning and end of an operation. In addition, representative features of the operation may be defined to distinguish the operation in the defined dynamic domain. Representative movements in football can be divided into six categories such as dribbling, running, walking, standing, kicking, and kicking after dribbling. Such representative movements can be classified into changes in leg angles below the waist. And, the recognition of soccer movement is the average of leg angle, average of deviation of leg angle between frames, maximum of leg angle, maximum distance of leg angle, distance between person and ball when leg angle is maximum, The maximum distance, and the number of times a person encounters a ball, is defined as a representative feature in the video scene of a motion.

2 to 7 are graphs showing the change of the leg angle L1 and the change of the distance between the ball and the player L2. The X axis represents the frame, the Y axis represents the leg angle, and the distance between the player and the ball. Here, the change of the leg angle (L1) is a measure to know the speed of the athlete, so that the wide angle of the leg suggests that the athlete's speed is fast.

First, Figure 2 is a dribble state, because the player runs at a constant speed and owns the ball several times the distance between the player and the ball is zero.

3 is a running state, it can be seen that the leg angle is changed quickly and constant.

4 is a walking state, it can be seen that the change in the leg angle is slow compared to FIG. 3 showing the running state.

5 is a standing state, it can be seen that there is little movement of the legs of the athlete. In other words, whether the athlete is spreading legs or not, the small variation in the angles of the legs is a clue that the athlete is hardly moving.

6 is a kick state, when the maximum value of the two leg angles is large, it is a hint for the probability of jumping quickly, even when the ball is strongly kicked, the two leg angles are temporarily widened. In order to kick the ball, the player must temporarily lift one leg back, with the leg angle usually at maximum in one scene. Also, in order to kick the ball, the ball must be attached to the player, which is a clue to kicking the ball.

7 is a kick state after dribbling, and it can be seen that the ball flew away after meeting the player several times.

In the above, the typical features of the motion are extracted by classifying and analyzing the motion of the moving object.

On the other hand, the number of events that occur during a given time (or space) can be analyzed by the Poisson distribution. The probability variables that satisfy the following two conditions follow the Poisson distribution. First, the probability of event occurrence at the same given time (or space) is the same. Second, the occurrence or non-appearance of an event at any given time is independent of the event or non-appearance of another given time. Looking at the Poisson distribution equation is as follows.

f (x) = (u ^x e ^-u ) / x!

(x = 0,1,2,3,4 ...)

Where x is a random variable, u is the average number of occurrences of a given time, e = 2.71828.

Meanwhile, Gaussian probability density function is as follows.

f (x) = (1 / σsqrt (2π)) × e-a

(a = (xu) ² / 2σ2,-∞ <x <∞

When the random variable x follows the Gaussian distribution with mean u and standard deviation σ, it is expressed as x to N (u, σ ² ). For this random variable x, z is followed by a Gaussian distribution with mean 0 and standard deviation 1. In other words, z to N (0, 1).

Z = x-u / σ

This random variable z is called the standard normal probability variable, and this distribution is the standard best distribution.

8 shows a motion recognition process applying the method using the Poisson non-zero probability distribution for each section and the method using a fuzzy set and a neural network.

Referring to the drawing, in step 100, a continuous scene on a video is captured. In step 102, a subject is identified, and in step 104, a subject is tracked. In step 106, the shape is extracted, in step 108, a statistical weight is extracted through a non-zero Poisson probability distribution, and in step 110, the result is recognized in step 110.

Meanwhile, in step 114, a fuzzy set is used through a Gaussian distribution and a distribution of various possibilities. In step 116 it is recognized in step 118 through the BP algorithm of the NNs, and the result is obtained in step 112.

On the other hand, as one of the statistical weight recognition method, we propose a recognition method through the non-zero Poisson probability distribution for each section. The weighting method is a method of assigning a weight to each extracted feature value, calculating a weight for each recognition operation, and distinguishing the operation according to the final result value. In other words, the event occurrence space of the Poisson distribution is classified into 10 units (but the number of encounters between the player and the ball is divided into 1 unit). For example, if the average leg angle in the run is 46.7, the run is performed. Will be distributed in the fifth. Based on these distributions, the Poisson probability distribution function is used to find the probability that an event will be zero in a given interval according to each distribution. If the probability of being 0 is a, the probability of being greater than 0 is (1.0-a). This value is the weight. For example, if the Poisson probability distribution is 0.2 with a leg leg average of 40.0 and a distribution between 40.0 and 0, then the probability of 1 or more is 0.8 and the weight is 0.8.

If u is the mean value of the y interval (eg, if the average leg angle of the run is 45.7, then 40 <= y <50, that is, the interval is the fifth of the run), then the probability of 0 in the Poisson distribution is Expressed as a function

f (x, y, u) = (u ^x e ^-u ) / x!

x = 0, u is the mean value in the y interval

Therefore, the weight function in the y interval according to the above function is expressed as follows. This is the non-zero Poisson probability distribution for each interval.

W (y) = 1.0-f (x, y, u)

If Min (f (x, y, u)) is defined as the minimum value of f (x, y, u), (y> 1), the maximum distance between the player and the ball since the player and the ball last met, and the player and the ball met The weight function of the number of times is expressed as follows.

W (y) = 1.0-Min (f (x, y, u)), (y> 1)

When W (y) is obtained, the result is that the largest value is obtained by multiplying the target value by the weight of each input value.

9 is a flowchart illustrating a recognition process through non-zero probability distribution.

Referring to the drawing, in step 200 a representative feature is extracted, and in step 202, an event occurrence section is defined. In step 204, the non-zero probability distribution is extracted through the Poisson distribution, and in step 206, each target value is weighted. In operation 208, the recognition result is extracted.

On the other hand, as another method of statistical weight recognition method, we propose a fuzzy set using standard Gaussian random variables and a recognition method using designed fuzzy sets and neural networks. The neural network uses the back propagation algorithm and uses the fuzzy membership values inferred through the fuzzy set as inputs to the neural network to perform motion recognition. We use the Gaussian distribution function as the membership function to create the fuzzy set. 10 to 12 show fuzzy sets using standard Gaussian random variables.

Here, the recognition feature values are defined as small (S), medium (M), and large (L). In addition, small and medium, middle and large are each expressed as small as SM and have an intermediate value as MS, and as medium as ML and as large as LM. In other words, the average value is not near the mean value and smaller than the mean is defined as small, the value around the mean value is defined as the middle, and the value is not near the mean and the big value is defined as large. However, the distance between the player and the ball at the maximum angle is defined as small (S) and large (L). When the fuzzy set follows a Gaussian distribution, the mean is zero of the standard probability distribution. Therefore, X is a random variable, Z is a standard normal probability variable, and the standard standard normal random variables for S, SM, MS, M, ML, LM, and L are called Z ', Z' ', and the random variable X is obtained. If the function is f (Z), the random variable function to find the random variable X is expressed as follows.

f (Z) = u + z * σ, (-3.0 <z 3.0 <z '± z`` <3.0, z'> = z '')

If (z <=-(z '+ z' ')), S,

If (-(Z '+ Z' ') <Z = <-(Z'-Z' ')), SM, MS,

If (-Z '' <Z = <Z '') M,

If (Z'-Z '' <Z = <Z '+ Z' '), ML, LM,

If (Z> Z ''), then L (see Figure 10).

The degree of SM and MS is expressed as follows.

SM = (u-z '* σ) -f (z) / {u- (z'-z' ') * σ}-{u- (z' + z '') * σ}

MS = 1.0-SM

In addition, the degree expression of ML and LM is as follows.

ML = (u + (z '+ z' ') * σ} -f (z) / {u + (z' + z '') * σ}-{u + (z'-z '') * σ}

LM = 1.0-ML

The fuzzy set according to the distance between the ball and the player at the maximum angle is as follows.

Likewise, conditions appear differently (see Fig. 11).

If (Z <= 0) then S,

If (0 <Z = <Z ') SL, LS

L if (Z '<Z).

The representation of the degree of SL and LS is as follows.

SL = u-f (z) / z '* σ

LS = 1.0-SL

The conditions of the number of times the ball and the player met are as follows. If the number of meetings is X,

If (X <= 0), S,

If (0 <X <2) M,

L if (X> = 2). (See Figure 12)

As a result of experiments combining the statistical weighting method through the Poisson non-zero probability distribution for each section and the fuzzy and neural networks, as shown in Table 1 below, recognition results of 87.1% and 93.8% were obtained, respectively.

Recognition rate Statistical weight Fuzzy neural network dribble 95% 97.5% jump 82.5% 85% walking 100% 90% Standing 100% 100% kick 70% 97.5% Dribble after kick 75% 92.5% system 87.1% 93.8%

The results shown in Table 1 show that the method using fuzzy and neural networks showed more errors than statistical weights in walking, but significantly reduced errors compared to statistical weights in recognition of kicks after kicking and dribbling.

In addition, there were many cases of misrecognition as walking, because the athlete ran very slowly, so the leg angle did not open much and appeared similar to the case of walking a little faster. And most of the misunderstanding of walking as standing is when walking slowly, most of which was recognized as standing. This is because the average deviation of leg angles, which is an important feature of standing, is clearly distinguishable.

In addition, dribbling, kicking, and kicking after dribbling were not recognized by misunderstanding each other as a whole. The distinction between dribbling, kicking, and dribbling kick is the distance between the ball and the player when the leg angle is maximum, the distance between the ball and the player since the player and the ball last met, and the number of times the ball and the player met. Because the kicks are all correlated with the ball, their correlations were not clear.

In conclusion, the overall error is not even biased to a specific motion, and the experimental results shown in Table 1 show that the combination of fuzzy and neural networks is superior to the method using the statistical weight.

What has been described above is only one embodiment, and the present invention is not limited to the above-described embodiment, and those skilled in the art without departing from the gist of the present invention as claimed in the following claims Anyone can make changes.

As described above, in the present invention, in order to define a motion that is difficult to predict or move quickly in a natural environment of the real world, such as a video recorded in a video, a range of moving motions is defined, and a dynamic motion region of one motion is defined. Extract representative features from. Then, the representative features of the dynamic motion domain are extracted using the weight recognition method through the Poisson non-zero probability distribution for each section and the method combining fuzzy and neural networks. The representative features thus extracted are used for game or sports analysis, crime prevention or human or animal behavior analysis.

Claims (3)

In object recognition method of video image: Defining an operating range of a moving video image; Extracting representative features of a video image moving according to the defined motion range; And classifying the extracted representative features by using a weight recognition method using a Poisson non-zero probability distribution for each section and a method combining a fuzzy and neural network. The object recognition method of claim 1, wherein the representative feature is a specific operation type included in a video image that is an object of object recognition. The method of claim 2, wherein the representative feature is, for example, a change in leg angle and a change in distance between a player and a ball in a soccer game.