CN103955699A - Method for detecting tumble event in real time based on surveillance videos - Google Patents

Abstract

The present application discloses a real-time fall event detection method based on monitoring video. In the detection scene, multiple cameras facing the same target area and with different shooting angles are installed, and the multiple cameras continuously shoot the target area, including the following steps: The camera shoots a video of the target area at the same time; from multiple videos of the same period shot by multiple cameras, extract the foreground image representing the target in each frame; extract the foreground image of the same target at the same time by multiple The respective shape and position features in the pictures captured by the camera, and use the RVM classifier to determine the target attitude category at the moment corresponding to each frame; the obtained target attitude category of each frame is input as a sequence of target attitude values Go to the HMM evaluator to obtain the posterior probability of the change of the target posture category, which represents the occurrence of the target fall event; if the posterior probability is greater than the predetermined threshold, it is determined that the fall occurred.

Description

A real-time fall event detection method based on surveillance video

technical field

The present invention relates to the field of image pattern recognition, more specifically, to a real-time fall detection method based on RVM and HMM.

Background technique

Because the research on fall detection has high theoretical significance and practical value, relevant research results and products have been published at home and abroad. According to the methods used in fall detection, detection technologies can be divided into three categories: wearable instrument detection methods, environmental device detection methods, and surveillance video detection methods. The first two technologies are sensor-based methods, and the latter technology is based on methods of image processing.

In the instrument-worn detection, the user needs to wear some instruments equipped with sensors or other devices to help the system obtain the user's movement information and body movement information. The system detects falls by classifying the collected information. Literature In [1], the speed and acceleration sensors are used to judge the sudden stillness of human body movements to achieve the purpose of detecting falls.

The detection method of wearable instruments is simple and easy to implement, but the main problem is: since the sensor-related parameter thresholds are set according to the precise relative positional relationship between the instrument and the wearer, once this relationship is destroyed (in fact, it often happens) , such as strenuous exercise or putting on and taking off clothes, a large number of false detections will be generated; in addition, since the user is required to wear the instrument, it will bring greater discomfort and inconvenience to the user.

The environmental monitoring instrument method mainly collects various data related to the user's human body through various sensors placed in the environment, and judges whether there is a fall event through data analysis. In the literature [2], Alwan et al. used the shock sensor installed on the floor to judge the fall; the Technical Solutions Australia system [3] collects the pressure information exerted by the user through the "get out of bed" alarm, the floor mat alarm, etc. , and judge the user's posture by analyzing the collected data.

Similar to the wearable instrument detection method, this method is also susceptible to false detection due to other interference in the environment; although it saves the user from the trouble of wearing the instrument, the complexity of the system has increased due to the addition of a large number of sensors.

Surveillance video detection, that is, computer vision detection, judges whether there is a fall event by analyzing the video data in the monitoring environment in real time. This method can be further subdivided into three types: (1) Static detection. Usually, people who fall will lie still on the ground for a period of time. Based on this assumption, Nait-Charif and McKenna [4] used a wide-angle lens mounted above the user's head to obtain the user's movement trajectory to detect falls. abrupt termination of the trajectory. (2) Body shape change detection. During a fall, the faller's body shape usually changes significantly, such as changing from standing to lying flat. Based on this principle, Ganapathy et al. [5] used the aspect ratio of the circumscribed rectangle of the human body and the inclination angle of the circumscribed rectangle as gesture features, and judged the change of the human body shape by analyzing the change of the feature value, and then detected whether there was a fall event. (3) Head movement/position detection. In this method, researchers detect the occurrence of falls by detecting the human head and tracking the trajectory of the head or locating the relative distance between the head and the ground. Shoaib et al. [6] detected the human head by ellipse fitting, and used the simulated Gaussian distribution scene ground information to calculate the distance of the head relative to the ground and judge whether a fall occurred.

In surveillance video detection methods, most studies only use a single motion feature or pose feature, so it is easy to cause a large number of false detections. On the other hand, the relevant literature does not consider dealing with falling events along the direction of the camera. In such cases, the shape of the falling person is similar to that of the standing person, and it is difficult to distinguish the two poses based on pure appearance features. .

List of references mentioned above

[1] Almeida, O., M. Zhang, and J.C. Liu. Dynamic fall detection and pacemeasurement in walking sticks. IEEE Joint Workshop on High Confidence Medical Devices, Software, and Systems and Medical Device Plug-and-Play Interoperability, 2007.

[2]Alwan,M.,et al.A smart and passive floor-vibration based fall detector forelderly.IEEE2nd Conf.on Information and Communication Technologies,2006.

[3] http://www.tecsol.com.au/

[4]Nait-Charif,H.and S.J.McKenna.Activity summarization and fall detection in a supportive home environment.IEEE17th Conf.on Pattern Recognition,2004.

[5]V.Vaidehi et al.Video based automatic fall detection in indoor environment.IEEE International Conference on Recent Trends in Information Technology,2011.

[6]Shoaib, Muhammad, Dragon, R., Ostermann, J. View-invariant fall detection for elderly in real home environment. 4th Pacific-Rim Symposium on Image and Video Technology, 2010.

Contents of the invention

The inventors of the present application have made the present invention in consideration of the above-mentioned circumstances of the prior art. The invention proposes a fall detection method based on a multi-angle camera, which can detect fall events in different spatial directions, has real-time processing capability, and has high practicability. For example, in an indoor home environment, it is possible to detect in time the possible falls of the elderly or patients (observation objects) who are alone, and reduce the injuries caused by the falls to a large extent.

Usually, when a fall occurs, the posture of the faller will change significantly. Based on this principle, the present invention divides the different postures during the fall into four categories, and extracts the target through two camera video images from different angles. Appearance, scene features, and use the Relevant Vector Machine (RVM) to perform fast gesture recognition on moving targets. The hidden Markov model (HMM) is used to model the posture changes during the fall process, and the model is used to evaluate each movement process in the surveillance video, so as to judge whether there is a fall event.

According to an embodiment of the present invention, a real-time fall event detection method based on surveillance video is provided, wherein multiple cameras facing the same target area and with different shooting angles are installed in the detection scene, and the multiple cameras continuously shoot Target area, the method includes the following steps: Step 1, the plurality of cameras shoot a section of video of the target area at the same time; Step 2, from the plurality of videos of the same period of time captured by the plurality of cameras, respectively extract the video Each frame of picture represents the foreground area of the target; Step 3, extracting the respective shape and position features of the foreground area of the same target at the same moment in the pictures taken by the plurality of cameras, and using the RVM classifier, Determine the target attitude category at the moment corresponding to each frame of picture; step 4, input the target attitude category of each frame picture obtained as the target attitude value sequence into the HMM evaluation-, obtain the posterior probability of the change of the target attitude category, where , the change process of the target posture category indicates the occurrence of a target fall event; and step 5, if the posterior probability is greater than a predetermined threshold, then determine the occurrence of the target fall event.

The present invention innovatively adopts the combination of RVM and HMM to recognize the pattern of the video image, not only can recognize the posture of the observed object at any time from the video, but also can recognize the posture change process within any period of time. In this way, for images such as Situations such as falling over a period of time that have a posture change can be detected in real time.

The present invention is mainly applied in home monitoring video scenes, and is used for monitoring possible falls (abnormal lying down) events that may appear in the monitoring video and alarming in time, so as to effectively protect the personal safety of the monitored person. Beneficial effects mainly include: (1) Carry out all-weather real-time monitoring of empty-nest elderly living alone, analyze the behavior and status of the elderly, automatically filter out useless information, and make quick judgments on possible falls of the elderly and Call the police for timely rescue and fundamentally guarantee the safety of the elderly living alone. (2) Analyze the physical status of patients who need to be supervised. When a fall occurs, it can automatically alarm the on-duty staff and prompt the medical staff to deal with it in time. On the one hand, it can reduce the workload of medical staff, and on the other hand, it also provides valuable time for the timely rescue of patients.

Description of drawings

Fig. 1 is a schematic diagram showing the included angle difference of the declination angle of the circumscribed ellipse of the human body in two mutually perpendicular imaging directions according to an embodiment of the present invention;

Fig. 2 is a schematic diagram showing scene information of a background where a human body is located according to an embodiment of the present invention;

Fig. 3 is a schematic structural diagram showing three RVM classifiers trained according to an embodiment of the present invention;

Fig. 4 is a graph showing the logarithmic posterior probability log(P(O|λ)) of the pose sequence under the training model in a video according to an embodiment of the present invention as it changes with the number of frames.

Detailed ways

Below, the implementation of the technical solution will be further described in detail in conjunction with the accompanying drawings.

First, the principle of the present invention will be briefly described.

According to an embodiment of the present invention, the real-time fall detection method based on HMM and RVM mainly includes the following steps in the model training stage: 1) feature extraction, which is used to extract the posture reflecting the human body from the training video frames of two cameras with different angles 2) Attitude classification, use the extracted features to train a classifier, and get the attitude category of each training video frame through the classifier; 3) Use the hidden Markov model (HMM) to classify the fall The pose changes during the process are modeled (generating an HMM model).

According to an embodiment of the present invention, the real-time fall detection method based on RVM and HMM mainly includes the following steps in the event detection stage: 1) feature extraction, which is used to extract the posture reflecting the human body from the test video frames of two cameras with different angles 2) Pose classification, use the trained classifier to obtain the pose category of each test video frame; 3) Use the HMM model generated in the above training stage to evaluate each movement process in the surveillance video ( The process of changing the posture category), so as to judge whether there is a fall event.

Next, the specific implementation process of the real-time fall detection method based on RVM and HMM of the present invention will be described respectively according to the above sequence. Those skilled in the art can understand that some of the following steps/operations exist in both the training phase and the testing phase, and for the sake of brevity, repeated descriptions are not repeated.

1. Feature extraction

Decomposing the process of falling, the posture change process of the human body can be attributed to standing-tilting-lying (to the ground). Based on this, the features composed of the geometric appearance and scene information of the human body are used as the input of the RVM classifier to judge the posture. The postures of the human body in the home video can be roughly divided into four categories:

1) standing;

2) Tilt;

3) Lying (ground only);

4) Others, including sitting, squatting, lying on the bed, etc.

The above classification is only an example, and those skilled in the art can understand that human body postures can also be divided into any number of categories different from the above four categories according to actual needs.

The geometric appearance features of the human body are:

1) The aspect ratio of the rectangle circumscribing the human body. For the standing posture, this ratio is small, and for the inclined posture, the circumscribed rectangle shape is close to a square, and the aspect ratio is close to 1;

2) The angle difference between the declination angle of the circumscribed ellipse of the human body in the two camera directions. In order to analyze human pose from multiple camera angles, the included angle of ellipse fitting needs to be considered. Due to the perspective transformation of the camera, there is a certain amount of information loss when the 3D scene is mapped to the 2D image, making it difficult to distinguish many poses, such as standing, lying down along the direction of the camera, etc. Two cameras with the same line of sight and perpendicular to each other can be used to effectively complement information. For the standing posture, since the target is perpendicular to the ground plane, in the two camera images, the angle between the major axis of the circumscribed ellipse of the human body and the horizontal axis is about 90°, so the difference between the two angles is about 0°; , the target and the ground plane form a certain angle, and the angle difference (absolute value) between the two camera directions is in the range of 0° to 90°, which is generally significantly higher than the angle difference in the standing posture (about 0° ); for the lying position (only on the ground), since the target is parallel to the ground plane, the angle difference (absolute value) of the included angle (considering positive and negative) in the two vertical camera directions is about 90°. The above principle can be explained by Figure 1 (from top to bottom are standing, leaning, lying down; the first column is a schematic diagram of the actual scene, and the second and third columns are the pictures taken by two cameras). The placement of the above-mentioned two cameras with the same height and vertical line of sight is just an example. Those skilled in the art can understand that there are actually other ways for the placement of the two cameras, as long as the changes of the above-mentioned included angles under different postures can present Some kind of law will do. Of course, more cameras can also be used to obtain more accurate and refined classification results.

The scene information features used are:

1) The histogram of the scene information of the background where the human body is located. According to the characteristics of the home video, the scene area is artificially marked in advance, mainly including the bed/sofa/chair area, wall, and ground, as shown in Figure 2 (where gray represents the wall, black represents the bed/sofa/chair, and white represents the ground ). The three areas are represented by different values, the scene information of the area where the human body is located is counted, and the proportions of the three gray values are calculated to form a 3-bin scene information histogram.

For the original video, use the foreground segmentation algorithm to extract the moving area (see literature [7]). For the above three features, they are all extracted from the above two camera images at the same time, thus forming a total of 2×1+1+3 ×2=9-dimensional feature vector. That is, for the moment corresponding to each video frame of the surveillance video, the aforementioned 9-dimensional feature vector is extracted.

2. Pose classification

Next, pose classification can be performed. The pose classifier uses RVM (for example, the RVM disclosed in [8] can be used because of its fast test speed). Since RVM is mainly used in the case of 2 classifications, multiple 2 classifiers need to be trained for layer-by-layer classification. Analyze the extracted scene information features and human appearance features. Since these two types of features are different, the classification structure of decision tree is used to consider these two types of features separately, and each time select some features for classification and judge layer by layer. Three features of the scene histogram, the aspect ratio of the circumscribed rectangle and the angle difference of the circumscribed ellipse are selected in turn, and three 2-classifiers are trained. The resulting classification structure is shown in Figure 3 below. Specifically, for example, the first classifier RVM1 can be used to distinguish the above-mentioned fourth type of posture from other three types of postures, the second classifier RVM2 can be used to distinguish the above-mentioned third type of posture from the first and second types of postures, and the third classifier The classifier RVM3 is used to distinguish the above-mentioned type 2 posture from the type 1 posture.

In this way, the above-mentioned 9-dimensional features extracted from each frame captured by the two cameras are sequentially sent to the three classifiers to obtain the pose classification result. According to the posture of the person in each frame, the corresponding posture type number is obtained, thereby generating the posture sequence, which is the observation sequence in the HMM model used in the following HMM evaluation, where the number of states N is 4, that is, the above 4 possible output states.

The prediction and classification process of the RVM classifier can be summarized as follows (for details, refer to [8]):

1) It is known that the feature matrix X∈R ^n×m participating in the training, the new feature vector x*∈R ^1×n obtained from the test sample, and the RVM model vector p∈R ^m×1 obtained from the training, where n is the feature dimension number, m is the number of samples participating in the training;

2) Calculate the basis vector b∈R ^1×m by using x* and X;

3) Multiply the base vector and the model to get the value y=b*p. If y>0.5, it is predicted as a positive class, otherwise it is predicted as a negative class.

The training process of described RVM classifier comprises:

1) Select an appropriate kernel function to map the feature vector to a high-dimensional space. Several kernel functions commonly used comprise RBF kernel function, Laplace kernel function, polynomial kernel function etc., adopt RBF kernel in the present invention;

2) Initialize RVM parameters;

3) Extract posture features from training samples of four postures (standing, lying on the ground, tilting, etc.), the features of all samples form a feature matrix X, and the corresponding posture labels of all samples form a vector Y;

4) According to the Bayesian criterion, use the training features and labels to iteratively solve the optimal weight distribution and distribution parameters of the training samples;

5) Output the RVM parameters, that is, the trained model.

3. HMM evaluation

In each frame, the attitude category of the target is recorded. In order to use HMM, the attitude of the target is represented by discrete values, that is, (0, 1, 2, 3), so that in a continuous period of time, a set of length T ( Corresponding to the frame number of the video), the target pose value sequence, that is, the observation sequence O ₁ O ₂ ... O _T . In the training phase, according to the HMM learning problem, use the observation sequence O ₁ O ₂ ... O _T extracted during the fall process to learn, and find a set of model parameters λ={π,A,B} such that P(O| λ) is the largest, which is the parameter of the HMM fall model.

The training procedure of described HMM model comprises:

1) Collect multiple falling videos of different fallers in different directions;

2) Extract the features in each fall video, and use RVM to classify the poses. Within a time sliding window, use the pose number output by each frame as the HMM observation sequence;

3) Use the Baum-Welch training algorithm based on multi-observation sequences to train the HMM model λ. The steps of the traditional Baum-Welch algorithm are as follows (see literature [9]):

3-1 Assign an initial value λ ₀ to the model parameters;

3-2 Using the forward-backward method (see literature [10]), calculate the posterior probability of observing sequence O under this model, that is, P(O|λ ₀ );

3-3 Based on the observation sequence O and the current model parameters, update the model parameter λ, the update formula is:

$\begin{array}{cc}{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; a</span>}}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&xi;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}& {\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; b</span>}}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; m</span>}\end{array}$

${\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>$

In the formula, N is the number of hidden states, and its meaning is the hidden target posture information (for example, falling posture, standing posture, inclined posture), and the empirical value 3 is taken in this patent; M is the number of observation values, also That is, the four state numbers output by RVM; a _ij is the transition probability between states, indicating the probability of transitioning from state i to state j; b _j (k) is the emission probability, indicating the probability of outputting observation value k in state j ; π _i is the probability distribution of the initial state; v _k represents the kth observed value; O _t represents the observed value at time t. ξ _t (i,j) and γ _t (i) are auxiliary variables calculated from the current model parameters (see literature [9]), and T is the length of the training video frame.

4. Calculate the posterior probability P(O|λ) of the observation sequence under the new model

5. If logP(O|λ)-logP(O|λ ₀ )<Delta (Delta is a very small number, usually around 1e-6), it means that the training has achieved the expected effect, the algorithm ends, and the current model is output λ, otherwise, set λ ₀ =λ, and return to step 3 to continue working.

Since traditional training algorithms are only based on single-observation sequences, the trained models are not universally applicable. This patent introduces a training algorithm based on multi-observation sequences.

Consider a sequence of observations O={O ⁽¹⁾ ,O ⁽²⁾ ,…O ^(K) } in the same mode, Each is a single observation sequence (1≤k≤K). Since each sequence does not affect each other, it is assumed that they are independent of each other. Based on this assumption, the update formula of the model parameters changes as (see literature [9]) :

$\begin{array}{cc}{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; a</span>}}}_{\mathrm{<span\; class="notranslate">\; mn</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&xi;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}& {\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; b</span>}}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; m</span>}\end{array}$

${\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>$

The model parameters of the HMM can be summarized in the following table:

parameter meaning T Observation sequence length O＝o ₁ ,o ₂ ,…o _T sequence of observations Q＝q ₁ q ₂ …q _T hidden state sequence N number of hidden states S＝{s ₁ ,s ₂ ,…s _N } The set of possible values for the hidden state m The number of possible values for an observation V＝{v ₁ ,v ₂ ,…v _M } The set of possible values for the observation A＝{a _ij |a _ij ＝P _r (q _t+1 ＝s _j |q _t ＝s _i )} Time-independent hidden state transition probability matrix B＝{b _j (k)|b _j (k)＝P _r (v _k |q _t ＝s _j )} Observation probability distribution given the hidden state π＝{π _i |π _i ＝P _r (q ₁ ＝s _i )} Initial hidden state distribution

In the test phase, according to the HMM evaluation problem, use the existing parameters and the observation sequence O=O ₁ O ₂ ... O _T extracted from the test video to calculate the probability P(O|λ) of the sequence (refer to [10]), so as to judge whether it is a fall.

Figure 4 is a graph of the logarithmic posterior probability logP(O|λ) of the attitude sequence under the training model in a video as it changes with the number of frames, in which the six peak intervals correspond to six different fall events. During the fall, The log-posterior probability of the pose output sequence reaches an extreme point with a strong difference from the non-fall pose.

References mentioned above

[7] Yuyang Chen, Yanyun Zhao, Anni Cai, A robust moving object segmentation algorithm using integrated mask-based background maintenance, 3rd IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC), 2012.

[8] Michael E. Tipping, Sparse Bayesian Learning and the Relevance Vector Machine, Journal of Machine Learning Research 1(2001) 211-244.

[9] LE Baum, T. Petrie, G. Soules, and N. Weiss, ^a A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains, ^o Annals of Math. Statistics, vol.41, no.1, pp. 164-171, 1970

[10] Xiaolin Li, Parizeau, M., Plamondon, Rejean, "Training hidden Markovmodels with multiple observations-a combinatorial method", in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.22, no.4, pp.371-377 , April, 2000.

In order to avoid making the description in this specification redundant, in the description in this specification, some technical details that can be obtained in the above-mentioned references or other prior art materials may be omitted, simplified, and modified. It is understandable to those skilled in the art, and this does not affect the adequacy of the disclosure of this specification. The above references are hereby incorporated by reference in their entirety.

In summary, those skilled in the art can understand that various modifications, variations, and replacements can be made to the above embodiments of the present invention, all of which fall within the protection scope of the present invention as defined by the appended claims.

Claims (6)

1. A real-time fall event detection method based on monitoring video, wherein, in the detection scene, a plurality of cameras facing the same target area and different shooting angles are installed, and the plurality of cameras continuously shoot the target area, and the method includes The following steps: Step 1, the plurality of cameras shoot a section of video of the target area at the same time; Step 2, extracting the foreground area representing the target of each frame of the video from the plurality of videos of the same period of time shot by the plurality of cameras respectively; Step 3, extracting the respective shape and position features of the foreground area of the same target in the pictures taken by the plurality of cameras at the same moment, and using the RVM classifier to determine the target posture category at the moment corresponding to each frame of picture ; Step 4. Input the obtained target posture category of each frame picture into the HMM evaluation- as a target posture value sequence to obtain the posterior probability of the target posture category change, wherein the target posture category change process indicates the target fall event the occurrence of; and Step 5. If the posterior probability is greater than a predetermined threshold, determine the occurrence of the target fall event. 2. The real-time falling event detection method according to claim 1, wherein, The target posture category includes the following four categories: 1) standing; 2) leaning; 3) lying on the ground; 4) other, The change in the target posture category represents the following changes: standing→tilting→lying on the ground, Wherein, the plurality of cameras are two cameras whose shooting angles of view are perpendicular to each other, and multiple types of sub-regions are marked in the target area shot by each camera in advance, The shape and position features include the following three features: 1) the aspect ratio of the circumscribing rectangle of the target; 2) the angle difference between the major axis of the circumscribing ellipse of the target and the horizontal line in the respective shooting directions of the two cameras ; 3) Which type of sub-area the target is in. 3. The real-time fall event detection method according to claim 2, wherein the RVM classifier comprises 3 2 classifiers, wherein the first 2 classifier distinguishes the first three classes in the four classes with the fourth class, the second 2-classifier distinguishes the first two of the four classes from the third class, and the third 2-classifier distinguishes the second of the four classes from the third a category. 4. real-time fall event detection method according to claim 3, wherein, the training procedure of described RVM classifier comprises the following steps: Step 11, extracting the shape and position features from the training sample videos belonging to the four categories respectively to form a feature matrix X, and the target pose categories corresponding to all training sample videos form a vector Y; Step 12, using the RBF kernel function to map the feature matrix X to a high-dimensional space; Step 13, according to the Bayesian criterion, the optimal weight distribution and distribution parameters are obtained as RVM parameters. 5. real-time fall event detection method according to claim 4, wherein, the training procedure of described HMM model comprises the following steps: Step 21, the input K section includes the fall sample video of the fall event; Step 22, extract the shape and position features from each frame of each fall sample video, and use the RVM classifier to determine the target posture category at the moment corresponding to each frame, and form the target posture category number into an HMM observation Sequence set O={O ⁽¹⁾ ,O ⁽²⁾ ,...O ^(K) }; Step 23, using the Baum-Welch training algorithm based on the multi-observation sequence to train the HMM model. 6. The real-time falling event detection method according to claim 5, wherein said step 23 comprises the following steps: Step 23-1. Assign an initial value λ ₀ to the current model parameter λ, where λ={π,A,B}; Step 23-2. Using the forward-backward method, calculate the posterior probability P(O|λ ₀ ) of the observation sequence set O under the initial value λ ₀ of the current model parameters; Step 23-3. Based on the following formula, update the model parameter λ: $\begin{array}{cc}{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; a</span>}}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&xi;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}& {\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; b</span>}}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; m</span>}\end{array}$ ${\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; ,</span>$ Among them, N is the number of hidden states; M is the number of types of observations, M=4; a _ij is the transition probability between hidden states, indicating the probability from hidden state i to hidden state j; b _j (k) is Emission probability, which means the probability of outputting observation value k in state j; π _i is the probability distribution of the initial state; v _k represents the kth observation value; O _t represents the observation amount at time t corresponding to the video frame, ξ _t (i, j) and γ _t (i) are auxiliary variables calculated from the current model parameters, T is the number of training video frames, 23-4. Calculate the posterior probability P(O|λ) of the observation sequence set O under the updated current model parameter λ; 23-5. If logP(O|λ)-logP(O|λ ₀ )<Delta, output the current model parameter λ, otherwise, set λ ₀ =λ, return to step 23-3, where Delta is 1E- constant around 6, Other model parameters of the HMM model are defined in the following table: parameter meaning Q＝q ₁ q ₂ …q _T hidden state sequence S＝{s ₁ ,s ₂ ,…s _N } The set of possible values for the hidden state V＝{v ₁ ,v ₂ ,…v _M } The set of possible values for the observation A＝{a _ij |a _ij ＝P _r (q _t+1 ＝s _j |q _t ＝s _i )} Time-independent hidden state transition probability matrix B＝{b _j (k)|b _j (k)＝P _r (v _k |q _t ＝s _j )} Observation probability distribution given the hidden state π＝{π _i |π _i ＝P _r (q ₁ ＝s _i )} Initial hidden state distribution