CN107832688B - A traffic pattern and abnormal behavior detection method for video surveillance of traffic intersections - Google Patents

Abstract

The invention discloses a method for detecting traffic modes and abnormal behaviors, which comprises the following steps: a1, dividing video with time length of T seconds into video with length of T seconds_sShort video clips of seconds, resulting in T/T_sA video document; a2, calculating optical flow vectors of every two adjacent pairs of video frames of each video document; a3, quantizing the obtained optical flow vector to obtain a video word of each pair of video frames; a4, counting the counting vector of the video words of each video document to obtain a document-word counting matrix of the video document set; a5, extracting topics by using a BNBP-PFA topic model to obtain topic-word distribution and document-topic distribution; a6, respectively using the obtained topic-word distribution and document-topic distribution as a new word and a new document, and obtaining the topic of the second layer topic model by utilizing a BNBP-PFA topic model; and A7, detecting abnormal behaviors in the video frame based on the log-likelihood function values of the two layers of topic models.

Description

A traffic pattern and abnormal behavior detection method for video surveillance of traffic intersections

technical field

The invention relates to a method for detecting traffic patterns and abnormal behaviors, in particular to a method for detecting traffic patterns and abnormal behaviors in video surveillance of traffic intersections.

Background technique

With the development of machine vision and data mining technology, it becomes possible to automatically discover useful information in video surveillance data. Among them, finding regular traffic patterns or abnormal traffic behaviors from video data of crowded people and traffic intersection scenes has become a kind of problem that has not been fully solved with important research value and technical application prospects. To solve such problems, there are often the following challenges: 1) In complex and crowded traffic scenes, the performance of existing computer vision tracking methods is often poor; 2) Traffic patterns are not related to the underlying features in the video, It reflects the high-level semantic information in the video, which is related to the high-level vision in machine vision, that is, visual understanding. There is often a huge semantic gap between low-level visual features and high-level visual semantics, which makes the methods based on low-level visual feature detection—object detection and object tracking methods unable to obtain the upper-level semantic information of the entire video. Specific to the problem of traffic pattern detection and abnormal traffic behavior detection at traffic intersections, due to the dense flow of people and vehicles at intersections, the scene is easily affected by noise, illumination, weather changes and complex background information. Based on target detection and target motion Trajectory clustering methods tend to perform poorly.

In order to overcome the shortcomings of the above methods, another method that directly utilizes the underlying motion information of the video, such as optical flow information, to acquire "events" or "activities" in a video scene has gradually become popular. This class of methods avoids the tracking of a single moving target, and mainly uses the rich local motion information between adjacent video frames - the position and motion information from the underlying features, and then uses complex dimensionality reduction models (such as topic models) Extract effective high-level semantic information from high-dimensional feature information. Common topic models, such as PLSA (Probabilistic Latent Semantic Analysis), LDA (Latent Dirichlet Allocation), HDP (Hierarchical Dirichlet Process), etc., were originally used for topic discovery in text corpora, and were gradually used for image and video analysis tasks. . Wang Xiaogang et al. [1] proposed an unsupervised learning framework using Hierarchical Bayesian models to model "events" and "behavioral patterns" in complex and crowded video scenes. Song et al. [2] proposed a traffic pattern mining method with a two-layer LDA structure, which can discover simple and complex traffic patterns in video scenes of traffic intersections and detect abnormal traffic behaviors. Reference [3] uses FSTM (Fully Sparse Topic Model) to detect anomaly in traffic video. Reference [4] proposed a method to use HDP and HDP-HMM to learn the typical activity and traffic state information in traffic videos, respectively, and use Gaussian process to classify the traffic state information. MCTM (Markov Clustering Topic Model) uses the LDA model to model the underlying features of traffic video frames—visual words and Markov chains to model the temporal relationship between adjacent video frames. This method can realize the hierarchical clustering of traffic visual features into local traffic patterns and global traffic patterns. WS-JTM (Weakly Supervised Joint Topic Model) is a weakly supervised joint topic model. Based on the LDA model, the model makes full use of the category features of different video documents to mine typical traffic patterns and abnormal traffic patterns. Another class of methods is mainly based on non-topic models. This type of method mainly uses modeling methods such as matrix factorization and sparse dictionary learning to perform topic modeling on the underlying visual features to obtain typical and abnormal traffic patterns.

Among the above two types of traffic video traffic pattern mining methods, the method based on probabilistic topic model is difficult to obtain sparse traffic patterns, that is, topics in video documents are not sparsely distributed. In addition, the learning and reasoning methods of the method model based on the probabilistic topic model are more complicated, which leads to the complex operation process of the algorithm and the large amount of calculation. The methods based on non-probabilistic topic models are widely used in the discovery of typical and abnormal traffic patterns in traffic videos because they can make full use of the sparsity in visual information. . In view of these problems, the present invention proposes a traffic pattern analysis method with a two-layer structure. The first layer uses the BNBP-PFA (negative binomial beta process-Poisson factorization) topic model to extract the topic, that is, the simple traffic pattern. The distribution of each video document on the topic (simple traffic pattern) is obtained, and the second-layer BNBP-PFA topic model obtains the second-layer topic based on the topic obtained in the first layer, that is, the complex traffic pattern in the video. Compared with the two-layer LDA model of the literature [2], the method proposed in the present invention does not need to pre-specify the number of topics in each layer because it adopts the BNBP-PFA topic model. In addition, compared with the LDA model, since BNBP is suitable for processing sparse count data, it is especially suitable for processing video motion feature data; compared with the HDP topic model, BNBP has better structural form and computational flexibility.

content of invention

The main purpose of the present invention is to overcome the shortcomings of the existing traffic pattern and abnormal behavior detection methods, and provide a new traffic pattern and abnormal behavior detection method based on a two-layer BNBP-PFA topic model. This method uses the two-layer BNBP-PFA topic model to simultaneously detect simple traffic patterns and complex traffic patterns in traffic videos. Compared with existing methods, it has more recognized patterns, higher accuracy, and can automatically learn patterns. On this basis, an abnormal behavior detection method based on the log-likelihood function value of the two-layer BNBP-PFA topic model is proposed, which has achieved better detection results than existing methods. better detection results.

The method proposed by the invention includes the extraction of video optical flow features and the generation of video documents, the detection of simple and complex traffic patterns based on the two-layer BNBP-PFA topic model, and the detection of abnormal behaviors in traffic videos. In order to solve these technical problems, the present invention provides a traffic pattern and abnormal behavior detection method for video surveillance at a traffic intersection, the method includes the following steps:

A1. A long video with a duration of T seconds is divided into short video clips with a length of T _s seconds in chronological order, and each video clip is used as a video file, and N=T/T _s video files are obtained altogether;

A2. For each video document, calculate the optical flow vector of every two adjacent pairs of video frames;

A3. quantify the optical flow vector obtained in A2 to obtain the video words of each pair of video frames of each video document;

A4. Based on the bag-of-words model, count the count vectors of the video words of each video document, and obtain the document-word count matrix M of the video document set composed of the entire long video;

A5. Use the BNBP-PFA topic model to extract topics from the video documents obtained in A4, and obtain topic-word distribution and document-topic distribution, and the resulting topics are the simple traffic patterns in the video;

A6. Take the topic obtained in A5 as a new word, take the document-topic distribution obtained in A5 as a new document, use the BNBP-PFA topic model to extract the topic, and obtain the topic-word distribution of the second-layer topic model, the obtained The subject is the complex traffic pattern in the video;

A7. Based on the two-layer BNBP-PFA topic model obtained in A5 and A6, and based on the log-likelihood function value of the two-layer topic model, detect abnormal behaviors in video frames.

The process of calculating the optical flow vectors of two adjacent pairs of frames in the above step A2 specifically includes:

A21. Calculate the optical flow information vector (v _x (i, j), v _y (i, j)) of each pixel (i, j) for two adjacent consecutive video frames I _x and I _y ;

和

得到每个像素点光流的强度和方向信息(M(i,j),D(i,j))；A22. According to the formula

and

Obtain the intensity and direction information of the optical flow of each pixel point (M(i,j), D(i,j));

The process of quantizing the optical flow vector to obtain video words in the above step A3 specifically includes:

个像素块(符号[*]表示取整数，N₁,N_x,N_y均为正整数)，用该像素块中心点的坐标作为该块的坐标；A31. Divide the position information of the optical flow: divide the video frame of size N _x ×N _y into N ₁ ×N ₁ pixel blocks, and obtain a total of

pixel blocks (the symbol [*] represents an integer, N ₁ , N _x , N _y are all positive integers), and the coordinates of the center point of the pixel block are used as the coordinates of the block;

当该块的光流强度值超过预先设置的门限值Th即

时，判定该像素块是运动像素块，否则是背景像素块；将该块的光流方向进行量化，共量化为S个方向；A32. Quantization of optical flow intensity and direction: each pixel block contains N ₁ ² pixels, and the average optical flow value of these N ₁ ² pixels is used as the optical flow vector of the pixel block

When the optical flow intensity value of the block exceeds the preset threshold Th, that is,

When , determine that the pixel block is a motion pixel block, otherwise it is a background pixel block; quantize the optical flow direction of the block, and quantize it into S directions in total;

A33. According to the above quantification method, the size of the vocabulary of the video document set that can be obtained is:

The above-mentioned step A5 specifically includes:

m_ijk～Pois(φ_ikθ_kj)，φ_k～Dir(α_φ,…,α_φ)，θ_kj～Gamma(r_k,p_k/(1-p_k))，r_k～Gamma(c₀r₀,1/c₀)，p_k～Beta(cε,c(1-ε))的BNBP-PFA主题建模过程，可以得到K个主题分布矩阵Φ∈R^P×K和K个主题在N个文档中的组成情况矩阵Θ∈R^K×N，其中主题分布矩阵表示K个主题在P个特征上的分布情况。Assuming that the resulting document count matrix in A4 is M _ij ∈ R ^P×N , the count matrix contains P features of N documents. According to the formula

m _ijk ～Pois(φ _ik θ _kj ), φ _k ～Dir(α _φ ,…,α _φ ), θ _kj ～Gamma(r _k ,p _k /(1-p _k )), r _k ～Gamma(c ₀ r ₀ ,1/c ₀ ), p _k ~Beta(cε,c(1-ε)) BNBP-PFA topic modeling process, K topic distribution matrices Φ∈R ^P×K and K topics can be obtained The composition matrix Θ∈R ^K×N in N documents, where the topic distribution matrix represents the distribution of K topics on P features.

The above-mentioned step A6 specifically includes:

θ_kjk′～Pois(φ′_kk′θ′_k′j)，φ′_k′～Dir(α′_φ′,…,α′_φ′)，θ′_k′j～Gamma(r′_k′,p′_k′/(1-p′_k′))，r′_k′～Gamma(c′₀r′₀,1/c′₀)，p′_k′～Beta(c′ε′,c′(1-ε′))的BNBP-PFA主题建模过程，可以得到K′个主题——词分布φ′_kk′和文档——主题的分布θ′_k′j。The topic φ _ik obtained in step A5 is regarded as the word in step A6, and the document-word distribution θ _kj in A6 is regarded as composed of the topics in A5. According to the formula

θ _kjk′ ～Pois(φ′ _kk′ θ′ _k′j ), φ′ _k′ ～Dir(α′ _φ′ ,…,α′ _φ′ ), θ′ _k′j ～Gamma(r′ _k′ , p′ _k′ /(1-p′ _k′ )), r′ _k′ ~Gamma(c′ ₀ r′ ₀ ,1/c′ ₀ ), p′ _k′ ~Beta(c′ε′,c′ (1-ε′)) BNBP-PFA topic modeling process, K′ topic-word distribution φ′ _kk′ and document-topic distribution θ′ _k′j can be obtained.

The above-mentioned step A7 specifically includes:

A71. On the document-visual word count matrix M of the entire video document set, randomly select 80% of the video documents to form the training video document set X, and the remaining 20% of the video document set form the test set Y=M-X;

(其中

y_pi＝Y(p,i)，φ_pk是第一层BNBP-PFA主题模型所得主题分布，θ_ki是视频帧y_pi在主题φ_pk上的分布的系数，N₁是视频帧y_pi长度即其包含的视觉词的数量)，计算每个视频帧y_pi在第一层BNBP-PFA主题模型上的归一化的似然函数值F₁；A72. On the test set Y, according to the formula

(in

y _pi = Y(p,i), φ _pk is the topic distribution obtained by the first-layer BNBP-PFA topic model, θ _ki is the coefficient of the distribution of video frame y _pi on topic φ _pk , N ₁ is the length of video frame y _pi That is, the number of visual words it contains), calculate the normalized likelihood function value F ₁ of each video frame y _pi on the first-layer BNBP-PFA topic model;

(其中

θ_ki是视频帧y_pi在主题φ_pk上的分布的系数，φ′_kk′是第二层BNBP-PFA主题模型所得主题分布，θ′_k′i是θ_ki在主题φ′_kk′上的分布的系数，N₂是θ_ki向量的长度即其包含的主题φ_pk的数量)，计算每个视频帧y_pi在主题φ_pk上的分布的系数θ_ki在第二层BNBP-PFA主题模型上的归一化的似然函数值F₂；A73. According to the formula

(in

θ _ki is the coefficient of the distribution of video frame y _pi on the topic φ _pk , φ′ _kk′ is the topic distribution obtained by the second-layer _BNBP -PFA topic model, θ′ k′ i is the distribution of θ _ki on the topic φ′ _kk′ Coefficient of distribution, N ₂ is the length of the θ _ki vector (that is, the number of topics φ _pk it contains), calculate the coefficient θ _ki of the distribution of each video frame y _pi on the topic φ _pk in the second-layer BNBP-PFA topic model The normalized likelihood function value F ₂ on ;

A74. Calculate the weighted likelihood function value F=η·F ₁ +(1-η)·F ₂ of the video frame y _pi on the two-layer BNBP-PFA topic model, where the parameter η∈(0,1) is taken;

A75. Compare the likelihood function value F calculated in A74 with the given threshold value Th ₁ , if there is F<Th ₁ , the video frame y _pi contains abnormal behavior, otherwise it does not.

The beneficial effects of the embodiments provided by the present invention:

The present invention applies the subject model to the understanding and analysis of traffic video scenes, and invents a method that can detect both simple traffic patterns and complex traffic patterns in traffic video scenes and abnormal traffic behaviors. Compared with existing methods, the method of the present invention finds more traffic patterns and is of higher quality. In addition, due to the use of a non-parametric topic model, the method of the present invention does not need to specify the number of topics in advance, which is very useful when dealing with some complex and unknown traffic video data. The method proposed in the present invention can be applied to the mining of traffic patterns in traffic videos and the detection of abnormal traffic behaviors, which is of great significance to the development of intelligent traffic and traffic video monitoring and other fields.

Description of drawings

1 is a flowchart of a specific embodiment of a method for detecting traffic patterns and abnormal behaviors in a traffic intersection video surveillance of the present invention;

Fig. 2 is the sample frame of the data set used in this embodiment;

FIG. 3 is the first-level theme found in the video data set by the method of the present invention in this embodiment, namely the simple traffic pattern;

Fig. 4 finds 15 topics in the video data set by LDA method in this embodiment;

5 is the 15 topics found in the video dataset by the FTM method in this embodiment;

FIG. 6 is the 15 topics found in the video dataset by the HDP method in this embodiment;

FIG. 7 is the second-layer theme found in the video data set by the method of the present invention in the present embodiment, that is, the complex traffic pattern;

FIG. 8 is four abnormal traffic behaviors detected on the video data set by the method of the present invention in this embodiment;

Detailed ways

The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, but not to be construed as a limitation of the present invention.

FIG. 1 is a flow chart of a specific embodiment of a method for detecting traffic patterns and abnormal behaviors in a video surveillance of a traffic intersection according to the present invention. As shown in FIG. 1 , the workflow of the method for detecting traffic patterns and abnormal behaviors in this embodiment includes the following steps:

A1: Divide a long video with a duration of T seconds into short video clips with a length of T _s seconds in chronological order. Each video clip is regarded as a video file, and a total of N ₁ =T/T _s video files are obtained.

In this step, the traffic intersection video data (http://www.eecs.qmul.ac.uk/～tmh/downloads.html) disclosed by the QMUL Junction Dataset 2 data set is downloaded through the computer as the video of the embodiment of the present invention data. The dataset contains a 52-minute video of a busy urban traffic intersection with a frame rate of 25Hz and a frame size of 360 × 288 pixels. A sample frame of this dataset is shown in Figure 2, and contains 6 motion patterns as shown in Table 1. The dataset contains a total of 4 abnormal behaviors. The total length of the video data is 3120 seconds, in which the length of one video document is 12 seconds of video (300 video frames in total), and after a total of 260 video documents are obtained, step A2 is entered.

Table 1 The possible motion patterns included in the QMUL Junction Dataset 2 in Figure 2 and their descriptions (in four directions, up, down, left, and right)

schema id a b c d e f Mode description Turn left and go up up down down right turn sidewalk right sidewalk left

A2: For each video document, calculate the optical flow vector of every two adjacent pairs of video frames.

和

计算得到每个像素点(i,j)光流的强度和方向矩阵(M(i,j),D(i,j))。In this step, for the 300 video frames included in each video document, starting from the second frame, calculate the distance between the frame I _x and the adjacent previous frame I _y in each pixel in chronological order. The optical flow vector (v _x (i, j), v _y (i, j)) at point (i, j), where the optical flow calculation adopts the standard Lucas-Kanade optical flow calculation method. Then follow the formula

and

The intensity and direction matrix (M(i,j), D(i,j)) of each pixel (i,j) optical flow is calculated.

A3: Quantize the optical flow vector (M(i,j), D(i,j)) obtained in A2 to obtain the video words of each pair of video frames of each video document.

In this step, the position information, intensity and direction of the optical flow are quantified respectively, which specifically includes three sub-steps:

1) The video frame with a size of 360×288 pixels is divided into 8×8 pixel blocks, 1620 pixel blocks are obtained in total, and the coordinates of the center point of each pixel block are used as the coordinates of the block;

当该块的光流强度值超过预先设置的门限值Th＝0.05即

时，判定该像素块是运动像素块，否则是背景像素块；将该块的光流方向进行量化，共量化为上、下、左、右共4个方向；2) Quantization of optical flow intensity and direction: take the average optical flow value of 64 pixels contained in each pixel block as the optical flow vector of the pixel block

When the optical flow intensity value of the block exceeds the preset threshold Th=0.05, that is

When , it is determined that the pixel block is a motion pixel block, otherwise it is a background pixel block; the optical flow direction of the block is quantized, and a total of 4 directions of up, down, left, and right are quantized;

3) According to the above quantization method, the size of the vocabulary of each video document set can be obtained as 6480.

A4. Based on the bag-of-words model, count the count vectors of the video words of each video document, and obtain the document of the video document set composed of the entire video data set—word count matrix M _260*6480 ;

A5. Use the BNBP-PFA topic model to extract the subject of the video document matrix M _260*6480 obtained in A4, and obtain the distribution Φ of the subject-word and the distribution Θ of the document-theme, and the obtained subject is the simple traffic pattern in the video;

The above-mentioned step A5 specifically includes:

m_ijk～Pois(φ_ikθ_kj)，φ_k～Dir(α_φ,…,α_φ)，θ_kj～Gamma(r_k,p_k/(1-p_k))，r_k～Gamma(c₀r₀,1/c₀)，p_k～Beta(cε,c(1-ε))的BNBP-PFA主题建模过程,按照如下的利用常用的马尔科夫链蒙特卡洛推理算法，可以得到上述BNBP-PFA主题模型的概率分布φ_k、θ_kj和相关参数r_k和p_k。具体推理算法如下：Suppose the resulting document count matrix in A4 is M _ij ∈ R ^6480×260 , which contains 6480 features for 260 documents. According to the formula

m _ijk ～Pois(φ _ik θ _kj ), φ _k ～Dir(α _φ ,…,α _φ ), θ _kj ～Gamma(r _k ,p _k /(1-p _k )), r _k ～Gamma(c ₀ r ₀ ,1/c ₀ ), p _k ~ Beta(cε,c(1-ε)) BNBP-PFA topic modeling process, according to the following using the commonly used Markov chain Monte Carlo reasoning algorithm, you can The probability distributions φ _k , θ _kj and related parameters r _k and p _k of the above BNBP-PFA topic model are obtained. The specific reasoning algorithm is as follows:

主题数量K的上界由cγαB(cε,c(1-ε))确定，其中P＝6480,N＝260,c＝1,γ＝1，α＝1，ε＝0.05；1) Note

The upper bound of the number of topics K is determined by cγαB(cε,c(1-ε)), where P=6480, N=260, c=1, γ=1, α=1, ε=0.05;

2) sample m _ijk according to the following formula (1);

[m _{ij1 ,…,m ijK ]} ～Mult(m _ij ;ζ _{ij1 ,…,ζ ijK )} (1)

可知p([m_1jk,…,m_pjk]|-)＝Mult(m_·jk；φ_k)，则可按照下面式(2)采样得到φ_k；3) Use the relationship between Poisson distribution and multinomial distribution, and the relationship

It can be known that p([m _1jk ,...,m _pjk ]|-)=Mult(m _·jk ; φ _k ), then φ _k can be obtained by sampling according to the following formula (2);

p(φ _k |-)～Dir(α _φ +m _1·k ,…,α _φ +m _P·k ) (2)

4) After marginalizing φ _k and θ _kj , m _·jk ~NB(r _k ,p _k ), p _k ~Beta(cε,c(1-ε)), then p _k can be sampled by the following formula (3) get;

p(p _k |-)～Beta(cε+m _··k ,c(1-ε)+Nr _k ) (3)

则可以按照下式(4)来采样得到r_k。5) Due to

Then, r _k can be obtained by sampling according to the following formula (4).

The composition matrix Θ∈R ^K×N of K=15 topic distribution matrix Φ∈R ^P×K and K=15 topics in N=260 documents can be obtained, wherein the topic distribution matrix represents K=15 topics in P = distribution over 6480 features. The obtained theme is the simple traffic pattern in the video as shown in Figure 3. As a comparison with the experimental results obtained by the method of the present invention, Figures 4 to 6 show the LDA method ^[4] , HDP method ^[1] and K=15 typical topics obtained by FTM method ^[3] on QMUL Junction Dataset 2 dataset. From the results shown in Figures 3-6, it can be seen that, except that the HDP method can only detect 5 modes, including 4 correct ones, the other three methods can correctly detect all valid motion modes. From the analysis of the above experimental results, it can be seen that the HDP method has the worst results among the four models, the method of the present invention has the best results, and the performance of the LDA method and the FTM method are not much different. In addition, judging from the experimental results in Figures 3 to 6, the quality of the topics generated by the four methods decreases in the order of the method of the present invention, the FTM method, the LDA method, and the HDP method.

A6. Take the topic distribution Φ obtained in A5 as a new word, take the document-topic distribution Θ obtained in A5 as a new document, use the BNBP-PFA topic model to extract topics, and obtain the topic-word distribution of the second-layer topic model Φ', the resulting theme is the complex traffic pattern in the video;

The above-mentioned step A6 specifically includes:

θ_kjk′～Pois(φ′_kk′θ′_k′j)，φ′_k′～Dir(α′_φ′,…,α′_φ′)，θ′_k′j～Gamma(r′_k′,p′_k′/(1-p′_k′))，r′_k′～Gamma(c′₀r′₀,1/c′₀)，p′_k′～Beta(c′ε′,c′(1-ε′))的BNBP-PFA主题建模过程，可以得到K′＝3个主题——词分布φ′_kk′和260个文档——主题的分布θ′_k′j。按照如下的利用常用的马尔科夫链蒙特卡洛推理算法，可以得到上述BNBP-PFA主题模型的概率分布φ′_k′、θ′_k′j和相关参数r′_k′和p′_k′。具体推理算法如下：The distribution of the 15 topics φ _ik obtained in step A5 is regarded as the words in A6, and the document-word distribution θ _kj in A6 is regarded as composed of the topics in A5. According to the formula

θ _kjk′ ～Pois(φ′ _kk′ θ′ _k′j ), φ′ _k′ ～Dir(α′ _φ′ ,…,α′ _φ′ ), θ′ _k′j ～Gamma(r′ _k′ , p′ _k′ /(1-p′ _k′ )), r′ _k′ ~Gamma(c′ ₀ r′ ₀ ,1/c′ ₀ ), p′ _k′ ~Beta(c′ε′,c′ (1-ε′)) BNBP-PFA topic modeling process, we can get K′=3 topics—word distribution φ′ _kk′ and 260 documents—topic distribution θ′ _k′j . The probability distributions φ′ _k′ , θ′ _k′j and related parameters r′ _k′ and p′ _k′ of the above-mentioned BNBP-PFA topic model can be obtained as follows using the commonly used Markov chain Monte Carlo inference algorithm. The specific reasoning algorithm is as follows:

主题数量K′的上界由c′γ′α′B(c′ε′,c′(1-ε′))确定，其中K＝15,N＝260,c'＝1,γ'＝1，α'＝1，ε'＝0.05；1) Note

The upper bound of the number of topics K' is determined by c'γ'α'B(c'ε',c'(1-ε')), where K=15, N=260, c'=1, γ'=1 , α'=1, ε'=0.05;

2) obtain θ _kjk′ by sampling according to the following formula (5);

[θ _kj1 ,…,θ _kjK′ ]～Mult(θ _kj ;ζ′ _kj1 ,…,ζ′ _kjK′ ) (5)

可知p([θ_1jk′,…,θ_kjk′]|-)＝Mult(θ_·jk′；φ′_k′)，则可按照下面式(6)采样得到φ′_k′；3) Use the relationship between Poisson distribution and multinomial distribution, and the relationship

It can be known that p([θ _1jk′ ,...,θ _kjk′ ]|-)=Mult(θ _·jk′ ; φ′ _k′ ), then φ′ _k ′ can be obtained by sampling according to the following formula (6);

p(φ′ _k′ |-)～Dir(α′ _φ′ +θ _1·k′ ,…,α′ _φ′ +θ _K·k′ ) (6)

4) After marginalizing φ′ _{k′ and} θ′ _k′j , θ _·jk′ ~NB(r′k′,p′k _′ ), p′k _′ ~Beta(c′ε′,c′(1 -ε′)), then p′ _k′ can be sampled by the following formula (7);

p(p′ _k′ |-)～Beta(c′ε′+θ _··k′ ,c′(1-ε′)+N′r′ _k′ ) (7)

则可以按照下式(8)来采样得到r′_k′。5) Due to

Then, r'_k' can be obtained by sampling according to the following formula (8).

K'=3 topic distribution matrices Φ'∈R ^K×K' and K'=3 topics in N=260 documents composition matrix Θ'∈R ^K'×N , where topic distribution matrix represents K' = distribution of 3 topics on K = 15 subtopics (ie simple traffic patterns). The resulting theme in A6 is the complex traffic pattern in the video as shown in Figure 7. Compared with the method in document [2], which can only obtain complex traffic patterns in two major directions, left and right and up and down, the method proposed in the present invention can obtain more detailed complex traffic patterns. Table 2 below gives the description of the subject composition and traffic flow state of the three traffic modes obtained by the method of the present invention on the QMUL Junction Dataset 2 dataset.

Table 2 The topic composition and traffic flow state descriptions of 3 traffic modes on QMUL Junction Dataset 2

A7. Based on the two-layer BNBP-PFA topic model obtained in A5 and A6, and based on the log-likelihood function value of the two-layer topic model, detect abnormal behaviors in video frames.

The above-mentioned step A7 specifically includes:

A71. On the document of the entire video document set—the visual word count matrix M _260×6480 , randomly select 80% of the video documents to form the training video document set X, and the remaining 20% of the video document set form the test set Y=MX ;

(其中

y_pi＝Y(p,i)，φ_pk是第一层BNBP-PFA主题模型所得主题分布，θ_ki是视频帧y_pi在主题φ_pk上的分布的系数，N₁＝6480是视频帧y_pi长度即其包含的视觉词的数量)，计算每个视频帧y_pi在第一层BNBP-PFA主题模型上的归一化的似然函数值F₁；A72. On the test set Y, according to the formula

(in

y _pi =Y(p,i), φ _pk is the topic distribution obtained by the first-layer BNBP-PFA topic model, θ _ki is the coefficient of the distribution of video frame y _pi on the topic φ _pk , N ₁ =6480 is the video frame y The length of _pi is the number of visual words it contains), calculate the normalized likelihood function value F ₁ of each video frame y _pi on the first-layer BNBP-PFA topic model;

(其中

θ_ki是视频帧y_pi在主题φ_pk上的分布的系数，φ′_kk′是第二层BNBP-PFA主题模型所得主题分布，θ′_k′i是θ_ki在主题φ′_kk′上的分布的系数，N₂＝15是θ_ki向量的长度即其包含的主题φ_pk的数量)，计算每个视频帧y_pi在主题φ_pk上的分布的系数θ_ki在第二层BNBP-PFA主题模型上的归一化的似然函数值F₂；A73. According to the formula

(in

θ _ki is the coefficient of the distribution of video frame y _pi on the topic φ _pk , φ′ _kk′ is the topic distribution obtained by the second-layer _BNBP -PFA topic model, θ′ k′ i is the distribution of θ _ki on the topic φ′ _kk′ The coefficient of distribution, N ₂ =15 is the length of the θ _ki vector (that is, the number of topics φ _pk it contains), and the coefficient θ _ki of the distribution of each video frame y _pi on the topic φ _pk is calculated in the second layer BNBP-PFA the normalized likelihood function value F ₂ on the topic model;

A74. Calculate the weighted likelihood function value F=η·F ₁ +(1-η)·F ₂ of the video frame y _pi on the two-layer BNBP-PFA topic model, where the parameter η=0.5;

A75. Compare the likelihood function value F calculated in A74 with the given threshold value Th ₁ =0.1, if there is F<Th ₁ , the video frame y _pi contains abnormal behavior, otherwise it does not.

Detecting each video document on the test dataset yields video frames that contain anomalous traffic behaviors. In Figure 8, the four abnormal traffic behaviors detected by the method of the present invention on the QMUL Junction Dataset 2 dataset are: (1) pedestrians cross the road without crossing the zebra crossing, (2) pedestrians cross the road on the sidewalk and run a red light, ( 3) The vehicle changes lanes in the middle of the intersection, (4) The vehicle passes between the two vehicles. In Figure 8, the objects and locations of abnormal behaviors are marked with red boxes.

In order to quantitatively evaluate the performance of the abnormal traffic behavior detection method proposed by the present invention, the method proposed by the present invention is compared with the MCTM method and the LDA method in the literature [6]. For the convenience of comparison experiments, the data of abnormal traffic behavior detection by the MCTM method and the LDA method on the QMUL Junction Dataset 2 dataset directly refer to the results in the literature [6]. Since the abnormal behavior patterns (3) and (4) shown in Figure 5 appear less on the QMUL Junction Dataset 2 dataset, and the MCTM method only roughly detects two abnormal traffic patterns of pedestrians, which corresponds to the one in Figure 5 The abnormal behaviors (1) and (2) are the same, so the present invention only uses the data of two abnormal behaviors for comparative experiments. Table 3 below presents the experimental results of the anomaly detection method, MCTM and LDA method of the present invention on the QMUL Junction Dataset 2 dataset, respectively.

Table 3. Experimental results of anomaly detection performance comparison of various methods on QMUL Junction Dataset 2

It can be seen from the experimental results in Table 3 that on the QMUL Junction Dataset 2 dataset, the abnormal traffic behavior detection method proposed by the present invention achieves the best results in the detection of two abnormal behaviors of pedestrians crossing the road and pedestrians running red lights, and the total TPR ( True rate) gets the maximum value and total FPR (false positive rate) gets the minimum value. To sum up, the method of the present invention achieves better abnormal behavior detection ability than MCTM and LDA methods on the QMUL Junction Dataset 2 dataset.

The above are only some embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

references

[1] X. Wang, X. Ma, E. Grimson, Unsupervised activity perception byhierarchical Bayesian models, in: IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8.

[2] L. Song, F. Jiang, Z. Shi, A. Katsaggelos, "Understanding dynamic scenes by hierarchical motion pattern mining", IEEE International Conference on Multimedia and Expo (ICME), pp.1–6, 2011.

[3] K. Than, and T.B. Ho, "Fully sparse topic models", Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases-Volume Part I, 2012.

[4] Liao W, Rosenhahn B, Yang M Y. Video Event Recognition by CombiningHDP and Gaussian Process[C].IEEE International Conference on Computer VisionWorkshop.IEEE,2015:166-174.

[5] Blei D M, Ng A Y, Jordan M I. Latent dirichlet allocation[J]. Journal of Machine Learning Research, 2003, 3:993-1022.

[6] T.Hospedales, S.Gong and T.Xiang, "A Markov Clustering Topic Model for Mining Behaviour in Video," in Proc.Int'l.Conf.Computer Vision, pp.1165-1172, 2009.

Claims (6)

1. the detection method of the traffic pattern of a traffic intersection video surveillance and abnormal behavior, is characterized in that, comprises the following steps: A1, the traffic intersection video with the duration of T seconds is divided into short video clips with the length of T _s seconds according to the time sequence, and each video clip is used as a video file, and N=T/T _s video files are obtained in total; A2, for each video document obtained in A1, calculate the optical flow vector of each adjacent pair of video frames; A3, quantify the optical flow vector obtained in A2 to obtain the video words of each pair of video frames of each video document; A4, based on the bag-of-words model, count the count vectors of the video words of each video document, and obtain the document-word count matrix M of the video document set composed of the entire long video; A5, use the BNBP-PFA topic model to extract the topic of the video document obtained in A4, and obtain the topic-word distribution and document-topic distribution, and the obtained topic is the simple traffic pattern in the video; A6, take the topic obtained in A5 as a new word, take the document-topic distribution obtained in A5 as a new document, use the BNBP-PFA topic model to extract the topic, and obtain the topic-word and document-topic of the second-layer topic model. distribution, the resulting theme is the complex traffic pattern in the video; A7, on the basis of the two-layer BNBP-PFA topic model obtained in A5 and A6, based on the topic-word distribution, document-topic distribution of the first-layer topic model and the topic-word distribution of the second-layer topic model in the two-layer topic model distribution, document-topic distribution, to calculate the log-likelihood function value of the video frame, and detect abnormal behavior in the video frame. 2. The traffic pattern of traffic intersection video surveillance according to claim 1 and the detection method of abnormal behavior, it is characterized in that, above-mentioned step A2 specifically comprises: 1). Calculate the optical flow information vector (v ₁ (a, b), v ₂ (a, b)) of each pixel (a, b) for two adjacent consecutive video frames I ₁ , I ₂ ; 2).According to the formula

and

Obtain the intensity and direction information (M(a,b), D(a,b)) of the optical flow of each pixel point. 3. the traffic pattern of traffic intersection video surveillance according to claim 2 and the detection method of abnormal behavior, it is characterized in that, in above-mentioned step A3, the process that optical flow vector is quantified to obtain video word specifically comprises: 1). Divide the position information of the optical flow: divide the video frame of size N _x ×N _y into N ₁ ×N ₁ pixel blocks, and obtain a total of

pixel blocks, where the symbol [*] represents an integer, N ₁ , N _x , and N _y are all positive integers, and the coordinates of the center point of the pixel block are used as the coordinates of the block; 2). Quantization of optical flow intensity and direction: each pixel block contains N ₁ ² pixels, and the average optical flow value of these N ₁ ² pixels is used as the optical flow vector of the pixel block

When the optical flow intensity value of the block exceeds the preset threshold Th, that is,

When , determine that the pixel block is a motion pixel block, otherwise it is a background pixel block; quantize the optical flow direction of the block, and quantize it into S directions, where Th∈(0,1); 3). According to the above quantification method, the size of the vocabulary of the video document set that can be obtained is:

4. The traffic pattern of traffic intersection video surveillance according to claim 3 and the detection method of abnormal behavior, it is characterized in that, above-mentioned step A5 specifically comprises: 1). Denote the document count matrix obtained in A4 as M _ij ∈R ^P×N , the count matrix contains P features of N documents, according to the formula

m _ijk ～Pois(φ _ik θ _kj ), φ _ik ～Dir(α _φ ,…,α _φ ), θ _kj ～Gamma(r _k ,p _k /(1-p _k )), r _k ～Gamma(c ₀ r ₀ ,1/c ₀ ), BNBP-PFA topic modeling process of p _k ~ Beta(cε,c(1-ε)), K topic distribution matrices Φ∈R ^P×K and topic-words can be obtained The distribution φ _ik , and the distribution matrix Θ∈R ^K×N of K topics in N documents and the document-topic distribution θ _kj , where the topic distribution matrix represents the distribution of K topics on P features; 2). Using the commonly used Markov Chain Monte Carlo inference algorithm, the probability distribution φ _ik , θ _kj and related parameters r _k and pk of the _BNBP -PFA topic model in 1) in step A5 above can be obtained. The reasoning algorithm is as follows: a) note

The upper bound of the number of topics K is determined by cγαBeta(cε, c(1-ε)), where P is the number of features, N is the number of documents, c=1, γ=1, α=1, ε=0.05; b) m _ijk is obtained by sampling according to the following formula (1);

in

c) Utilize the relationship between Poisson distribution and multinomial distribution, and the relational expression

It can be known that p([m _1jk ,...,m _pjk ]|-)=Mult(m _·jk ; φ _ik ), then φ _k can be obtained by sampling according to the following formula (2); p(φ _ik |-)～Dir(α _φ +m _1·k ,…,α _φ +m _P·k ) (2) d) After marginalizing φ _ik and θ _kj , m _jk ~NB(r _k ,p _k ), p _k ~Beta(cε,c(1-ε)), then p _k can be sampled by the following formula (3) get; p(p _k |-)～Beta(cε+m _··k ,c(1-ε)+Nr _k ) (3) e) due to

Then, r _k can be obtained by sampling according to the following formula (4);

5. The traffic pattern of traffic intersection video surveillance according to claim 4 and the detection method of abnormal behavior, it is characterized in that, above-mentioned step A6 specifically comprises: 1) The topic φ _ik obtained in step A5 is regarded as the word in step A6, and the document in A6-word distribution θ _kj is regarded as composed of the topic in A5, according to the formula

φ′ _k′ ～Dir(α′ _φ′ ,…,α′ _φ′ ), θ′ _k′j ～Gamma(r′ _k′ ,p′ _k′ /(1-p′ _k′ )), r′ The BNBP-PFA topic modeling process of _k′ ～Gamma(c′ ₀ r′ ₀ ,1/c′ ₀ ), p′ _k′ ～Beta(c′ε′,c′(1-ε′)) can be Obtain K' topics-word distribution φ'_kk' and document-topic distribution θ'_k'j; 2) Using the commonly used Markov Chain Monte Carlo inference algorithm, the probability distribution of the BNBP-PFA topic model in 1) in step A6 above can be obtained φ′ _kk′ , θ′ _k′j and related parameters r′ _{k ′} and p′ _k′ , the specific reasoning algorithm is as follows: a) note

The upper bound of the number of topics K′ is determined by c′γ′α′Beta(c′ε′,c′(1-ε′)), where K is the number of features, the number of topics in the first layer, and N is the number of video documents , c'=1, γ'=1, α'=1, ε'=0.05; b) obtain θ _kjk′ by sampling according to the following formula (5);

[θ _kj1 ,…,θ _kjK′ ]～Mult(θ _kj ;ζ′ _kj1 ,…,ζ′ _kjK′ ) (5) c) Utilize the relationship between Poisson distribution and multinomial distribution, and the relational expression

It can be known that p([θ _1jk′ ,...,θ _kjk′ ]|-)=Mult(θ _·jk′ ; φ′ _k′ ), then φ′ _k ′ can be obtained by sampling according to the following formula (6); p(φ′ _k′ |-)～Dir(α′ _φ′ +θ _1·k′ ,…,α′ _φ′ +θ _K·k′ ) (6) d) After marginalizing φ′ _{k′ and} θ′ _k′j , θ _·jk′ ~NB(r′k′,p′k _′ ), p′k _′ ~Beta(c′ε′,c′(1 -ε′)), then p′ _k′ can be sampled by the following formula (7); p(p′ _k′ |-)～Beta(c′ε′+θ _··k′ ,c′(1-ε′)+N′r′ _k′ ) (7) e) due to

Then r′ _k′ can be obtained by sampling according to the following formula (8);

6. The method for detecting traffic patterns and abnormal behavior of traffic intersection video monitoring according to claim 5, wherein the above-mentioned step A7 specifically comprises: A71. On the document-visual word count matrix M of the entire video document set, randomly select 80% of the video documents to form the training video document set X, and the remaining 20% of the video document set form the test set Y=M-X; A72. On the test set Y, according to the formula

in

y _pi = Y(p,i), φ _pk is the topic distribution obtained by the first-layer BNBP-PFA topic model, θ _ki is the coefficient of the distribution of video frame y _pi on topic φ _pk , N ₁ is the length of video frame y _pi That is, the number of visual words it contains, calculate the normalized likelihood function value F ₁ of each video frame y _pi on the first-layer BNBP-PFA topic model; A73. According to the formula

in

θ _ki is the coefficient of the distribution of video frame y _pi on the topic φ _pk , φ′ _kk′ is the topic distribution obtained by the second-layer _BNBP -PFA topic model, θ′ k′ i is the distribution of θ _ki on the topic φ′ _kk′ Coefficient of distribution, _N2 is the length of the θ _ki vector i.e. the number of topics φ _pk it contains, calculate the coefficient θ _ki of the distribution of each video frame y _pi on the topic φ _pk on the second-layer BNBP-PFA topic model The normalized likelihood function value F ₂ of ; A74. Calculate the weighted likelihood function value F=η·F ₁ +(1-η)·F ₂ of the video frame y _pi on the two-layer BNBP-PFA topic model, where the parameter η∈(0,1) is taken; A75. Compare the likelihood function value F calculated in A74 with the given threshold Th ₁ , if there is F < Th ₁ , where Th ₁ ∈(0,1), the video frame y _pi contains Unusual behavior, otherwise nothing.

Images