[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Abnormal event detection via covariance matrix for optical flow based feature

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Abnormal event detection is one of the most important objectives in security surveillance for public scenes. In this paper, a new high-performance algorithm based on spatio-temporal motion information is proposed to detect global abnormal events from the video stream as well as the local abnormal event. We firstly propose a feature descriptor to represent the movement by adopting the covariance matrix coding optical flow and the corresponding partial derivatives of multiple connective frames or the patches of the frames. The covariance matrix of multi-RoI (region of interest) which consists of frames or patches can represent the movement in high accuracy. For public surveillance video, the normal samples are abundant while there are few abnormal samples. Thus the one-class classification method is suitable for handling this problem inherently. The nonlinear one-class support vector machine based on a proposed kernel for Lie group element is applied to detect abnormal events by merely training the normal samples. The computational complexity and time performance of the proposed method is analyzed. The PETS, UMN and UCSD benchmark datasets are employed to verify the advantages of the proposed method for both global abnormal and local abnormal event detection. This method can be used for event detection for a surveillance video and outperforms the state-of-the-art algorithms. Thus it can be adopted to detect the abnormal event in the monitoring video.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Adam A, Rivlin E, Shimshoni I, Reinitz D (2008) Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell 30(3):555–560

    Article  Google Scholar 

  2. Benezeth Y, Jodoin PM, Saligrama V (2011) Abnormality detection using low-level co-occurring events. Pattern Recogn Lett 32(3):423–431

    Article  Google Scholar 

  3. Bhatnagar G, Wu QJ, Raman B (2013) Discrete fractional wavelet transform and its application to multiple encryption. Inf Sci 223:297–316

    Article  MathSciNet  MATH  Google Scholar 

  4. Bianco S, Ciocca G, Schettini R (2015) How far can you get by combining change detection algorithms? arXiv:1505.02921

  5. Bojanowski P, Bach F, Laptev I, Ponce J, Schmid C, Sivic J et al (2013) Finding actors and actions in movies. In: Proceedings of IEEE International Conference on Computer Vision (ICCV)

  6. Burton A, Radford J (1978) Thinking in perspective: critical essays in the study of thought processes. Methuen

  7. Canu S, Grandvalet Y, Guigue V, Rakotomamonjy A (2005) Svm and kernel methods matlab toolbox. Perception Systèmes et Information. INSA de Rouen, Rouen

    Google Scholar 

  8. Chen C, Ren Y, Kuo CCJ (2014) Large-scale indoor/outdoor image classification via expert decision fusion (edf). In: Asian Conference on Computer Vision (ACCV). Springer, Berlin, pp 426–442

  9. Cheng MM, Zhang Z, Lin WY, Torr P (2014) Bing: Binarized normed gradients for objectness estimation at 300fps. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3286–3293

  10. Cong Y, Yuan J, Liu J (2011) Sparse reconstruction cost for abnormal event detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3449–3456

  11. Cricri F, Roininen MJ, Leppanen J, Mate S, Curcio ID, Uhlmann S, Gabbouj M (2014) Sport type classification of mobile videos. IEEE Trans Multimed 16(4):917–932

    Article  Google Scholar 

  12. Cui P, Wang F, Sun LF, Zhang JW, Yang S (2012) A matrix-based approach to unsupervised human action categorization. IEEE Trans Multimed 14(1):102–110

    Article  Google Scholar 

  13. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 886–893, vol 1. IEEE, USA

  14. Ergezer H, Leblebicioġlu K (2016) Anomaly detection and activity perception using covariance descriptor for trajectories. In: European Conference on Computer Vision (ECCV). Springer, Berlin, pp 728–742

  15. Gorelick L, Blank M, Shechtman E, Irani M, Basri R (2007) Actions as space-time shapes. IEEE Trans Pattern Anal Mach Intell 29(12):2247–2253

    Article  Google Scholar 

  16. Hall B (2003) Lie groups, Lie algebras and representations: an elementary introduction, vol 222. Springer, Berlin

    Book  Google Scholar 

  17. Harandi M, Salzmann M, Porikli F (2014) Bregman divergences for infinite dimensional covariance matrices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1003–1010

  18. Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS (2016) Learning temporal regularity in video sequences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 733–742

  19. Henriques JF, Caseiro R, Martins P, Batista J (2012) Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision. Springer, Berlin, pp 702–715

  20. Horn BK, Schunck BG (1981) Determining optical flow. Artif Intell 17(1):185–203

    Article  Google Scholar 

  21. Hussein ME, Torki M, Gowayyed MA, El-Saban M (2013) Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), vol 13, pp 2466–2472

  22. Jiménez-Hernández H, González-Barbosa JJ, Garcia-Ramírez T (2010) Detecting abnormal vehicular dynamics at intersections based on an unsupervised learning approach and a stochastic model. Sensors 10(8):7576–7601

    Article  Google Scholar 

  23. Kalal Z, Matas J, Mikolajczyk K (2010) Pn learning: Bootstrapping binary classifiers by structural constraints. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 49–56

  24. Kosmopoulos D, Chatzis SP (2010) Robust visual behavior recognition. IEEE Signal Process Mag 27(5):34–45

    Article  Google Scholar 

  25. Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1–8

  26. Li W, Mahadevan V, Vasconcelos N (2014) Anomaly detection and localization in crowded scenes. IEEE Trans Pattern Anal Mach Intell 36(1):18–32

    Article  Google Scholar 

  27. Li H, Lin Z, Shen X, Brandt J, Hua G (2015) A convolutional neural network cascade for face detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 5325–5334

  28. Liu Y, Pados DA (2016) Compressed-sensed-domain l 1-pca video surveillance. IEEE Trans Multimed 18(3):351–363

    Article  Google Scholar 

  29. Lucas BD, Kanade T et al (1981) An iterative image registration technique with an application to stereo vision

  30. Mahadevan V, Li W, Bhalodia V, Vasconcelos N (2010) Anomaly detection in crowded scenes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1975–1981

  31. Mazloom M, Li X, Snoek C (2016) Tagbook: A semantic video representation without supervision for event detection. IEEE Trans Multimed 18(7):1378–1388

    Article  Google Scholar 

  32. Mehran R, Oyama A, Shah M (2009) Abnormal crowd behavior detection using social force model. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, pp 935–942

  33. Orozco J, Martinez B, Pantic M (2015) Empirical analysis of cascade deformable models for multi-view face detection. Image Vis Comput 42:47–61

    Article  Google Scholar 

  34. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British Machine Vision Conference, vol 1, p 6

  35. PETS (2009) Performance evaluation of tracking and surveillance (pets) 2009 benchmark data. multisensor sequences containing different crowd activities. http://www.cvg.rdg.ac.uk/pets2009/a.html

  36. Piciarelli C, Micheloni C, Foresti GL (2008) Trajectory-based anomalous event detection. IEEE Trans Circ Syst Video Technol 18(11):1544–1554

    Article  Google Scholar 

  37. Porikli F, Tuzel O (2005) Bayesian background modeling for foreground detection. In: Proceedings of the third ACM international workshop on Video surveillance & sensor networks (VSSN), pp 55–58

  38. Rosani A, Boato G, De Natale FG (2015) Eventmask: A game-based framework for event-saliency identification in images. IEEE Trans Multimed 17(8):1359–1371

    Article  Google Scholar 

  39. Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38

    Article  Google Scholar 

  40. Schölkopf B, Platt JC, Shawe-Taylor J, Smola AJ, Williamson RC (2001) Estimating the support of a high-dimensional distribution. Neurals Comput 13(7):1443–1471

    Article  MATH  Google Scholar 

  41. Shi Y, Gao Y, Wang R (2010) Real-time abnormal event detection in complicated scenes. In: Proceedings of International Conference on Pattern Recognition (ICPR), Istanbul, pp 3653–3656

  42. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: 1999. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 2. IEEE, Berlin

  43. Sun C, Nevatia R (2013) Active: Activity concept transitions in video event classification. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp 913–920

  44. Sun D, Roth S, Black MJ (2010) Secrets of optical flow estimation and their principles. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2432–2439

  45. Sun Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. pp 1988–1996

  46. Tang K, Fei-Fei L, Koller D (2012) Learning latent temporal structure for complex event detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1250–1257

  47. Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: 1991. Proceedings CVPR’91., IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, USA, pp 586–591

  48. Tuzel O, Porikli F, Meer P (2006) Region covariance: A fast descriptor for detection and classification. In: Proceeding of European Conference on Computer Vision (ECCV). Springer, Berlin, pp 589–600

  49. UCSD (2010) UCSD anomaly detection dataset, available from http://www.svcl.ucsd.edu/projects/anomaly/dataset.html

  50. UMN (2006) Unusual crowd activity dataset of university of minnesota, department of computer science and engineering, http://mha.cs.umn.edu/movies/crowd-activity-all.avi

  51. Utasi Á, Czúni L (2010) Detection of unusual optical flow patterns by multilevel hidden markov models. Opt Eng 49(1):017,201–017,201

    Article  Google Scholar 

  52. Varadarajan J, Odobez JM (2009) Topic models for scene analysis and abnormality detection. In: Proceedings of the 12th International Conference on Computer Vision Workshops (ICCV Workshops), pp 1338–1345

  53. Wang T, Snoussi H (2014) Detection of abnormal visual events via global optical flow orientation histogram. IEEE Trans Inf Forensic Secur 9(6):988–998

    Article  Google Scholar 

  54. Wang T, Chen J, Zhou Y, Snoussi H (2013) Online least squares one-class support vector machines based abnormal visual event detection. Sensors 13(12):17130–17155

  55. Wang F, Sun Z, Jiang YG, Ngo CW (2014) Video event detection using motion relativity and feature selection. IEEE Trans Multimed 16(5):1303–1315

    Article  Google Scholar 

  56. Warren DH, Strelow ER (2013) Electronic spatial sensing for the blind: contributions from perception, rehabilitation, and computer vision, vol 99. Springer Science & Business Media, Berlin

    Google Scholar 

  57. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  58. You X, Du L, Cheung Ym, Chen Q (2010) A blind watermarking scheme using new nontensor product wavelet filter banks. IEEE Trans Image Process 19(12):3271–3284

    Article  MathSciNet  MATH  Google Scholar 

  59. Zhang K, Zhang L, Yang MH (2014) Fast compressive tracking. IEEE Trans Pattern Anal Mach Intell 36(10):2002–2015

    Article  Google Scholar 

  60. Zhang X, Yang S, Tang YY, ZhangW(2016) A thermodynamics-inspired feature for anomaly detection on crowd motions in surveillance videos. Multimed Tools Appl 75(14):8799–8826

  61. Zhou S, Shen W, Zeng D, Fang M, Wei Y, Zhang Z (2016) Spatial–temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Signal Process Image Commun 47:358–368

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the National Natural Science Foundation of China (61503017, U1435220, 61365003), the Aeronautical Science Foundation of China (2016ZC51022), Gansu Province Basic Research Innovation Group Project (1506RJIA031), the Fundamental Research Funds for the Central Universities (YWF-14-RSC-102), the ANR AutoFerm project and the Platform CAPSEC funded by Région Champagne-Ardenne and FEDER.

Author information

Authors and Affiliations

  1. School of Automation Science and Electrical Engineering, Beihang University, Beihang, China

    Tian Wang, Meina Qiao & Yida Niu

  2. School of Computer Science and Technology, Nanjing Tech University, Nanjing, China

    Aichun Zhu

  3. College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou, China

    Ce Li

  4. Institut Charles Delaunay-LM2S-UMR STMR 6279 CNRS, University of Technology of Troyes, Troyes, France

    Hichem Snoussi

Authors
  1. Tian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meina Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aichun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yida Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ce Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hichem Snoussi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aichun Zhu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, T., Qiao, M., Zhu, A. et al. Abnormal event detection via covariance matrix for optical flow based feature. Multimed Tools Appl 77, 17375–17395 (2018). https://doi.org/10.1007/s11042-017-5309-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5309-2

Keywords

Search

Navigation