More Web Proxy on the site http://driver.im/

research-article

NM-GAN: : Noise-modulated generative adversarial network for video anomaly detection

Authors:

Tong JiaAuthors Info & Claims

Volume 116, Issue C

https://doi.org/10.1016/j.patcog.2021.107969

Published: 01 August 2021 Publication History

Highlights

•

A more accurate and stable model for video anomaly detection is achieved within a refined end-to-end GAN-like architecture.

•

The reconstruction network has stronger and more controllable generalization ability.

•

The discrimination network uses the reconstruction error map to distinguish anomaly samples.

•

The proposed noise-modulated adversarial learning method enhances the ability of the discriminator to detect anomalies.

Graphical abstract

Display Omitted

Abstract

As an important and challenging task for intelligent video surveillance systems, video anomaly detection is generally referred to as automatic recognition of video frames that contain abnormal targets, behavior or events. Although it has been widely applied in real scenes, anomaly detection remains a challenging task because of the vague definition of anomaly and the lack of the anomaly samples. Inspired by the widespread application of Generative Adversarial Network (GAN), we propose an end-to-end pipeline called NM-GAN which assembles an encode-decoder reconstruction network and a CNN-based discrimination network in a GAN-like architecture. The generalization ability of the reconstruction network is properly modulated via the adversarial learning around reconstruction error maps and noise maps. Meanwhile, the discrimination network is trained to distinguish anomaly samples from normal samples based on the reconstruction error maps. Finally, the output of the discrimination network is transferred to evaluate anomaly score of the input frame. The thorough proof-of-principle experiments and ablation tests on several popular datasets reveal that the proposed model enhance the generalization ability of the reconstruction network and the distinguishability of the discrimination network significantly. The comparison with the state-of-the-art shows that the proposed NM-GAN model outperforms most competing models in precision and stability.

References

[1]

Q. Sun, H. Liu, T. Harada, Online growing neural gas for anomaly detection in changing surveillance scenes, Pattern Recognit 64 (2017) 187–201,.

Digital Library

[2]

M. Sabokrou, M. Fayyaz, M. Fathy, Z. Moayed, R. Klette, Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes, Comput. Vision Image Understanding 172 (2018) 88–97,.

[3]

R. Ye, X. Li, Collective representation for abnormal event detection, J Comput Sci Technol 32 (2017) 470–479,.

[4]

M. Shuang, Y. Wang, G. Wen, Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model, Sensors 18 (4) (2018) 1064,.

[5]

T. Schlegl, P. Seeböck, S.M. Waldstein, U. Schmidt-Erfurth, G. Langs, Unsupervised anomaly detection with generative adversarial networks to guide marker discovery, International Conference on Information Processing in Medical Imaging. Springer, Cham (2017) 146–157,.

[6]

D. Li, S. Liu, H. Zhang, A method of anomaly detection and fault diagnosis with online adaptive learning under small training samples, Pattern Recognit 64 (2018) 374–385,.

Digital Library

[7]

D. Abati, A. Porrello, S. Calderara, R. Cucchiara, Latent space autoregression for novelty detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019) 481–490,.

[8]

M. Sabokrou, M. Khalooei, M. Fathy, E. Adeli, Adversarially learned one-class classifier for novelty detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018) 3379–3388,.

[9]

J. Wang, A. Cherian, Gods: generalized one-class discriminative subspaces for anomaly detection, International Conference on Computer Vision (2019) 8201–8211,.

[10]

M. Xu, X. Yu, D. Chen, C. Wu, Y. Jiang, An efficient anomaly detection system for crowded scenes using variational autoencoders, Applied Sciences 9 (2019) 3337,.

[11]

W. Liu, W. Luo, D. Lian, S. Gao, Future frame prediction for anomaly detection a new baseline, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018) 6536–6545,.

[12]

M. Sabokrou, M. Fathy, G. Zhao, E. Adeli, Deep end-to-end one-class classifier, IEEE Trans Neural Netw Learn Syst (2020) 1–10,.

[13]

M. Hasan, J. Choi, J. Neumann, M.K. Roy-Chowdhury, L.S. Davis, Learning temporal regularity in video sequences, Proceedings of the IEEE conference on computer vision and pattern recognition (2016) 733–742,.

[14]

Y. Zhao, B. Deng, C. Shen, Y. Liu, Spatio-temporal autoencoder for video anomaly detection, Proceedings of the 25th ACM international conference on Multimedia (2017) 1933–1941,.

Digital Library

[15]

M. Ravanbakhsh, M. Nabi, E. Sangineto, L. Marcenaro, C. Regazzoni, M. Sebe, Abnormal event detection in videos using generative adversarial nets, 2017 IEEE International Conference on Image Processin (2017) 1577–1581,.

Digital Library

[16]

A. Samet, A.-A. Amir, B. Toby P., Ganomaly: semi-supervised anomaly detection via adversarial training, Asian Conference on Computer Vision.Springer, Cham (2018) 622–637,.

[17]

N. Li, F. Chang, C. Liu, Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes, IEEE Trans Multimedia 99 (2020),.

[18]

R. Manassés, E.L. André, S.L. Heitor, A study of deep convolutional auto-encoders for anomaly detection in videos, Pattern Recognit Lett 105 (2018) 13–22,.

Digital Library

[19]

D. Xu, Y. Yan, E. Ricci, N. Sebe, Detecting anomalous events in videos by learning deep representations of appearance and motion, Computer Vision and Image Understandin 156 (2017) 117–127,.

Digital Library

[20]

Y.S. Chong, Y.H. Tay, Abnormal event detection in videos using spatiotemporal autoencoder, International Symposium on Neural Networks (2017) 189–196,.

[21]

R. Chalapathy, A.K. Menon, S. Chawla, Robust, deep and inductive anomaly detection, Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Cham (2017) 36–51,.

[22]

D. Gong, L. Liu, V. Le, B. Saha, M.R. Mansour, S. Venkatesh, A.V.D. Hengel, Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection, Proceedings of the IEEE International Conference on Computer Vision (2019) 1705–1714,.

[23]

H. Park, J. Noh, B. Ham, Learning memory-guided normality for anomaly detection, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020) 14372–14381,.

[24]

T.-N. Nguyen, J. Meunier, Anomaly detection in video sequence with appearance-motion correspondence, Proceedings of the IEEE International Conference on Computer Vision (2019) 1273–1283,.

[25]

I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A.C. Courville, Y. Bengio, Generative adversarial nets, In Advances in Neural Information Processing Systems (2014) 2672–2680.

[26]

B.R. Kiran, D.M. Thomas, R. Parakkal, An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos, Journal of Imaging 4 (2) (2018) 36,.

[27]

S. Lee, H.G. Kim, Y.M. Ro, Stan: spatio-temporal adversarial networks for abnormal event detection, International Conference on Acoustics, Speech, and Signal Processing (2018) 1323–1327,.

Digital Library

[28]

M. Sabokrou, M. Pourreza, M. Fayyaz, R. Entezari, M. Fathy, J. Gall, E. Adeli, Avid: adversarial visual irregularity detection, Asian Conference on Computer Vision (2018) 488–505,.

[29]

H. Song, C. Sun, X. Wu, M. Chen, Y. Jia, Learning normal patterns via adversarial attention-based autoencoder for abnormal event detection in videos, IEEE Trans Multimedia 99 (2019),.

[30]

F. Dong, Y. Zhang, X. Nie, Dual discriminator generative adversarial network for video anomaly detection, IEEE Access 99 (2020),.

[31]

M. Ravanbakhsh, E. Sangineto, M. Nabi, N. Sebe, Training adversarial discriminators for cross-channel abnormal event detection in crowds, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) (2019) 1896–1904,.

[32]

M. Pourreza, B. Mohammadi, M. Khaki, S. Bouindour, H. Snoussi, M. Sabokrou, G2d: Generate to detect anomalies, arXiv preprint arXiv:2006.11629 (2020).

[33]

O. Ronneberger, P. Fischer, T. Brox, U-Net: convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-assisted Intervention (2015) 234–241,.

[34]

Vijay Mahadevan W.L., Bhalodia V., Ucsd anomaly detection dataset.

[35]

J.S. Cewu Lu, J. Jia, Avenue dataset for abnormal event detection.

[36]

Liu W., Lou W., Lian D., Gao S., Shanghaitech campus dataset.

[37]

R.T. Ionescu, S. Smeureanu, B. Alexe, M. Popescu, Unmasking the abnormal events in video, Proceedings of the IEEE International Conference on Computer Vision (2017) 2895–2903,.

[38]

W. Luo, W. Liu, S. Gao, Remembering history with convolutional LSTM for anomaly detection, 2017 IEEE International Conference on Multimedia and Expo (ICME) (2017) 439–444,.

[39]

W. Luo, W. Liu, S. Gao, A revisit of sparse coding based anomaly detection in stacked RNN framework, Proceedings of the IEEE International Conference on Computer Visio (2017) 341–349,.

[40]

P. Perera, R. Nallapati, B. Xiang, Ocgan: one-class novelty detection using gans with constrained latent representations, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 18 (2019) 2898–2906,.

Cited By

Liu YYang DWang YLiu JLiu JBoukerche ASun PSong L(2024)Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep ModelsACM Computing Surveys10.1145/364510156:7(1-38)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3645101
Leyva RSanchez VLi CMaple C(2024)Specific event detection for video surveillance using variational Bayesian inferenceNeurocomputing10.1016/j.neucom.2024.128291603:COnline publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128291
Chiranjeevi VMalathi D(2024)Anomaly graph: leveraging dynamic graph convolutional networks for enhanced video anomaly detection in surveillance and security applicationsNeural Computing and Applications10.1007/s00521-024-09738-336:20(12011-12028)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00521-024-09738-3
Show More Cited By

Index Terms

NM-GAN: Noise-modulated generative adversarial network for video anomaly detection
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

DeepNAP: Deep neural anomaly pre-detection in a semiconductor fab
Abstract
Anomaly detection in an industrial process is crucial for preventing unexpected economic loss. Among various signals, multivariate time series signals are one of the most difficult signals to analyze for detecting anomalies. Moreover, ...
Attention-guided generator with dual discriminator GAN for real-time video anomaly detection
Abstract
Detecting anomalies in videos presents a significant challenge in the field of video surveillance. The primary goal is identifying and detecting uncommon actions or events within a video sequence. The difficulty arises from the limited ...
CVAD-GAN: Constrained video anomaly detection via generative adversarial network
Abstract
Automatic detection of abnormal behavior in video sequences is a fundamental and challenging problem for intelligent video surveillance systems. However, the existing state-of-the-art Video Anomaly Detection (VAD) methods are computationally ...
Highlights
- The paper introduced a novel GAN-based framework for real-time video anomaly detection.
- The proposed model (CVAD-GAN) is robust and attained competitive AUC score on datasets.
- The model used dilated convolution layers and skip ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Pattern Recognition

Pattern Recognition Volume 116, Issue C

Aug 2021

405 pages

ISSN:0031-3203

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 August 2021

Author Tags

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu YYang DWang YLiu JLiu JBoukerche ASun PSong L(2024)Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep ModelsACM Computing Surveys10.1145/364510156:7(1-38)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3645101
Leyva RSanchez VLi CMaple C(2024)Specific event detection for video surveillance using variational Bayesian inferenceNeurocomputing10.1016/j.neucom.2024.128291603:COnline publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128291
Chiranjeevi VMalathi D(2024)Anomaly graph: leveraging dynamic graph convolutional networks for enhanced video anomaly detection in surveillance and security applicationsNeural Computing and Applications10.1007/s00521-024-09738-336:20(12011-12028)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00521-024-09738-3
Megahed MMohammed A(2024)A comprehensive review of generative adversarial networksWIREs Computational Statistics10.1002/wics.162916:1Online publication date: 21-Jan-2024
https://dl.acm.org/doi/10.1002/wics.1629
Ferrari NFraccaroli MLamma E(2023)GRD-NetInternational Journal of Intelligent Systems10.1155/2023/77734812023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/7773481
Li HShen XSun XWang YLi CChen J(2023)Video anomaly detection based on scene classificationMultimedia Tools and Applications10.1007/s11042-023-15328-782:29(45345-45365)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s11042-023-15328-7
Song GQian YWang Y(2023)Analysis of abnormal pedestrian behaviors at grade crossings based on semi-supervised generative adversarial networksApplied Intelligence10.1007/s10489-023-04639-953:19(21676-21691)Online publication date: 8-Jun-2023
https://dl.acm.org/doi/10.1007/s10489-023-04639-9
Khan MMenouar HHamila R(2023)Visual crowd analysisAI Magazine10.1002/aaai.1211744:3(296-311)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1002/aaai.12117
Bao QLiu FLiu YJiao LLiu XLi LMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Hierarchical Scene Normality-Binding Modeling for Anomaly Detection in Surveillance VideosProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548199(6103-6112)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3548199
Li CLi HZhang G(2022)Future frame prediction based on generative assistant discriminative network for anomaly detectionApplied Intelligence10.1007/s10489-022-03488-253:1(542-559)Online publication date: 20-Apr-2022
https://dl.acm.org/doi/10.1007/s10489-022-03488-2

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents