More Web Proxy on the site http://driver.im/

short-paper

Self-supervision Meets Adversarial Perturbation: A Novel Framework for Anomaly Detection

Authors:

Yun FuAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 4555 - 4559

https://doi.org/10.1145/3511808.3557697

Published: 17 October 2022 Publication History

Abstract

Anomaly detection is a fundamental yet challenging problem in machine learning due to the lack of label information. In this work, we propose a novel and powerful framework, dubbed as SLA²P, for unsupervised anomaly detection. After extracting representative embeddings from raw data, we apply random projections to the features and regard features transformed by different projections as belonging to distinct pseudo-classes. We then train a classifier network on these transformed features to perform self-supervised learning. Next, we add adversarial perturbation to the transformed features to decrease their softmax scores of the predicted labels and design anomaly scores based on the predictive uncertainties of the classifier on these perturbed features. Our motivation is that because of the relatively small number and the decentralized modes of anomalies, 1) the pseudo label classifier's training concentrates more on learning the semantic information of normal data rather than anomalous data; 2) the transformed features of the normal data are more robust to the perturbations than those of the anomalies. Consequently, the perturbed transformed features of anomalies fail to be classified well and accordingly have lower anomaly scores than those of the normal samples. Extensive experiments on image, text, and inherently tabular benchmark datasets back up our findings and indicate that SLA² achieves state-of-the-art anomaly detection performance consistently. Our code is made publicly available at https://github.com/wyzjack/SLA2P

References

[1]

Davide Abati, Angelo Porrello, Simone Calderara, and Rita Cucchiara. 2019. Latent Space Autoregression for Novelty Detection. In CVPR.

[2]

Aderemi O Adewumi and Andronicus A Akinyelu. 2017. A survey of machine-learning and nature-inspired based credit card fraud detection techniques. International Journal of System Assurance Engineering and Management (2017).

[3]

Mennatallah Amer, Markus Goldstein, and Slim Abdennadher. 2013. Enhancing one-class support vector machines for unsupervised anomaly detection. In Proceedings of the ACM SIGKDD workshop on outlier detection and description.

Digital Library

[4]

Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. 2000. LOF: identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data.

Digital Library

[5]

Glenn W Brier et al. 1950. Verification of forecasts expressed in terms of probability. Monthly weather review (1950).

[6]

Eugênio F. Cabral and Robson L.F. Cordeiro. 2020. Fast and Scalable Outlier Detection with Sorted Hypercubes. In CIKM.

[7]

Raghavendra Chalapathy and Sanjay Chawla. 2019. Deep learning for anomaly detection: A survey. arXiv preprint arXiv:1901.03407 (2019).

[8]

Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2007. Outlier detection: A survey. Comput. Surveys (2007).

[9]

Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly detection: A survey. ACM computing surveys (CSUR) (2009).

Digital Library

[10]

Jinghui Chen, Saket Sathe, Charu C. Aggarwal, and Deepak S. Turaga. 2017. Outlier Detection with Autoencoder Ensembles. In SDM.

[11]

Minshuo Chen, Yizhou Wang, Tianyi Liu, Zhuoran Yang, Xingguo Li, Zhaoran Wang, and Tuo Zhao. 2020. On Computation and Generalization of Generative Adversarial Imitation Learning. In ICLR.

[12]

Zhengying Chen, Yonghong Tian, Wei Zeng, and Tiejun Huang. 2015. Detecting abnormal behaviors in surveillance videos based on fuzzy clustering and multiple auto-encoders. In ICME.

[13]

Changyu Deng, Yizhou Wang, Can Qin, Yun Fu, and Wei Lu. 2022. Self-directed online machine learning for topology optimization. Nature communications, Vol. 13, 1 (2022), 1--14.

[14]

Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml

[15]

Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD.

[16]

Li Fei-Fei, Rob Fergus, and Pietro Perona. 2004. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In CVPRW.

[17]

Kang Fu, Dawei Cheng, Yi Tu, and Liqing Zhang. 2016. Credit card fraud detection using convolutional neural networks. In International Conference on Neural Information Processing.

[18]

Spyros Gidaris, Praveer Singh, and Nikos Komodakis. 2018. Unsupervised Representation Learning by Predicting Image Rotations. In ICLR.

[19]

Izhak Golan and Ran El-Yaniv. 2018. Deep Anomaly Detection Using Geometric Transformations. In NeurIPS.

[20]

Xiang Gu, Jian Sun, and Zongben Xu. 2020. Spherical Space Domain Adaptation With Robust Pseudo-Label Loss. In CVPR.

[21]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR.

[22]

Zengyou He, Xiaofei Xu, and Shengchun Deng. 2003. Discovering cluster-based local outliers. Pattern Recognition Letters (2003).

[23]

Samir Khan and Takehisa Yairi. 2018. A review on the application of deep learning in system health management. Mechanical Systems and Signal Processing (2018).

[24]

JooSeuk Kim and Clayton D Scott. 2012. Robust kernel density estimation. JMLR (2012).

[25]

Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009).

[26]

Donghwoon Kwon, Hyunjoo Kim, Jinoh Kim, Sang C Suh, Ikkyun Kim, and Kuinam J Kim. 2019. A survey of deep learning-based network anomaly detection. Cluster Computing (2019).

[27]

Chieh-Hsin Lai, Dongmian Zou, and Gilad Lerman. 2020a. Robust Subspace Recovery Layer for Unsupervised Anomaly Detection. In ICLR.

[28]

Chieh-Hsin Lai, Dongmian Zou, and Gilad Lerman. 2020b. Novelty Detection via Robust Variational Autoencoding. arXiv preprint arXiv:2006.05534 (2020).

[29]

Ken Lang. 1995. Newsweeder: Learning to filter netnews. In Machine Learning Proceedings.

[30]

Gilad Lerman and Tyler Maunu. 2018. An overview of robust subspace recovery. Proc. IEEE (2018).

[31]

Kai Li, Chang Liu, Handong Zhao, Yulun Zhang, and Yun Fu. 2021. ECACL: A holistic framework for semi-supervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8578--8587.

[32]

Chang Liu, Lichen Wang, and Yun Fu. [n.d.]. Meta Adversarial Weight for Unsupervised Domain Adaptation. 10--18. https://doi.org/10.1137/1.9781611977172.2

[33]

Chang Liu, Lichen Wang, Kai Li, and Yun Fu. 2021. Domain Generalization via Feature Variation Decorrelation. In Proceedings of the 29th ACM International Conference on Multimedia. 1683--1691.

Digital Library

[34]

Chang Liu, Xiangrui Zeng, Kaiwen Wang, Qiang Guo, and Min Xu. 2018. Multi-task learning for macromolecule classification, segmentation and coarse structural recovery in cryo-tomography. arXiv preprint arXiv:1805.06332 (2018).

[35]

Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isolation forest. In ICDM.

Digital Library

[36]

Seonwoo Min, Byunghan Lee, and Sungroh Yoon. 2017. Deep learning in bioinformatics. Briefings in bioinformatics (2017).

[37]

Randy Paffenroth, Kathleen Kay, and Les Servi. 2018. Robust pca for anomaly detection in cyber networks. arXiv preprint arXiv:1801.01571 (2018).

[38]

Guansong Pang, Chunhua Shen, Longbing Cao, and Anton van den Hengel. 2020. Deep Learning for Anomaly Detection: A Review. arXiv preprint arXiv:2007.02500 (2020).

[39]

Pramuditha Perera, Ramesh Nallapati, and Bing Xiang. 2019. OCGAN: One-Class Novelty Detection Using GANs With Constrained Latent Representations. In CVPR.

[40]

Stanislav Pidhorskyi, Ranya Almohsen, and Gianfranco Doretto. 2018. Generative Probabilistic Novelty Detection with Adversarial Autoencoders. In NeurIPS.

[41]

Anand Rajaraman and Jeffrey David Ullman. 2011. Mining of massive datasets. Cambridge University Press.

[42]

Bernhard Schölkopf, Robert C Williamson, Alex J Smola, John Shawe-Taylor, and John C Platt. 2000. Support vector method for novelty detection. In NeurIPS. 582--588.

[43]

Aihua Shen, Rencheng Tong, and Yaochen Deng. 2007. Application of classification models on credit card fraud detection. In 2007 International conference on service systems and service management. IEEE, 1--4.

[44]

Mei-Ling Shyu, Shu-Ching Chen, Kanoksri Sarinnapakorn, and LiWu Chang. 2003. A novel anomaly detection scheme based on principal component classifier. Technical Report. MIAMI UNIV CORAL GABLES FL DEPT OF ELECTRICAL AND COMPUTER ENGINEERING.

[45]

Waqas Sultani, Chen Chen, and Mubarak Shah. 2018. Real-World Anomaly Detection in Surveillance Videos. In CVPR.

[46]

Swee Chuan Tan, Kai Ming Ting, and Fei Tony Liu. 2011. Fast Anomaly Detection for Streaming Data. In IJCAI.

[47]

Siqi Wang, Yijie Zeng, Xinwang Liu, En Zhu, Jianping Yin, Chuanfu Xu, and Marius Kloft. 2019. Effective End-to-end Unsupervised Outlier Detection via Inlier Priority of Discriminative Network. In NeurIPS.

[48]

Yizhou Wang, Yue Kang, Can Qin, Huan Wang, Yi Xu, Yulun Zhang, and Yun Fu. 2021. Adapting stepsizes by momentumized gradients improves optimization and generalization. arXiv preprint arXiv:2106.11514 (2021).

[49]

Haolong Xiang, Zoran Salcic, Wanchun Dou, Xiaolong Xu, Lianyong Qi, and Xuyun Zhang. 2020. OPHiForest: Order Preserving Hashing Based Isolation Forest for Robust and Scalable Anomaly Detection. In CIKM.

[50]

Xingwei Yang, Longin Jan Latecki, and Dragoljub Pokrajac. 2009. Outlier detection with globally optimal exemplar-based GMM. In SDM.

[51]

Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, and Vijay Ramaseshan Chandrasekhar. 2018. Efficient gan-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018).

[52]

Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Dae-ki Cho, and Haifeng Chen. 2018. Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection. In ICLR.

Cited By

Wang YQin CWei RXu YBai YFu Y(2024)SLA2P: Self-Supervised Anomaly Detection With Adversarial PerturbationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.344847336:12(9282-9293)Online publication date: Dec-2024
https://doi.org/10.1109/TKDE.2024.3448473
Huang HLi BLi T(2024)A Cooperative Differential Evolution Based Intrusion Detection System for Unknown CyberattacksIEEE INFOCOM 2024 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)10.1109/INFOCOMWKSHPS61880.2024.10620711(1-2)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOMWKSHPS61880.2024.10620711
Wang YKang YQin CWang HXu YZhang YFu Y(2023)Momentum is All You Need for Data-Driven Adaptive Optimization2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00179(1385-1390)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00179

Index Terms

Self-supervision Meets Adversarial Perturbation: A Novel Framework for Anomaly Detection
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Anomaly detection
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Unsupervised learning and clustering

Recommendations

Better Self-training for Image Classification Through Self-supervision
AI 2021: Advances in Artificial Intelligence
Abstract
Self-training is a simple semi-supervised learning approach: Unlabelled examples that attract high-confidence predictions are labelled with their predictions and added to the training set, with this process being repeated multiple times. Recently, ...
Efficient time series anomaly detection by multiresolution self-supervised discriminative network
Abstract
Time series anomaly detection aims to identify abnormal subsequences in time series that are markedly different from the temporal behaviors of the entire sequence. Although previous density-based or proximity-based anomaly detection ...
Self-Supervision Can Be a Good Few-Shot Learner
Computer Vision – ECCV 2022
Abstract
Existing few-shot learning (FSL) methods rely on training with a large labeled dataset, which prevents them from leveraging abundant unlabeled data. From an information-theoretic perspective, we propose an effective unsupervised FSL method, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
554
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)11

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YQin CWei RXu YBai YFu Y(2024)SLA2P: Self-Supervised Anomaly Detection With Adversarial PerturbationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.344847336:12(9282-9293)Online publication date: Dec-2024
https://doi.org/10.1109/TKDE.2024.3448473
Huang HLi BLi T(2024)A Cooperative Differential Evolution Based Intrusion Detection System for Unknown CyberattacksIEEE INFOCOM 2024 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)10.1109/INFOCOMWKSHPS61880.2024.10620711(1-2)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOMWKSHPS61880.2024.10620711
Wang YKang YQin CWang HXu YZhang YFu Y(2023)Momentum is All You Need for Data-Driven Adaptive Optimization2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00179(1385-1390)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00179

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents