[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/2213836.2213974acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
poster

SigSpot: mining significant anomalous regions from time-evolving networks (abstract only)

Published: 20 May 2012 Publication History

Abstract

Anomaly detection in dynamic networks has a rich gamut of application domains, such as road networks, communication networks and water distribution networks. An anomalous event, such as a traffic accident, denial of service attack or a chemical spill, can cause a local shift from normal behavior in the network state that persists over an interval of time. Detecting such anomalous regions of network and time extent in large real-world networks is a challenging task. Existing anomaly detection techniques focus on either the time series associated with individual network edges or on global anomalies that affect the entire network. In order to detect anomalous regions, one needs to consider both the time and the affected network substructure jointly, which brings forth computational challenges due to the combinatorial nature of possible solutions.
We propose the problem of mining all Significant Anomalous Regions (SAR) in time-evolving networks that asks for the discovery of connected temporal subgraphs comprised of edges that significantly deviate from normal in a persistent manner. We propose an optimal Baseline algorithm for the problem and an efficient approximation, called S IG S POT. Compared to Baseline, SIGSPOT is up to one order of magnitude faster in real data, while achieving less than 10% average relative error rate. In synthetic datasets it is more than 30 times faster than Baseline with 94% accuracy and solves efficiently large instances that are infeasible (more than 10 hours running time) for Baseline. We demonstrate the utility of SIGSPOT for inferring accidents on road networks and study its scalability when detecting anomalies in social, transportation and synthetic evolving networks, spanning up to 1GB.

References

[1]
J. Abello, T. Eliassi-rad, and N. Devanur. Detecting Novel Discrepancies in Communication Networks. ICDM, 2010.
[2]
R. K. Ahuja, Ö. Ergun, J. B. Orlin, and A. P. Punnen. A survey of very large-scale neighborhood search techniques. Discr. Appl. Math., 123(1--3):75--102, 2002.
[3]
L. Akoglu and C. Faloutsos. Event detection in time series of mobile communication graphs. In Army Sc. Conf., 2010.
[4]
L. Akoglu, M. McGlohon, and C. Faloutsos. OddBall: Spotting Anomalies in Weighted Graphs. In PAKDD, 2010.
[5]
Z. Baig. On the use of pattern matching for rapid anomaly detection in smart grid infrastructures. In SmartGridComm, pages 214 --219, oct. 2011.
[6]
P. Bogdanov, M. Mongiovi, and A. K. Singh. Mining heavy subgraphs in time-evolving networks. In ICDM, 2011.
[7]
Y. Chen, S. Nyemba, W. Zhang, and B. Malin. Leveraging social networks to detect anomalous insider actions in collaborative environments. In ISI, 2011.
[8]
M. Davis, W. Liu, P. Miller, and G. Redpath. Detecting anomalies in graphs with numeric labels. In CIKM, 2011.
[9]
J. Diesner, T. L. Frantz, and K. M. Carley. Communication Networks from the Enron Email Corpus It's Always About the People. Enron is no Different. J. of Computational and Mathematical Organization Theory, 2006.
[10]
W. Eberle and L. Holder. Discovering structural anomalies in graph-based data. In ICDMW, 2007.
[11]
W. Eberle and L. Holder. Graph-based approaches to insider threat detection. In CSIIRW, 2009.
[12]
W. Eberle, L. Holder, and D. Cook. Identifying threats using graph-based anomaly detection. In Mach. Learn. in Cyber Trust. 2009.
[13]
W. He, G. Hu, and Y. Zhou. Large-scale ip network behavior anomaly detection and identification using substructure-based approach and multivariate time series mining. Telecom. Syst., 2012.
[14]
K. Henderson, T. Eliassi-Rad, S. Papadimitriou, and C. Faloutsos. HCDF: A hybrid community discovery algorithm. SIAM (SDM'2010), April-May 2010.
[15]
D. S. Johnson, M. Minkoff, and S. Phillips. The Prize Collecting Steiner Tree Problem : Theory and Practice. Proc. of SODA, 2000.
[16]
D. Q. Le, T. Jeong, H. E. Roman, and J. W.-K. Hong. Traffic dispersion graph based anomaly detection. In SoICT, 2011.
[17]
S.-d. Lin and H. Chalupsky. Unsupervised link discovery in multi-relational data via rarity analysis. In ICDM, 2003.
[18]
X. Ma, H. Xiao, S. Xie, Q. Li, Q. Luo, and C. Tian. Continuous, online monitoring and analysis in large water distribution networks. In ICDE, 2011.
[19]
J. Neil. Scan Statistics for the Online Detection of Locally Anomalous Subgraphs. PhD thesis, U. of New Mexico, 2011.
[20]
D. Neill and G. Cooper. A multivariate bayesian scan statistic for early event detection and characterization. Machine Learning, 79:261--282, 2010. 10.1007/s10994-009--5144--4.
[21]
D. B. Neill and A. W. Moore. Rapid detection of significant spatial clusters. In KDD, 2004.
[22]
D. B. Neill, A. W. Moore, M. Sabhnani, and K. Daniel. Detection of emerging space-time clusters. In KDD, 2005.
[23]
C. C. Noble and D. J. Cook. Graph-based anomaly detection. In KDD, 2003.
[24]
E. Papalexakis, N. Sidiropoulos, and M. Garofalakis. Reviewer profiling using sparse matrix regression. In Data Mining Workshops (ICDMW), 2010 IEEE International Conference on, pages 1214--1219. IEEE, 2010.
[25]
C. E. Priebe, J. M. Conroy, D. J. Marchette, and Y. Park. Scan statistics on enron graphs. Comput. Math. Organ. Theory, 11, 2005.
[26]
W. L. Ruzzo and M. Tompa. A linear time algorithm for finding all maximal scoring subsequences. In ISMB, 1999.
[27]
J. Sun, H. Qu, D. Chakrabarti, and C. Faloutsos. Relevance search and anomaly detection in bipartite graphs. KDD Explor. Newsl., 2005.
[28]
X. Wan, E. Milios, N. Kalyaniwalla, and J. Janssen. Link-based event detection in email communication networks. In SAC, 2009.
[29]
J. Wang, B.-H. Chou, and E. Suzuki. Finding the k-Most Abnormal Subgraphs from a Single Graph. In Discovery Science, LNCS. 2009.
[30]
H. R. Zeidanloo and A. B. A. Manaf. Botnet detection by monitoring similar communication patterns. CoRR, abs/1004.1232, 2010.
[31]
Y. Zhou, G. Hu, and W. He. Using graph to detect network traffic anomaly. In ICCCAS, 2009.

Cited By

View all
  • (2016)Efficient and flexible algorithms for monitoring distance-based outliers over data streamsInformation Systems10.1016/j.is.2015.07.00655:C(37-53)Online publication date: 1-Jan-2016

Index Terms

  1. SigSpot: mining significant anomalous regions from time-evolving networks (abstract only)

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGMOD '12: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data
    May 2012
    886 pages
    ISBN:9781450312479
    DOI:10.1145/2213836

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 May 2012

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. anomaly detection
    2. dynamic network
    3. significant anomalous regions

    Qualifiers

    • Poster

    Conference

    SIGMOD/PODS '12
    Sponsor:

    Acceptance Rates

    SIGMOD '12 Paper Acceptance Rate 48 of 289 submissions, 17%;
    Overall Acceptance Rate 785 of 4,003 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 30 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2016)Efficient and flexible algorithms for monitoring distance-based outliers over data streamsInformation Systems10.1016/j.is.2015.07.00655:C(37-53)Online publication date: 1-Jan-2016

    View Options

    View options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media