More Web Proxy on the site http://driver.im/

research-article

Anomaly detection via feature-aided tracking and hidden Markov models

Authors:

Krishna Pattipati,

Peter WillettAuthors Info & Claims

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, Volume 39, Issue 1

Pages 144 - 159

https://doi.org/10.1109/TSMCA.2008.2007944

Published: 01 January 2009 Publication History

Abstract

The problem of detecting an anomaly (or abnormal event) is such that the distribution of observations is different before and after an unknown onset time, and the objective is to detect the change by statistically matching the observed pattern with that predicted by a model. In the context of asymmetric threats, the detection of an abnormal situation refers to the discovery of suspicious activities of a hostile nation or group out of noisy, scattered, and partial intelligence data. The problem becomes complex in a low signal-to-noise ratio environment, such as asymmetric threats, because the "signal" observations are far fewer than "noise" observations. Furthermore, the signal observations are "hidden" in the noise. In this paper, we illustrate the capabilities of hidden Markov models (HMMs), combined with feature-aided tracking, for the detection of asymmetric threats. A transaction-based probabilistic model is proposed to combine HMMs and feature-aided tracking. A procedure analogous to Page's test is used for the quickest detection of abnormal events. The simulation results show that our method is able to detect the modeled pattern of an asymmetric threat with a high performance as compared to a maximum likelihood-based data mining technique. Performance analysis shows that the detection of HMMs improves with increase in the complexity of HMMs (i.e., the number of states in an HMM).

References

[1]

T. Fawcett, "Activity monitoring: Anomaly detection as on-line classification," in Proc. Symp. Mach. Learn. Anomaly Detection, May 2004.

[2]

E. Page, "Continuous inspection schemes," Biometrika, vol. 41, no. 1/2, pp. 100-115, Jun. 1954.

[3]

J. Ying, T. Kirubarajan, K. R. Pattipati, and A. Patterson-Hine, "A hidden Markov model-based algorithm for fault diagnosis with partial and imperfect tests," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 30, no. 4, pp. 463-473, Nov. 2000.

Digital Library

[4]

B. Chen and P. Willett, "Detection of hidden Markov model transient signals," IEEE Trans. Aerosp. Electron. Syst., vol. 36, no. 4, pp. 1253- 1268, Dec. 2000.

[5]

B. Chen and P. Willett, "Superimposed HMM transient detection via target tracking ideas," IEEE Trans. Aerosp. Electron. Syst., vol. 37, no. 3, pp. 946-956, Jul. 2001.

[6]

L. R. Rabiner and B. H. Juang, "An introduction to hidden Markov models," IEEE ASSP Mag., vol. 3, no. 1, pp. 4-16, Jan. 1986.

[7]

L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.

[8]

P. Smyth, "Markov monitoring with unknown states," IEEE J. Sel. Areas Commun., vol. 12, no. 9, pp. 1600-1612, Dec. 1994.

Digital Library

[9]

S. Joshi and V. V. Phoha, "Investigating hidden Markov model for anomaly detection," in Proc. 43rd ACM South East Conf., Kennesaw, GA, 2005.

[10]

S. Salvador, P. Chan, and J. Brodie, "Learning states and rules for time series anomaly detection," in Proc. 17th Int. FLAIRS Conf., 2004, pp. 300-305.

[11]

D. Agarwal, J. Feng, and V. Torres, "Monitoring massive streams simultaneously: A holistic approach," in Proc. Interface, 2006.

[12]

S. Bay, K. Saito, N. Ueda, and P. Langley, "A framework for discovering anomalous regimes in multivariate time-series data with local models," in Proc. Symp. Mach. Learn. Anomaly Detection, May 2004.

[13]

G. Godfrey, TerrorAlert System, Metron. {Online}. Available: http:// www.metsci.com/about/terralert.html

[14]

J. A. Rosen, Influence Net Modeling. {Online}. Available: http://www. inet.saic.com/docs/docs/inet-for-planning.pdf

[15]

P. Schrodt, "Pattern recognition of international crises using hidden Markov models," in Political Complexity: Nonlinear Models of Politics, D. Richards, Ed. Ann Arbor, MI: Univ. of Michigan Press, 2000, pp. 296-328.

[16]

P. A. Schrodt and D. J. Gerner, "Using cluster analysis to derive early warning indicators for political change in the Middle East, 1979-1996," Amer. Polit. Sci. Rev., vol. 94, no. 4, pp. 803-818, Dec. 2000.

[17]

S. Singh, W. Donat, H. Tu, J. Lu, K. Pattipati, and P. Willett, "An advanced system for modeling asymmetric threats," in Proc. IEEE Int. Conf. Syst., Man Cybern., Taipei, Taiwan, Oct. 2006, pp. 3943-3948.

[18]

S. Singh, J. Allanach, H. Tu, K. Pattipati, and P. Willett, "Stochastic modeling of a terrorist event via the ASAM system," in Proc. IEEE Int. Conf. Syst., Man Cybern., Hague, The Netherlands, Oct. 2004, pp. 5673-5678.

[19]

S. Singh, H. Tu, J. Allanach, K. Pattipati, and P. Willett, "Modeling threats," IEEE Potentials, vol. 23, no. 3, pp. 18-21, Aug./Sep. 2004.

[20]

H. Tu, J. Allanach, S. Singh, P. Willett, and K. Pattipati, "Information integration via hierarchical and hybrid Bayesian networks," IEEE Trans. Syst., Man Cybern. A, Syst., Humans--Special Issue on Advances in Heterogeneous and Complex System Integration, vol. 1, no. 1, pp. 19-34, Jan. 2006.

[21]

Qualtech Systems Inc. {Online}. Available: http://www.teamqsi.com

[22]

L. Baum, T. Petrie, G. Soules, and N. Weiss, "A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains," Ann. Math. Stat., vol. 41, no. 1, pp. 164-171, 1970.

[23]

T. Moon, "The expectation-maximization algorithm," IEEE Signal Process. Mag., vol. 13, no. 6, pp. 47-60, Nov. 1996.

[24]

F. Barnaby, How to Build A Nuclear Bomb and Other Weapons of Mass Destruction. New York: Nation Books, 2004.

[25]

F. Settle, Nuclear Chemistry--Nuclear Proliferation. {Online}. Available: http://www.chemcases.com/2003version/nuclear/nc-12.htm

[26]

L. Spector and J. Smith, Nuclear Ambitions: The Spread of Nuclear Weapons 1989-1990. Boulder, CO: Westview, 1990.

[27]

R. Paternoster, Nuclear Weapon Proliferation Indicators and Observables. Los Alamos, NM: Los Alamos Nat. Lab., Dec. 1992.

[28]

"Technologies underlying weapons of mass destruction," Office Technol. Assess., U.S. Printing Office, U.S. Congr., Washington, DC, OTA-BPISC- 115, Dec. 1993.

[29]

A. Wald, Sequential Analysis. New York: Wiley, 1947.

[30]

D. Siegmund, Sequential Analysis: Tests and Confidence Intervals. New York: Springer-Verlag, 1985.

[31]

S. S. Blackman, Multiple Targets Tracking With Radar Application. Norwood, MA: Artech House, 1986.

[32]

Z. Ghahramani and M. I. Jordan, "Factorial hidden Markov models," in Machine Learning. Boston, MA: Kluwer, 1997.

Cited By

Granstrom KWillett PBar-Shalom Y(2016)Asymmetric Threat Modeling Using HMMsIEEE Transactions on Signal Processing10.1109/TSP.2016.252958464:10(2587-2601)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1109/TSP.2016.2529584
Hsieh JChuang CChen SChen CFan K(2010)Segmentation of human body parts using deformable triangulationIEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans10.1109/TSMCA.2010.204027240:3(596-610)Online publication date: 1-May-2010
https://dl.acm.org/doi/10.1109/TSMCA.2010.2040272
Huddleston SBrown D(2009)A statistical threat assessmentIEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans10.1109/TSMCA.2009.202761139:6(1307-1315)Online publication date: 1-Nov-2009
https://dl.acm.org/doi/10.1109/TSMCA.2009.2027611
Show More Cited By

Anomaly detection via feature-aided tracking and hidden Markov models

Recommendations

Combining hidden Markov models for improved anomaly detection
ICC'09: Proceedings of the 2009 IEEE international conference on Communications

In host-based intrusion detection systems (HIDS), anomaly detection involves monitoring for significant deviations from normal system behavior. Hidden Markov Models (HMMs) have been shown to provide a high level performance for detecting anomalies in ...
Network anomaly detection by continuous hidden markov models: An evolutionary programming approach

Information security is an important and growing need. The most common schemes used for detection systems include pattern-or signature-based and anomaly-based. Anomaly-based schemes use a set of metrics, which outline the normal system behavior and any ...
Hidden Markov models on a self-organizing map for anomaly detection in 802.11 wireless networks
Abstract
The present work introduces a hybrid integration of the self-organizing map and the hidden Markov model (HMM) for anomaly detection in 802.11 wireless networks. The self-organizing hidden Markov model map (SOHMMM) deals with the spatial ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans Volume 39, Issue 1

Special section: Best papers from the 2007 biometrics: Theory, applications, and systems (BTAS 07) conference

January 2009

279 pages

ISSN:1083-4427

Issue’s Table of Contents

Copyright © 2009.

Publisher

IEEE Press

Publication History

Published: 01 January 2009

Revised: 14 July 2007

Received: 01 September 2006

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Granstrom KWillett PBar-Shalom Y(2016)Asymmetric Threat Modeling Using HMMsIEEE Transactions on Signal Processing10.1109/TSP.2016.252958464:10(2587-2601)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1109/TSP.2016.2529584
Hsieh JChuang CChen SChen CFan K(2010)Segmentation of human body parts using deformable triangulationIEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans10.1109/TSMCA.2010.204027240:3(596-610)Online publication date: 1-May-2010
https://dl.acm.org/doi/10.1109/TSMCA.2010.2040272
Huddleston SBrown D(2009)A statistical threat assessmentIEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans10.1109/TSMCA.2009.202761139:6(1307-1315)Online publication date: 1-Nov-2009
https://dl.acm.org/doi/10.1109/TSMCA.2009.2027611
Granström KWillett PBar-Shalom Y(undefined)Detectability prediction of hidden Markov models with cluttered observation sequences2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2016.7472482(4269-4273)
https://dl.acm.org/doi/10.1109/ICASSP.2016.7472482

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents