n‐Grams exclusion and inclusion filter for intrusion detection in Internet of Energy big data systems
Abstract
The advent of Internet of Energy (IoE) and the seamless integration of grid operators, power generators, distributors, sensors, and end users promise more efficient use of energy. However, the IoE will inherit the vulnerabilities from all of the integrated systems, and this raises concerns for trust and privacy. The evolving complexity and increased speed of network‐based attacks emphasizes the need for an efficient intrusion detection system. Consequently, with the emergence of new attacks and the increasing number of signatures, traditional signature‐based intrusion detection systems cannot both sift through big data and meet high network speeds. Detection performance severely deteriorates when matching hundreds of gigabits per second to the growing number of attack signatures. Given that pattern matching takes up to 60% of the overall intrusion detection time, this paper presents a new and fast software‐based pattern matching system, Exscind. It proposes an exclusion‐inclusion filter to preclude clean traffic before having to do expensive pattern matching. Additionally, if the traffic is malicious, the system only matches against a subset of signatures that have a high probability of being a match. We extensively evaluate the system's performance and conclude that using 6‐grams signature prefix provides the best speedup and memory consumption with negligible false positives and linear scaling. We report a best‐case speedup of 6.5 times for normal traffic and 1.53 times for the worst possible scenario. For best‐case normal traffic, Exscind skips pattern matching for 98.36% of the packets.
Graphical Abstract
6‐grams attack signature prefix exclusion and inclusion Bloom filter. Skipping pattern matching for 98.4% of normal traffic. Up to 6 times faster intrusion detection with linear scaling.
References
[1]
Proudfoot R, Kent K, Aubanel E, Chen N. High performance software‐hardware network intrusion detection system. In: Proceedings of the 2007 International Conference on Field‐Programmable Technology (ICFPT 2007); 2007; Kitakyushu, Japan.
[2]
Aldwairi M. Hardware‐Efficient Pattern Matching Algorithms and Architectures for Fast Intrusion Detection [PhD thesis]. Raleigh, NC: North Carolina State University; 2006.
[3]
Aldwairi M, Khamayseh Y, Al‐Masri M. Application of artificial bee colony for intrusion detection systems. Secur Commun Netw 2015;8(16):2730‐2740. https://doi.org/10.1002/sec.588
[4]
Scarfone K, Mell P. Guide to intrusion detection and prevention systems (IDPS). NIST Special Publ. 2007;800(2007):94.
[5]
Otoum S, Kantarci B, Mouftah HT. Mitigating false negative intruder decisions in WSN‐based smart grid monitoring. In: Proceedings of the 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC); 2017; Valencia, Spain.
[6]
Aloqaily M, Otoum S, Al Ridhawi I, Jararweh Y. An intrusion detection system for connected vehicles in smart cities. Ad Hoc Netw. 2019;90:101842. https://doi.org/10.1016/j.adhoc.2019.02.001
[7]
Baker T, Al‐Dawsari B, Tawfik H, Reid D, Ngoko Y. GreeDi: an energy efficient routing algorithm for big data on cloud. Ad Hoc Netw. 2015;35:83‐96. https://doi.org/10.1016/j.adhoc.2015.06.008
[8]
Wang K, Li H, Feng Y, Tian G. Big data analytics for system stability evaluation strategy in the energy Internet. IEEE Trans Ind Inform. 2017;13(4):1969‐1978. https://doi.org/10.1109/TII.2017.2692775
[9]
Aldawsari B, Baker T, England D. Trusted energy‐efficient cloud‐based services brokerage platform. Int J Intell Comput Res. 2015;6:630‐639. https://doi.org/10.20533/ijicr.2042.4655.2015.0078
[10]
Baker T, Aldawsari B, Asim M, Tawfik H, Maamar Z, Buyya R. Cloud‐SEnergy: a bin‐packing based multi‐cloud service broker for energy efficient composition and execution of data‐intensive applications. Sustain Comput Inform Syst. 2018;19:242‐252. https://doi.org/10.1016/j.suscom.2018.05.011
[11]
Wang K, Zhang Y, Guo S, Dong M, Hu RQ, He L. IEEE access special section editorial: the Internet of energy: architectures, cyber security, and applications ‐ part II. IEEE Access. 2018;6:79276‐79279. https://doi.org/10.1109/ACCESS.2018.2885242
[12]
Sani AS, Yuan D, Jin J, Gao L, Yu S, Dong ZY. Cyber security framework for Internet of Things‐based energy Internet. Future Gener Comput Syst. 2019;93:849‐859. https://doi.org/10.1016/j.future.2018.01.029
[13]
Jaradat M, Jarrah M, Bousselham A, Jararweh Y, Al‐Ayyoub M. The Internet of energy: Smart sensor networks and big data management for smart grid. Procedia Comput Sci. 2015;56:592‐597. https://doi.org/10.1016/j.procs.2015.07.250
[14]
Aldwairi M, Alansari D. Exscind: fast pattern matching for intrusion detection using exclusion and inclusion filters. In: Proceedings of the 2011 7th International Conference on Next Generation Web Services Practices; 2011; Salamanca, Spain.
[15]
Anagnostakis KG, Antonatos S, Markatos EP, Polychronakis M. E2xB: A domain‐specific string matching algorithm for intrusion detection. In: Proceedings of the IFIP International Information Security Conference; 2003; Athens, Greece.
[16]
Aldwairi M, Flaifel Y, Mhaidat K. Efficient Wu‐Manber pattern matching hardware for intrusion and malware detection. In: Proceedings of the 2018 International Conference on Electrical, Electronics, Computers, Communication, Mechanical and Computing (EECCMC); 2018; Vellore, India.
[17]
Wang L. Big data in intrusion detection systems and intrusion prevention systems. J Comput Netw. 2017;4(1):48‐55. https://doi.org/10.12691/jcn-4-1-5
[18]
Aldwairi M, Abu‐Dalo AM, Jarrah M. Pattern matching of signature‐based IDS using Myers algorithm under MapReduce framework. EURASIP J Inf Secur. 2017;2017(1):9. https://doi.org/10.1186/s13635-017-0062-7
[19]
Roesch M. Snort ‐ lightweight intrusion detection for networks. In: Proceedings of the 13th Conference on Systems Administration (LISA'99); 1999; Seattle, WA.
[20]
Amin A, Shah B, Anwar S, Al‐Obeidat F, Khattak AM, Adnan A. A prudent based approach for compromised user credentials detection. Cluster Computing. 2017;21:423‐441. https://doi.org/10.1007/s10586-017-0878-4
[21]
Rehman RU. Intrusion Detection With SNORT (Bruce Perens' Open Source Series): Advanced IDS Techniques Using Snort, Apache, MySQL, PHP, and ACID. London, UK: Pearson Education; 2003.
[22]
Boyer RS, Moore JS. A fast string searching algorithm. Commun ACM. 1977;20(10):762‐772. https://doi.org/10.1145/359842.359859
[23]
Wu S, Manber U. A fast algorithm for multi pattern searching. Commun ACM. 1994.
[24]
Aho AV, Corasick MJ. Efficient string matching: an aid to bibliographic search. Commun ACM. 1975;18(6):333‐340.
[25]
snort.org . Snort IDS Rules. 2018. https://www.snort.org/downloads
[26]
Bloom BH. Space/time trade‐offs in hash coding with allowable errors. Commun ACM. 1970;13(7):422‐426.
[27]
Mohammad M, Lin E‐B. Gibbs phenomenon in tight framelet expansions. Commun Nonlinear Sci Numer Simul. 2018;55:84‐92. https://doi.org/10.1016/j.cnsns.2017.06.029
[28]
Dharmapurikar S, Krishnamurthy P, Sproull T, Lockwood J. Deep packet inspection using parallel bloom filters. In: Proceedings of the 11th Annual IEEE Symposium on High Performance Interconnects; 2003; Stanford, CA.
[29]
Mitzenmacher M. Compressed bloom filters. IEEE/ACM Trans Netw. 2002;10(5):604‐612.
[30]
Artan NS, Sinkar K, Patel J, Chao HJ. Aggregated bloom filters for intrusion detection and prevention hardware. In: Proceedings of the IEEE GLOBECOM 2007‐IEEE Global Telecommunications Conference; 2007; Washington, DC.
[31]
Laufer RP, Velloso PB, Duarte O. Generalized Bloom Filters. Electrical Engineering Program, COPPE/UFRJ, Technical Report GTA‐05‐43. 2005.
[32]
Attig M, Dharmapurikar S, Lockwood J. Implementation results of bloom filters for string matching. In: Proceedings of the 12th Annual IEEE Symposium on Field‐Programmable Custom Computing Machines; 2004; Napa, CA.
[33]
Song H, Lockwood JW. Multi‐pattern signature matching for hardware network intrusion detection systems. In: Proceedings of the GLOBECOM '05 IEEE Global Telecommunications Conference; 2005; St. Louis, MO.
[34]
Chaudhary D. Parallel processing of Bloom filter. Int J Electron Eng Res. 2010;2(1):35‐40.
[35]
Antonatos S, Polychronakis M, Akritidis P, Anagnostakis KG, Markatos EP. Piranha: fast and memory‐efficient pattern matching for intrusion detection. In: Proceedings of the IFIP International Information Security Conference; 2005; Chiba, Japan.
[36]
Chen Z, Zhang Y, Chen Z, Delis A. A digest and pattern matching‐based intrusion detection engine. Comput J. 2009;52(6):699‐723.
[37]
Kandhan R, Teletia N, Patel JM. Sigmatch: fast and scalable multi‐pattern matching. Proc VLDB Endow. 2010;3(1‐2):1173‐1184.
[38]
Balamurugan V, Saravanan R. Enhanced intrusion detection and prevention system on cloud environment using hybrid classification and OTS generation. Cluster Computing. 2017:1‐13.
[39]
Lee C‐L, Lin Y‐S, Chen Y‐C. A hybrid CPU/GPU pattern‐matching algorithm for deep packet inspection. PLOS ONE. 2015;10(10):e0139301.
[40]
Lin Y‐S, Lee C‐L, Chen Y‐C. Length‐bounded hybrid CPU/GPU pattern matching algorithm for deep packet inspection. Algorithms. 2017;10(1):16.
[41]
Baker T, Ngoko Y, Tolosana‐Calasanz R, Rana OF, Randles M. Energy efficient cloud computing environment via autonomic meta‐director framework. In: Proceedings of the 2013 Sixth International Conference on Developments in eSystems Engineering; 2013; Abu Dhabi, UAE.
[42]
Zuech R, Khoshgoftaar TM, Wald R. Intrusion detection and big heterogeneous data: a survey. J Big Data. 2015;2(1):3. https://doi.org/10.1186/s40537-015-0013-4
[43]
Peng K, Leung VCM, Zheng L, Wang S, Huang C, Lin T. Intrusion detection system based on decision tree over big data in fog environment. Wireless Communications and Mobile Computing. 2018;2018. Article ID 4680867. https://doi.org/10.1155/2018/4680867
[44]
Li G, Yan Z, Fu Y, Chen H. Data fusion for network intrusion detection: a review. Secur Commun Netw. 2018;2018. Article ID 8210614. https://doi.org/10.1155/2018/8210614
[45]
Suthaharan S. Big data classification: problems and challenges in network intrusion prediction with machine learning. ACM SIGMETRICS Perform Eval Rev 2014;41(4):70‐73. https://doi.org/https://doi.acm.org/10.1145/2627534.2627557
[46]
Hannan MA, Faisal M, Ker PJ, et al. A review of Internet of energy based building energy management systems: issues and recommendations. IEEE Access. 2018;6:38997‐39014. https://doi.org/10.1109/ACCESS.2018.2852811
[47]
Wang H, Ruan J, Ma Z, Zhou B, Fu X, Cao G. Deep learning aided interval state prediction for improving cyber security in energy Internet. Energy. 2019;174:1292‐1304. https://doi.org/10.1016/j.energy.2019.03.009
[48]
Aldwairi M, Al‐Khamaiseh A‐K, Alharbi F, Shah B. Bloom filters optimized Wu‐Manber for intrusion detection. J Digit Forensics Secur Law. 2016;11(4):5.
[49]
CTF . DEFCON capture the flag competition packet captures. 2018. https://www.defcon.org/html/links/dc-torrent.html
[50]
Lemire D, Kaser O. Recursive n‐gram hashing is pairwise independent, at best. Comput Speech Lang. 2010;24(4):698‐710.
[51]
Cohen JD. Recursive hashing functions for N‐grams. ACM Trans Inf Syst. 1997;15(3):291‐320. https://doi.org/10.1145/256163.256168
[52]
Gonnet GH, Baeza‐Yates RA. An analysis of the Karp‐Rabin string matching algorithm. Inf Process Lett. 1990;34(5):271‐274.
[53]
Lemire D. rollinghashjava. GitHub. 2018. https://github.com/lemire/rollinghashjava/blob/master/README.md
[54]
Bellare M, Micciancio D. A new paradigm for collision‐free hashing: incrementality at reduced cost. In: Proceedings of the 16th Annual International Conference on Theory and Application of Cryptographic Techniques, EUROCRYPT'97; 1997; Konstanz, Germany.
[55]
Aldwairi M, Ekailan N. Hybrid pattern matching algorithm for intrusion detection systems. J Inf Assur Secur. 2011;6(6):512‐521.
[56]
Burkholder RP. C++ implementation of Wu Manber's multi‐pattern search algorithm. 2018. https://blog.raymond.burkholder.net/index.php?/archives/362-C++-Implementation-of-Wu-Manbers-Multi-Pattern-Search-Algorithm.html
[57]
CommFront . ASCII table. 2018. https://www.commfront.com/pages/ascii-chart
[58]
cliran. ExecutionStopwatch. CodeProject. 2018. https://www.codeproject.com/Articles/31152/ExecutionStopwatch
[59]
Microsoft . MSDN Microsoft GC. 2018. https://msdn.microsoft.com/en-us/library/system.gc.gettotalmemory(v=vs.110).aspx
[60]
DEFCON . DEFCON hacker convention. 2018. https://www.defcon.org/
[61]
Netresec . Publicly available PCAP files. 2018. https://www.netresec.com/?page=PcapFiles
[62]
Wireshark . Wireshark. 2018. https://www.wireshark.org/
Index Terms
- n‐Grams exclusion and inclusion filter for intrusion detection in Internet of Energy big data systems
Index terms have been assigned to the content through auto-classification.
Recommendations
A Survey on Intrusion Detection and Prevention Systems
AbstractIn the digital world, malicious activities that violate the confidentiality, integrity, or availability of data and devices are known as intrusions. An intrusion detection system (IDS) analyses the activities of a single system or a network to ...
Network intrusion detection
Intrusion detection is a new, retrofit approach for providing a sense of security in existing computers and data networks, while allowing them to operate in their current "open" mode. The goal of intrusion detection is to identify unauthorized use, ...
Hybrid Intrusion Detection with Weighted Signature Generation over Anomalous Internet Episodes
This paper reports the design principles and evaluation results of a new experimental hybrid intrusion detection system (HIDS). This hybrid system combines the advantages of low false-positive rate of signature-based intrusion detection system (IDS) and ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© 2019 John Wiley & Sons, Ltd.
Publisher
John Wiley & Sons, Inc.
United States
Publication History
Published: 21 March 2022
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025