Abstract
With the exponential increase in devices connected to the Internet, the risk of security breaches has in turn led to an increase in traction for machine learning based intrusion detection systems. These systems involve either supervised classifiers to detect known threats or unsupervised techniques to separate anomalies from normal data. Supervised learning enables accurate detection of known attack behaviours but requiring quality ground-truth data, it is ineffective against new emerging threats. Unsupervised learning-based systems address this issue due to their generalizable approach; however, they can result in a high false detection rate and are generally unable to detect specific types of each threat. We propose an ensemble technique that addresses the shortcomings of both approaches through a semi-supervised approach which detects both known and unknown threats in the network by analysing traffic metadata. The robust approach integrates A) an adversarial regularisation based autoencoder for unsupervised representation learning and B) supervised gradient boosted trees to detect the type of detected threats. The adversarial regularisation enables a reduced false positive rate and the combination of the autoencoder with the supervised stage enables resiliency against class imbalance and caters to the ever-evolving threat landscape by detecting previously unseen threats and anomalies. SANTA’s ability to detect never-before-seen threats also indicates its potential to address the concept drift, a phenomenon where the known threat changes its behaviour/attack sequence over time. The system is evaluated on the CSE-CIC-IDS2018 dataset, and the results confirm the resilience and adaptability of the SANTA system against known shortcomings of both supervised and unsupervised approaches.
M. F. Zia and S. H. Kalidass—Equal Contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Statistica Research Department: Internet of things - Number of Connected Devices Worldwide 2015–2025 (2022). https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide
Charles, G.: AAG IT: The Latest Cybercrime Statistics (2023). https://aag-it.com/the-latest-cyber-crime-statistics
Li, B., Springer, J., Bebis, G., Gunes, M.H.: A survey of network flow applications. J. Netw. Comput. Appl. 36(2), 567–581 (2013)
Lunardi, W.T., Lopez, M.A., Giacalone, J.-P.: Arcade: adversarially regularized convolutional autoencoder for network anomaly detection. IEEE Trans. Netw. Serv. Manag. 20(2), 1305–1318 (2023). https://doi.org/10.1109/TNSM.2022.3229706
Thaseen, S., Kumar, C.A.: An analysis of supervised tree based classifiers for intrusion detection system. In: Proceedings of the 2013 International Conference on Pattern Recognition, Informatics and Mobile Engineering, Salem, India, 21–22 February 2013, pp. 294–299 (2013)
Syarif, I., Prugel-Bennett, A., Wills, G.: Unsupervised clustering approach for network anomaly detection. In: Benlamri, R. (ed.) NDT 2012. CCIS, vol. 293, pp. 135–145. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30507-8_13
Ran, J., Ji, Y., Tang, B.: A semi-supervised learning approach to IEEE 802.11 network anomaly detection. In: 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring), Kuala Lumpur, Malaysia, pp. 1–5 (2019). https://doi.org/10.1109/VTCSpring.2019.8746576
Zhao, Y., Hryniewicki, M.K.: XGBOD: improving supervised outlier detection with unsupervised representation learning. In: IJCNN. IEEE (2018). https://arxiv.org/abs/1912.00290
Jerome, H.: Friedman: greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)
A Realistic Cyber Defense Dataset (CSE-CIC-IDS2018). https://registry.opendata.aws/cse-cic-ids2018. Accessed 01 June 2023
Gharib, A., Sharafaldin, I., Habibi Lashkari, A., Ghorbani, A.A.: An evaluation framework for intrusion detection dataset. In: International Conference on Information Science and Security (ICISS), pp. 1–6 (2016)
Sharafaldin, I., Lashkari, A.H., Ghorbani, A.A.: Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1, 108–116 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zia, M.F., Kalidass, S.H., Roscoe, J.F. (2023). SANTA: Semi-supervised Adversarial Network Threat and Anomaly Detection System. In: Bramer, M., Stahl, F. (eds) Artificial Intelligence XL. SGAI 2023. Lecture Notes in Computer Science(), vol 14381. Springer, Cham. https://doi.org/10.1007/978-3-031-47994-6_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-47994-6_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47993-9
Online ISBN: 978-3-031-47994-6
eBook Packages: Computer ScienceComputer Science (R0)