[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

AutoRoC-DBSCAN: automatic tuning of DBSCAN to detect malicious DNS tunnels

  • Published:
Annals of Telecommunications Aims and scope Submit manuscript

Abstract

Modern attacks, such as advanced persistent threats, hide command-and-control channels inside authorized network traffic like DNS or DNS over HTTPS to infiltrate the local network and exfiltrate sensitive data. Detecting such malicious traffic using traditional techniques is cumbersome especially when the traffic encrypted like DNS over HTTPS. Unsupervised machine learning techniques, and more specifically density-based spatial clustering of applications with noise (DBSCAN), can achieve good results in detecting malicious DNS tunnels. However, DBSCAN requires manually tuning two hyperparameters, whose optimal values can differ depending on the dataset. In this article, we propose an improved algorithm called AutoRoC-DBSCAN that can automatically find the best hyperparameters. We evaluated and obtained good results on two different datasets: a dataset we created with malicious DNS tunnels and the CIRA-CIC-DoHBrw-2020 dataset with malicious DoH tunnels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31

Similar content being viewed by others

Notes

  1. https://github.com/quynnhnguyen/DNS-dataset/tree/main

References

  1. Nguyen TQ, Laborde R, Benzekri A, Qu’hen B (2020) Detecting abnormal DNS traffic using unsupervised machine learning. In: 2020 4th Cyber Security in Networking Conference (CSNet), pp 1–8. https://doi.org/10.1109/CSNet50428.2020.9265466

  2. Nguyen TQ, Laborde R, Benzekri A et al (2022) AutoRoC-DBSCAN: automatic tuning of DBSCAN to detect malicious DNS tunnels. In: Chen J, He D, Lu R (eds) Emerging information security and applications. Springer Nature, Switzerland, Cham, pp 126–144. https://doi.org/10.1007/978-3-031-23098-1_8

  3. Habibi Lashkari A, Seo A, Gil G, Ghorbani A (2017) CIC-AB: online ad blocker for browsers, pp 1–7. https://doi.org/10.1109/CCST.2017.8167846

  4. DBSCAN clustering algorithm in machine learning. In: KDnuggets. https://www.kdnuggets.com/dbscan-clustering-algorithm-in-machine-learning.html/. Accessed 1 Jul 2020

  5. Cunningham P, Delany S (2007) k-Nearest neighbour classifiers. Mult Classif Syst 54. https://doi.org/10.1145/3459665

  6. scipy.signal.find_peaks — SciPy v1.8.0 Manual. https://docs.scipy.org/doc/scipy-1.8.0/html-scipyorg/reference/generated/scipy.signal.find_peaks.html#scipy.signal.find_peaks. Accessed 8 Feb 2022

  7. DoHBrw 2020 | Datasets | Research | Canadian Institute for Cybersecurity | UNB. https://www.unb.ca/cic/datasets/dohbrw-2020.html. Accessed 16 Sep 2021

  8. Applications | Research | Canadian Institute for Cybersecurity | UNB. https://www.unb.ca/cic/research/applications.html. Accessed 24 Aug 2021

  9. Liu FT, Ting K, Zhou Z-H (2009) Isolation forest, pp 413–422 https://doi.org/10.1109/ICDM.2008.17

  10. Schölkopf B, Williamson R, Smola A et al (1999) Support vector method for novelty detection, pp 582–588

  11. Breunig MM, Kriegel H-P, Ng RT, Sander J. LOF Identifying density-based local outliers, p 12

  12. Starczewski A, Goetzen P, Er MJ (2020) A new method for automatic determining of the DBSCAN parameters. J Artif Intell Soft Comput Res 10:209–221. https://doi.org/10.2478/jaiscr-2020-0014

    Article  Google Scholar 

  13. Falahiazar Z, Bagheri A, Reshadi M (2021) Determining the parameters of DBSCAN automatically using the multi-objective genetic algorithm. J Inf Sci Eng

  14. Karami A, Johansson R (2014) Choosing DBSCAN parameters automatically using differential evolution. Int J Comput Appl 91:1–11. https://doi.org/10.5120/15890-5059

    Article  Google Scholar 

  15. Miglani J, Thorpe C (2021) Employing machine learning paradigms for detecting DNS tunnelling

  16. Palau F, Catania C, Guerra J, et al (2020) DNS tunneling: a deep learning based lexicographical detection approach. ArXiv200606122 Cs

  17. MontazeriShatoori M, Davidson L, Kaur G, Lashkari AH (2020) Detection of DoH tunnels using time-series classification of encrypted traffic. In: 2020 IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), pp 63–70. https://doi.org/10.1109/DASC-PICom-CBDCom-CyberSciTech49142.2020.00026

  18. Banadaki Y, Robert S (2020) Detecting malicious DNS over HTTPS traffic in domain name system using machine learning classifiers. J Comput Sci Appl 8:46–55. https://doi.org/10.12691/jcsa-8-2-2

    Article  Google Scholar 

  19. Singh SK, Roy PK (2020) Detecting malicious DNS over HTTPS traffic using machine learning. In: 2020 international conference on innovation and intelligence for informatics, computing and technologies (3ICT), pp 1–6. https://doi.org/10.1109/3ICT51146.2020.9312004

  20. Lin H, Liu G, Yan Z (2019) Detection of application-layer tunnels with rules and machine learning. In: Wang G, Feng J, Bhuiyan MZA, Lu R (eds) Security, privacy, and anonymity in computation, communication, and storage. Springer International Publishing, Cham, pp 441–455

    Chapter  Google Scholar 

  21. Berg A, Forsberg D (2019) Identifying DNS-tunneled traffic with predictive models. ArXiv190611246 Cs

  22. Almusawi A, Amintoosi H (2018) DNS tunneling detection method based on multilabel support vector machine. Secur Commun Netw 2018:1–9. https://doi.org/10.1155/2018/6137098

    Article  Google Scholar 

  23. Homem I, Papapetrou P (2017) Harnessing predictive models for assisting network forensic investigations of DNS tunnels. 12

  24. Do VT, Engelstad P, Feng B, van Do T (2017) Detection of DNS tunneling in mobile networks using machine learning. In: Kim K, Joukov N (eds) Information science and applications 2017. Springer Singapore, Singapore, pp 221–230. https://doi.org/10.1007/978-981-10-4154-9_26

  25. Buczak AL, Hanke PA, Cancro GJ et al (2016) Detection of tunnels in PCAP data by random forests. In: Proceedings of the 11th annual cyber and information security research conference. ACM, Oak Ridge, pp 1–4. https://doi.org/10.1145/2897795.2897804

  26. Aiello M, Mongelli M, Papaleo G (2015) DNS tunneling detection through statistical fingerprints of protocol messages and machine learning: DNS tunneling detection. Int J Commun Syst 28:1987–2002. https://doi.org/10.1002/dac.2836

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IRIT - University Toulouse 3 Paul Sabatier, 31000, Toulouse, France

    Thi Quynh Nguyen, Romain Laborde, Abdelmalek Benzekri & Arnaud Oglaza

  2. Akkodis, 31700, Blagnac, France

    Thi Quynh Nguyen & Mehdi Mounsif

Authors
  1. Thi Quynh Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Romain Laborde
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdelmalek Benzekri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arnaud Oglaza
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mehdi Mounsif
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thi Quynh Nguyen.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, T.Q., Laborde, R., Benzekri, A. et al. AutoRoC-DBSCAN: automatic tuning of DBSCAN to detect malicious DNS tunnels. Ann. Telecommun. (2024). https://doi.org/10.1007/s12243-024-01025-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12243-024-01025-5

Keywords

Search

Navigation