Abstract
Sharing plenty and accurate structured Cyber Threat Intelligence (CTI) will play a pivotal role in adapting to rapidly evolving cyber attacks and malware. However, the traditional CTI generation methods are extremely time and labor-consuming. The recent work focuses on extracting CTI from well structured Open Source Intelligence (OSINT). However, many challenges are still to generate CTI and Indicators of Compromise(IoC) from non-human-written malware traces. This work introduces a method to automatically generate concise, accurate and understandable CTI from unstructured malware traces. For a specific class of malware, we first construct the IoC expressions set from malware traces. Furthermore, we combine the generated IoC expressions and other meaningful information in malware traces to organize the threat intelligence which meets open standards such as Structured Threat Information Expression (STIX). We evaluate our algorithm on real-world dataset. The experimental results show that our method achieves a high average recall rate of 89.4% on the dataset and successfully generates STIX reports for every class of malware, which means our methodology is practical enough to automatically generate effective IoC and CTI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
ANTIY. https://www.antiy.cn/
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Catakoglu, O., Balduzzi, M., Balzarotti, D.: Automatic extraction of indicators of compromise for web applications. In: Proceedings of the 25th International Conference on World Wide Web, WWW 2016, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, pp. 333–343 (2016). https://doi.org/10.1145/2872427.2883056
Corporation, T.M.: CybOX: cyber observable expression. https://cyboxproject.github.io
Corporation, T.M.: Malware attribute enumeration and characterization (MAEC). https://maecproject.github.io/documentation/overview/
David, B.: The pyramid of pain: Intel-driven detection & response to increase your adversary’s cost of operation. Technical Report, FireEye. https://rvasec.com/slides/2014/Bianco_Pyramid
Firdausi, I., Erwin, A., Nugroho, A.S., et al.: Analysis of machine learning techniques used in behavior-based malware detection. In: 2010 Second International Conference on Advances in Computing, Control, and Telecommunication Technologies, pp. 201–203. IEEE (2010)
Gao, Y., Li, X., Peng, H., Fang, B., Yu, P.: HinCTI: a cyber threat intelligence modeling and identification system based on heterogeneous information network. IEEE Trans. Knowl. Data Eng., 1 (2020)
Husari, G., Al-Shaer, E., Ahmed, M., Chu, B., Niu, X.: TTPDrill: automatic and accurate extraction of threat actions from unstructured text of CTI sources. In: Proceedings of the 33rd Annual Computer Security Applications Conference, pp. 103–115 (2017)
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Kurogome, Y., et al.: EIGER: automated IOC generation for accurate and interpretable endpoint malware detection. In: Proceedings of the 35th Annual Computer Security Applications Conference. ACSAC 2019, pp. 687–701. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3359789.3359808
Lakkaraju, H., Bach, S.H., Leskovec, J.: Interpretable decision sets: a joint framework for description and prediction. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1675–1684 (2016)
Li, J.: Cyberspace threat intelligence perception, sharing and analysis technology: A survey. Chin. J. Netw. Inf. Secur. 2(002), 16–29 (2016)
Li, V.G., Dunn, M., Pearce, P., McCoy, D., Voelker, G.M., Savage, S.: Reading the tea leaves: a comparative analysis of threat intelligence. In: 28th \(\{\)USENIX\(\}\) Security Symposium (\(\{\)USENIX\(\}\) Security 19), pp. 851–867 (2019)
Liao, X., Yuan, K., Wang, X., Li, Z., Xing, L., Beyah, R.: Acing the IOC game: toward automatic discovery and analysis of open-source cyber threat intelligence. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 755–766 (2016)
Liu, F., Wen, Y., Zhang, D., Jiang, X., Xing, X., Meng, D.: Log2vec: a heterogeneous graph embedding based approach for detecting cyber threats within enterprise. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 1777–1794 (2019)
Liu, L., Chen, S., Yan, G., Zhang, Z.: BotTracer: execution-based bot-like malware detection. In: Wu, T.-C., Lei, C.-L., Rijmen, V., Lee, D.-T. (eds.) ISC 2008. LNCS, vol. 5222, pp. 97–113. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85886-7_7
MANDIANT: IOC writer. https://github.com/mandiant/ioc_writer
MANDIANT: Sophisticated indicators for the modern threat intelligence: an introduction to openIOC. Technical Report, MANDIANT. https://www.academia.edu/31820654/An_Introduction_to_Open_IOC
McMillan, R., Pratap, K.: Market guide for security threat intelligence services. Technical Report, Gartner (2014). https://www.gartner.com/en/documents/2874317
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
OASIS: Lightweight visualization for STIX 2.0 objects and relationships. https://github.com/oasis-open/cti-stix-visualization/
OASIS: openIOC-to-SITX. https://github.com/STIXProject/openioc-to-stix
OASIS: STIX version 2.0. part 2: STIX objects. Technical Report. https://docs.oasis-open.org/cti/stix/v2.0/stix-v2.0-part2-stix-objects.pdf
Perdisci, R., Lee, W., Feamster, N.: Behavioral clustering of http-based malware and signature generation using malicious network traces. In: NSDI, vol. 10, p. 14 (2010)
SANS: The sans state of cyber threat intelligence survey: CTI important and maturing. Technical Report. https://www.sans.org/reading-room/whitepapers/analyst/state-cyber-threat-intelligence-survey-cti-important-maturing-37177
Stinson, E., Mitchell, J.C.: Characterizing bots’ remote control behavior. In: M. Hämmerli, B., Sommer, R. (eds.) DIMVA 2007. LNCS, vol. 4579, pp. 89–108. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73614-1_6
Xu, W., Wang, Y., Xue, Z.: Automatic generation of IOC for threat intelligence. Commun. Technol. 50(1), 116–123 (2017)
Yan, J., Yan, G., Jin, D.: Classifying malware represented as control flow graphs using deep graph convolutional neural network. In: 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 52–63. IEEE (2019)
Ye, Y., Li, T., Adjeroh, D., Iyengar, S.S.: A survey on malware detection using data mining techniques. ACM Comput. Surv. (CSUR) 50(3), 1–40 (2017)
Zhao, J., Yan, Q., Liu, X., Li, B., Zuo, G.: Cyber threat intelligence modeling based on heterogeneous graph convolutional network. In: 23rd International Symposium on Research in Attacks, Intrusions and Defenses (\(\{\)RAID\(\}\) 2020), pp. 241–256 (2020)
Acknowledgement
We thank the anonymous reviewers for the valuable comments and suggestions. This work is supported by National Key Research and Development Program of China [No. 2020YFB1807500] and National Natural Science Foundation of China [No. 61831007].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Wei, Y., Zou, F. (2021). Automatic Generation of Malware Threat Intelligence from Unstructured Malware Traces. In: Garcia-Alfaro, J., Li, S., Poovendran, R., Debar, H., Yung, M. (eds) Security and Privacy in Communication Networks. SecureComm 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 398. Springer, Cham. https://doi.org/10.1007/978-3-030-90019-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-90019-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90018-2
Online ISBN: 978-3-030-90019-9
eBook Packages: Computer ScienceComputer Science (R0)