Abstract
Since the inception of Andoroid OS, smartphones sales have been growing exponentially, and today it enjoys the monopoly in the smartphone marketplace. The widespread adoption of Android smartphones has drawn the attention of malware designers, which threatens the Android ecosystem. The current state-of-the-art Android malware detection systems are based on machine learning and deep learning models. Despite having superior performance, these models are susceptible to adversarial attack. Therefore in this paper, we developed eight Android malware detection models based on machine learning and deep neural network and investigated their robustness against the adversarial attacks. For the purpose, we created new variants of malware using Reinforcement Learning, which will be misclassified as benign by the existing Android malware detection models. We propose two novel attack strategies, namely single policy attack and multiple policy attack using reinforcement learning for white-box and grey-box scenario respectively. Putting ourselves in adversary’ shoes, we designed adversarial attacks on the detection models with the goal of maximising fooling rate, while making minimum modifications to the Android application and ensuring that the app’s functionality and behaviour does not change. We achieved an average fooling rate of 44.21% and 53.20% across all the eight detection models with maximum five modifications using a single policy attack and multiple policy attack, respectively. The highest fooling rate of 86.09% with five changes was attained against the decision tree based model using the multiple policy approach. Finally, we propose an adversarial defence strategy which reduces the average fooling rate by threefold to 15.22% against a single policy attack, thereby increasing the robustness of the detection models i.e. the proposed model can effectively detect variants (metamorphic) of malware. The experimental analysis shows that our proposed Android malware detection system using reinforcement learning is more robust against adversarial attacks.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Appice, A., Andresini, G., & Malerba, D. (2020). Clustering-aided multi-view classification: a case study on android malware detection. Journal of Intelligent Information Systems, 1–26.
Arp, D., Spreitzenbarth, M., Hubner, M., Gascon, H., Rieck, K., & Siemens, C. (2014). Drebin: effective and explainable detection of android malware in your pocket. In Ndss, (Vol. 14 pp. 23–26).
Arshad, S., Shah, M. A., Wahid, A., Mehmood, A., Song, H., & Yu, H. (2018). Samadroid: a novel 3-level hybrid malware detection model for android operating system. IEEE Access, 6, 4321–4339.
AVTEST. (2019). Malware. Available: https://www.av-test.org/en/statistics/malware/. Last accessed: April 2020.
Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G., & Roli, F. (2013). Evasion attacks against machine learning at test time. In Joint european conference on machine learning and knowledge discovery in databases. Springer (pp. 387–402).
Chinavle, D., Kolari, P., Oates, T., & Finin, T. (2009). Ensembles in adversarial classification for spam. In Proceedings of the 18th ACM conference on information and knowledge management (pp. 2015–2018).
Dash, S. K., Suarez-Tangil, G., Khan, S., Tam, K., Ahmadi, M., Kinder, J., & Cavallaro, L. (2016). Droidscribe: classifying android malware based on runtime behavior. In 2016 IEEE Security and privacy workshops (SPW). IEEE (pp. 252–261).
Faruki, P., Bharmal, A., Laxmi, V., Ganmoor, V., Gaur, M. S., Conti, M., & Rajarajan, M. (2014). Android security: a survey of issues, malware penetration, and defenses. IEEE Communications Surveys & Tutorials, 17(2), 998–1022.
Fonteneau, R., Murphy, S. A., Wehenkel, L., & Ernst, D. (2010). Towards min max generalization in reinforcement learning. In International conference on agents and artificial intelligence. Springer (pp. 61–77).
G DATA CyberDefense AG. (2019). Mobile malware report. Available: https://www.gdatasoftware.com/news/2019/07/35228-mobile-malware-report-no-let-up-with-android-malware. Last accessed: April 2020.
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv:1412.6572.
Hispasec Sistemas. (2019). VirusTotal. Available: https://www.virustotal.com/gui/home. Last accessed: April 2020.
Ji, Y., Bowman, B., & Huang, H. H. (2019). Securing malware cognitive systems against adversarial attacks. In 2019 IEEE international conference on cognitive computing (ICCC). IEEE (pp. 1–9).
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: a survey. Journal of Artificial Intelligence Research, 4, 237–285.
Kurakin, A., Goodfellow, I., & Bengio, S. (2016). Adversarial machine learning at scale. arXiv:1611.01236.
Lindorfer, M., Neugschwandtner, M., Weichselbaum, L., Fratantonio, Y., Van Der Veen, V., & Platzer, C. (2014). Andrubis–1,000,000 apps later: a view on current android malware behaviors. In 2014 third international workshop on building analysis datasets and gathering experience returns for security (BADGERS). IEEE (pp. 3–17).
LLC, G. (2010). Google play. Available: https://play.google.com/store?hl=en. Last accessed: April 2020.
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y. C., & Kim, D. I. (2019). Applications of deep reinforcement learning in communications and networking: a survey. IEEE Communications Surveys & Tutorials, 21(4), 3133–3174.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083.
O’Dea, S. (2020). Smartphones - statistics & facts. Available: https://www.statista.com/topics/840/smartphones/. Last accessed: April 2020.
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016). The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on security and privacy (EuroS&P). IEEE (pp. 372–387).
Paudice, A., Muñoz-González, L., & Lupu, E. C. (2018). Label sanitization against label flipping poisoning attacks. In Joint European conference on machine learning and knowledge discovery in databases. Springer (pp. 5–15).
Rathore, H., Agarwal, S., Sahay, S. K., & Sewak, M. (2018). Malware detection using machine learning and deep learning. In International conference on big data analytics. Springer (pp. 402–411).
Rathore, H., Sahay, S. K., Chaturvedi, P., & Sewak, M. (2018). Android malicious application classification using clustering. In International conference on intelligent systems design and applications. Springer (pp. 659–667).
Sahay, S. K., Sharma, A., & Rathore, H. (2020). Evolution of malware and its detection techniques. In Information and communication technology for sustainable development. Springer (pp. 139–150).
Serban, A. C., Poll, E., & Visser, J. (2018). Adversarial examples-a complete characterisation of the phenomenon. arXiv:1810.01185.
Sewak, M., Sahay, S. K., & Rathore, H. (2018). Comparison of deep learning and the classical machine learning algorithm for the malware detection. In 2018 19th IEEE/ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (SNPD). IEEE (pp. 293– 296).
Sharma, A., & Sahay, S. K. (2014). Evolution and detection of polymorphic and metamorphic malwares: a survey. International Journal of Computer Applications, 90(2).
Simon Kemp (Hootsuite). (2018). Global digital report. Available: https://digitalreport.wearesocial.com/. Last accessed: April 2020.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: an introduction. Cambridge: MIT Press.
Symantec. (2019). Internet security threat report. Available: https://www-west.symantec.com/content/dam/symantec/docs/reports/istr-24-2019-en.pdf. Last accessed: April 2020.
Tam, K., Feizollah, A., Anuar, N. B., Salleh, R., & Cavallaro, L. (2017). The evolution of android malware and android analysis techniques. ACM Computing Surveys (CSUR), 49(4), 1–41.
Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2017). Ensemble adversarial training: attacks and defenses. arXiv:1705.07204.
Wiśniewski, R., & Tumbleson, C. (2020). Apktool. Available: https://ibotpeaches.github.io/Apktool/.
Wu, D. J., Mao, C. H., Wei, T. E., Lee, H. M., & Wu, K. P. (2012). Droidmat: android malware detection through manifest and api calls tracing. In 2012 Seventh Asia joint conference on information security. IEEE (pp. 62–69).
Ye, Y., Li, T., Adjeroh, D., & Iyengar, S. S. (2017). A survey on malware detection using data mining techniques. ACM Computing Surveys (CSUR), 50(3), 41.
Yerima, S. Y., Sezer, S., McWilliams, G., & Muttik, I. (2013). A new android malware detection approach using bayesian classification. In 2013 IEEE 27th international conference on advanced information networking and applications (AINA). IEEE (pp. 121–128).
Yuan, Z., Lu, Y., Wang, Z., & Xue, Y. (2014). Droid-sec: deep learning in android malware detection. In Proceedings of the 2014 ACM conference on SIGCOMM (pp. 371–372).
Zhou, Y., & Jiang, X. (2012). Dissecting android malware: characterization and evolution. In 2012 IEEE symposium on security and privacy. IEEE (pp. 95–109).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Rathore, H., Sahay, S.K., Nikam, P. et al. Robust Android Malware Detection System Against Adversarial Attacks Using Q-Learning. Inf Syst Front 23, 867–882 (2021). https://doi.org/10.1007/s10796-020-10083-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10796-020-10083-8