[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content

Advertisement

Springer Nature Link
Log in

Stealthy Energy Consumption-oriented Attacks on Training Stage in Deep Learning

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Deep Learning as a Service (DLaaS) is rapidly developing recently to enable applications including self-driving, face recognition, and natural language processing for small enterprises. However, DLaaS can also introduce enormous computing power consumption at the service ends. Existing works focus on the optimization of the training process such as using low-cost chips or optimizing the training settings for better energy efficiency. In this paper, we revisit this issue from an adversary perspective which attempts to maliciously make victims waste more training efforts without being noticed. In particular, we propose a novel attack targeting enlarging the training costs stealthily via poisoning the training data. By adopting the Projected Gradient Descent (PGD) method to generate poisoned samples, we show that attackers can significantly increase the training costs by as much as 88% in both the white-box scenario and the black-box scenario with a very tiny influence on the model’s accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Algorithm 1
Figure 3
Figure 4
Figure 5
Figure 6

Similar content being viewed by others

Data Availability

Will be open-sourced.

Code Availability

Will be open-sourced.

References

  1. Wang, M., & Deng, W. (2020). Deep face recognition: a survey. Neurocomputing.

  2. Shi, C., Ding, J., Cao, X., Hu, L., Wu, B., & Li, X. (2021). Entity set expansion in knowledge graph: A heterogeneous information network perspective. Frontiers of Computer Science, 15(1), 1–12.

    Article  Google Scholar 

  3. Fu, Z., Gao, H., Guo, W., Jha, S. K., Jia, J., Liu, X., Long, B., Shi, J., Wang, S., & Zhou, M. (2020). Deep Learning for Search and Recommender Systems in Practice. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3515–3516.

  4. Qiu, H., Qiu, M., & Lu, R. (2019). Secure v2x communication network based on intelligent pki and edge computing. IEEE Network, 34(2), 172–178.

    Article  Google Scholar 

  5. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778.

  6. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. arXiv preprint arXiv:1706.03762.

  7. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

  8. Zeng, W., Ren, X., Su, T., Wang, H., Liao, Y., Wang, Z., Jiang, X., Yang, Z., Wang, K., Zhang, X., et al. (2021). Pangu-\(\alpha\): Large-scale autoregressive pretrained chinese language models with auto-parallel computation. arXiv preprint arXiv:2104.12369.

  9. Qiu, H., Zheng, Q., Memmi, G., Lu, J., Qiu, M., & Thuraisingham, B. (2020). Deep residual learning-based enhanced jpeg compression in the internet of things. IEEE Transactions on Industrial Informatics, 17(3), 2124–2133.

    Google Scholar 

  10. Chen, J., & Ran, X. (2019). Deep learning with edge computing: A review. Proceedings of the IEEE, 107(8), 1655–1674.

    Article  Google Scholar 

  11. Joshi, A. V. (2020). Amazon’s machine learning toolkit: Sagemaker. In: Machine Learning and Artificial Intelligence, pp. 233–243. Springer.

  12. Ciaburro, G., Ayyadevara, V. K., & Perrier, A. (2018). Hands-On Machine Learning on Google Cloud Platform: Implementing Smart and Efficient Analytics Using Cloud ML Engine. Packt Publishing Ltd.

  13. Barga, R., Fontama, V., & Tok, W. H. (2015). Introducing microsoft azure machine learning. In: Predictive Analytics with Microsoft Azure Machine Learning, pp. 21–43. Springer.

  14. Henderson, P., Hu, J., Romoff, J., Brunskill, E., Jurafsky, D., & Pineau, J. (2020). Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248), 1–43.

    MathSciNet  Google Scholar 

  15. So, D., Le, Q., & Liang, C. (2019). The evolved transformer. In: International Conference on Machine Learning, pp. 5877–5886. PMLR.

  16. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243.

  17. Wang, Y., Ding, C., Li, Z., Yuan, G., Liao, S., Ma, X., Yuan, B., Qian, X., Tang, J., Qiu, Q., et al. (2018). Towards ultra-high performance and energy efficiency of deep learning systems: an algorithm-hardware co-optimization framework. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32.

  18. Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017). In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, pp. 1–12.

  19. Qiu, H., Dong, T., Zhang, T., Lu, J., Memmi, G., & Qiu, M. (2020). Adversarial attacks against network intrusion detection in IoT systems. IEEE Internet of Things Journal.

  20. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013). Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199.

  21. Li, Y., Wu, B., Jiang, Y., Li, Z., & Xia, S.-T. (2020). Backdoor learning: A survey. arXiv preprint arXiv:2007.08745.

  22. Zhai, T., Li, Y., Zhang, Z., Wu, B., Jiang, Y., & Xia, S.-T. (2021). Backdoor attack against speaker verification. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2560–2564. IEEE.

  23. Qiu, H., Zeng, Y., Guo, S., Zhang, T., Qiu, M., & Thuraisingham, B. (2021). Deepsweep: An evaluation framework for mitigating dnn backdoor attacks using data augmentation. In: Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security, pp. 363–377.

  24. Carlini, N., & Wagner, D. (2017). Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (sp), pp. 39–57. IEEE.

  25. Athalye, A., Carlini, N., & Wagner, D. (2018). Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In: International Conference on Machine Learning, pp. 274–283. PMLR.

  26. Qiu, H., Zeng, Y., Zheng, Q., Guo, S., Zhang, T., & Li, H. (2021). An efficient preprocessing-based approach to mitigate advanced adversarial attacks. IEEE Transactions on Computers.

  27. Hong, S., Kaya, Y., Modoranu, I.-V., & Dumitraş, T. (2020). A Panda? No, It’s a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference. arXiv preprint arXiv:2010.02432.

  28. Çalik, R. C., & Demirci, M. F. (2018). Cifar-10 image classification with convolutional neural networks for embedded systems. In: 2018 IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA), pp. 1–2. IEEE.

  29. Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017). Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI Conference on Artificial Intelligence.

  30. Zhang, Q., Bai, C., Liu, Z., Yang, L. T., Yu, H., Zhao, J., & Yuan, H. (2020). A gpu-based residual network for medical image classification in smart medicine. Information Sciences, 536, 91–100.

    Article  MathSciNet  Google Scholar 

  31. Hassan, M. M., Gumaei, A., Alsanad, A., Alrubaian, M., & Fortino, G. (2020). A hybrid deep learning model for efficient intrusion detection in big data environment. Information Sciences, 513, 386–396.

    Article  Google Scholar 

  32. He, Y., Liu, P., Wang, Z., Hu, Z., & Yang, Y. (2019). Filter pruning via geometric median for deep convolutional neural networks acceleration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4340–4349.

  33. Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.

  34. Han, S., Mao, H., & Dally, W. J. (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149.

  35. Wang, C., Gong, L., Yu, Q., Li, X., Xie, Y., & Zhou, X. (2016). DLAU: A scalable deep learning accelerator unit on FPGA. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 36(3), 513–517.

    Google Scholar 

  36. Liao, H., Tu, J., Xia, J., & Zhou, X. (2019). Davinci: A scalable architecture for neural network computing. In: 2019 IEEE Hot Chips 31 Symposium (HCS), pp. 1–44. IEEE Computer Society.

  37. Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014). Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning. ACM SIGARCH Computer Architecture News, 42(1), 269–284.

    Article  Google Scholar 

  38. Acun, B., Murphy, M., Wang, X., Nie, J., Wu, C.-J., & Hazelwood, K. (2021). Understanding training efficiency of deep learning recommendation models at scale. In: 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pp. 802–814. IEEE.

  39. Zhang, L., & Suganthan, P. N. (2016). A survey of randomized algorithms for training neural networks. Information Sciences, 364, 146–155.

    Article  Google Scholar 

  40. Akita, R., Yoshihara, A., Matsubara, T., & Uehara, K. (2016). Deep learning for stock prediction using numerical and textual information. In: 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS), pp. 1–6. IEEE.

  41. Qiu, H., Noura, H., Qiu, M., Ming, Z., & Memmi, G. (2019). A user-centric data protection method for cloud storage based on invertible DWT. IEEE Transactions on Cloud Computing.

  42. Grosse, K., Trost, T. A., Mosbach, M., Backes, M., & Klakow, D. (2019). On the security relevance of weights in deep learning. arXiv preprint arXiv:1902.03020.

  43. Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018). Recent advances in convolutional neural networks. Pattern Recognition, 77, 354–377.

    Article  Google Scholar 

  44. Shumailov, I., Shumaylov, Z., Kazhdan, D., Zhao, Y., Papernot, N., Erdogdu, M. A., & Anderson, R. (2021). Manipulating SGD with data ordering attacks. arXiv preprint arXiv:2104.09667.

  45. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083.

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Wencheng Chen & Hongyu Li

Authors
  1. Wencheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wencheng Chen.

Ethics declarations

Ethics Approval

Not applicable.

Consent to Participate

Yes.

Consent for Publication

Yes.

Competing Interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, W., Li, H. Stealthy Energy Consumption-oriented Attacks on Training Stage in Deep Learning. J Sign Process Syst 95, 1425–1437 (2023). https://doi.org/10.1007/s11265-023-01895-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-023-01895-3

Keywords

Search

Navigation