More Web Proxy on the site http://driver.im/

survey

Adversarial Examples on Object Recognition: A Comprehensive Survey

Authors:

Joost VisserAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 53, Issue 3

Article No.: 66, Pages 1 - 38

https://doi.org/10.1145/3398394

Published: 12 June 2020 Publication History

Abstract

Deep neural networks are at the forefront of machine learning research. However, despite achieving impressive performance on complex tasks, they can be very sensitive: Small perturbations of inputs can be sufficient to induce incorrect behavior. Such perturbations, called adversarial examples, are intentionally designed to test the network’s sensitivity to distribution drifts. Given their surprisingly small size, a wide body of literature conjectures on their existence and how this phenomenon can be mitigated. In this article, we discuss the impact of adversarial examples on security, safety, and robustness of neural networks. We start by introducing the hypotheses behind their existence, the methods used to construct or protect against them, and the capacity to transfer adversarial examples between different machine learning models. Altogether, the goal is to provide a comprehensive and self-contained survey of this growing field of research.

References

[1]

Mahdieh Abbasi and Christian Gagné. 2017. Robustness to adversarial examples through an ensemble of specialists. arXiv:1702.06856 (2017).

[2]

Naveed Akhtar and Ajmal Mian. 2018. Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access 6 (2018), 14410--14430.

[3]

Moustafa Alzantot, Yash Sharma, Supriyo Chakraborty, and Mani Srivastava. 2018. GenAttack: Practical black-box attacks with gradient-free optimization. arXiv:1805.11090 (2018),

[4]

Anish Athalye, Nicholas Carlini, and David Wagner. 2018. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Proceedings of the ICML.

[5]

Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2018. Synthesizing robust adversarial examples. In Proceedings of the ICML.

[6]

Shumeet Baluja and Ian Fischer. 2018. Adversarial transformation networks: Learning to generate adversarial examples. In Proceedings of the AAAI.

[7]

Marco Barreno, Blaine Nelson, Anthony D. Joseph, and J. D. Tygar. 2010. The security of machine learning. Mach. Learn. 81 (2010), 121--148.

Digital Library

[8]

Marco Barreno, Blaine Nelson, Russell Sears, Anthony D. Joseph, and J. Doug Tygar. 2006. Can machine learning be secure? In Proceedings of the ASIACCS. ACM.

[9]

Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos, Dimitrios Vytiniotis, Aditya Nori, and Antonio Criminisi. 2016. Measuring neural net robustness with constraints. In Proceedings of the NeurIPS.

[10]

Vahid Behzadan and Arslan Munir. 2017. Vulnerability of deep reinforcement learning to policy induction attacks. In Proceedings of the MLDM. Springer.

[11]

Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. 2009. Robust Optimization. Princeton University Press.

[12]

Arjun Nitin Bhagoji, Daniel Cullina, Chawin Sitawarin, and Prateek Mittal. 2018. Enhancing robustness of machine learning systems via data transformations. In Proceedings of the CISS. IEEE.

[13]

Arjun Nitin Bhagoji, Warren He, Bo Li, and Dawn Song. 2017. Exploring the space of black-box attacks on deep neural networks. arXiv:1712.09491 (2017).

[14]

Battista Biggio, Giorgio Fumera, and Fabio Roli. 2014. Security evaluation of pattern classifiers under attack. IEEE Trans. Knowl. Data Eng. 26 (2014), 984--996.

Digital Library

[15]

Battista Biggio, Blaine Nelson, and Pavel Laskov. 2012. Poisoning attacks against support vector machines. arXiv:1206.6389 (2012).

[16]

Battista Biggio and Fabio Roli. 2018. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recog. 84 (2018), 317--331.

Digital Library

[17]

Stephen Boyd and Lieven Vandenberghe. 2004. Convex Optimization. Cambridge University Press.

Digital Library

[18]

Michael Brückner, Christian Kanzow, and Tobias Scheffer. 2012. Static prediction games for adversarial learning problems. J. Mach. Learn. Res. 13 (2012), 2617--2654.

Digital Library

[19]

Sébastien Bubeck, Eric Price, and Ilya Razenshteyn. 2018. Adversarial examples from computational constraints. arXiv:1805.10204 (2018).

[20]

Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. 2018. Thermometer encoding: One hot way to resist adversarial examples. In Proceedings of the ICLR.

[21]

Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian J. Goodfellow, Aleksander Madry, and Alexey Kurakin. 2019. On evaluating adversarial robustness. arXiv:1902.06705 (2019).

[22]

Nicholas Carlini, Guy Katz, Clark Berret, and David Dill. 2018. Provably minimally-distorted adversarial examples. arXiv:1711.00851 (2018).

[23]

Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David Wagner, and Wenchao Zhou. 2016. Hidden voice commands. In Proceedings of the USENIX Security.

[24]

Nicholas Carlini and David Wagner. 2017. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the AISec. ACM.

Digital Library

[25]

Nicholas Carlini and David Wagner. 2017. Magnet and “Efficient defenses against adversarial attacks” are not robust to adversarial examples. arXiv:1711.08478 (2017).

[26]

Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In Proceedings of the S8P. IEEE.

[27]

Nicholas Carlini and David Wagner. 2018. Audio adversarial examples: Targeted attacks on speech-to-text. arXiv:1801.01944 (2018).

[28]

Jiefeng Chen, Xi Wu, Yingyu Liang, and Somesh Jha. 2018. Improving adversarial robustness by data-specific discretization. arXiv:1805.07816 (2018).

[29]

Pin-Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsieh. 2018. EAD: Elastic-net attacks to deep neural networks via adversarial examples. In Proceedings of the AAAI.

[30]

Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. 2017. ZOO: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the AISec. ACM.

Digital Library

[31]

Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. 2017. Parseval networks: Improving robustness to adversarial examples. In Proceedings of the ICML.

[32]

Taco Cohen and Max Welling. 2016. Group equivariant convolutional networks. In Proceedings of the ICML.

[33]

Daniel Cullina, Arjun Nitin Bhagoji, and Prateek Mittal. 2018. PAC-learning in the presence of evasion adversaries. arXiv:1806.01471 (2018).

[34]

Nilesh Dalvi, Pedro Domingos, Sumit Sanghai, Deepak Verma et al. 2004. Adversarial classification. In Proceedings of the SIGKDD. ACM.

Digital Library

[35]

Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Li Chen, Michael E. Kounavis, and Duen Horng Chau. 2017. Keeping the bad guys out: Protecting and vaccinating deep learning with JPEG compression. arXiv:1705.02900 (2017).

[36]

Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita-Rotaru, and Fabio Roli. 2019. Why do adversarial attacks transfer? Explaining transferability of evasion and poisoning attacks. In Proceedings of the USENIX Security.

[37]

Ambra Demontis, Paolo Russu, Battista Biggio, Giorgio Fumera, and Fabio Roli. 2016. On security and sparsity of linear classifiers for adversarial settings. In Proceedings of the International Workshops on SPR and SSPR. Springer.

Digital Library

[38]

Guneet S. Dhillon, Kamyar Azizzadenesheli, Zachary C. Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. 2018. Stochastic activation pruning for robust adversarial defense. arXiv:1803.01442 (2018).

[39]

Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. 2018. Boosting adversarial attacks with momentum. In Proceedings of the CVPR. IEEE.

[40]

Beranger Dumont, Simona Maggio, and Pablo Montalvo. 2018. Robustness of rotation-equivariant networks to adversarial perturbations. arXiv:1802.06627 (2018).

[41]

Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy Mann, and Pushmeet Kohli. 2018. A dual approach to scalable verification of deep networks. In Proceedings of the UAI.

[42]

Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M. Roy. 2016. A study of the effect of jpg compression on adversarial images. arXiv:1608.00853 (2016).

[43]

Ruediger Ehlers. 2017. Formal verification of piece-wise linear feed-forward neural networks. In Proceedings of the ATVA.

[44]

Gamaleldin Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alexey Kurakin, Ian Goodfellow, and Jascha Sohl-Dickstein. 2018. Adversarial examples that fool both computer vision and time-limited humans. In Proceedings of the NeurIPS.

[45]

Logan Engstrom, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. 2017. A rotation and a translation suffice: Fooling CNNs with simple transformations. arXiv:1712.02779 (2017).

[46]

Ivan Evtimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Song. 2017. Robust physical-world attacks on deep learning models. arXiv:1707.08945 (2017).

[47]

Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. 2018. Robust physical-world attacks on deep learning visual classification. In Proceedings of the CVPR. IEEE.

[48]

Alhussein Fawzi, Hamza Fawzi, and Omar Fawzi. 2018. Adversarial vulnerability for any classifier. arXiv:1802.08686 (2018).

[49]

Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2015. Fundamental limits on adversarial robustness. In Proceedings of the ICML Workshop.

[50]

Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2018. Analysis of classifiers’ robustness to adversarial perturbations. Mach. Learn. 107 (2018), 481--508.

Digital Library

[51]

Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. 2016. Robustness of classifiers: From adversarial to random noise. In Proceedings of the NeurIPS.

[52]

Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, and Andrew B. Gardner. 2017. Detecting adversarial samples from artifacts. arXiv:1703.00410 (2017).

[53]

Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the ICML.

[54]

Angus Galloway, Graham W. Taylor, and Medhat Moussa. 2017. Attacking binarized neural networks. arXiv:1711.00449 (2017).

[55]

Ji Gao, Beilun Wang, Zeming Lin, Weilin Xu, and Yanjun Qi. 2017. Deepcloak: Masking deep neural network models for robustness against adversarial samples. In Proceedings of the ICLR.

[56]

Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin Vechev. 2018. AI 2: Safety and robustness certification of neural networks with abstract interpretation. In Proceedings of the S8P. IEEE.

[57]

Partha Ghosh, Arpan Losalka, and Michael J. Black. 2018. Resisting adversarial attacks using Gaussian mixture variational autoencoders. arXiv:1806.00081 (2018).

[58]

Justin Gilmer, Ryan P. Adams, Ian Goodfellow, David Andersen, and George E. Dahl. 2018. Motivating the rules of the game for adversarial example research. arXiv:1807.06732 (2018).

[59]

Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S. Schoenholz, Maithra Raghu, Martin Wattenberg, and Ian Goodfellow. 2018. Adversarial spheres. arXiv:1801.02774 (2018).

[60]

Federico Girosi, Michael Jones, and Tomaso Poggio. 1995. Regularization theory and neural networks architectures. Neural Comput. 7 (1995), 219--269.

Digital Library

[61]

Amir Globerson and Sam Roweis. 2006. Nightmare at test time: Robust learning by feature deletion. In Proceedings of the ICML.

Digital Library

[62]

Zhitao Gong, Wenlu Wang, and Wei-Shinn Ku. 2017. Adversarial and clean data are not twins. arXiv:1704.04960 (2017).

[63]

Ian Goodfellow. 2018. Gradient masking causes CLEVER to overestimate adversarial perturbation size. arXiv:1804.07870 (2018).

[64]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the NeurIPS.

[65]

Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of the ICLR.

[66]

Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. 2018. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv:1810.12715 (2018).

[67]

Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. 2017. On the (statistical) detection of adversarial examples. arXiv:1702.06280 (2017).

[68]

Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. 2016. Adversarial perturbations against deep neural networks for malware classification. arXiv:1606.04435 (2016).

[69]

Shixiang Gu and Luca Rigazio. 2014. Towards deep neural network architectures robust to adversarial examples. arXiv:1412.5068 (2014).

[70]

Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. 2018. Countering adversarial images using input transformations. In Proceedings of the ICLR.

[71]

David Ha, Andrew Dai, and Quoc V Le. 2017. Hypernetworks. In Proceedings of the ICLR.

[72]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the CVPR. IEEE.

[73]

Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. 2017. Adversarial example defense: Ensembles of weak defenses are not strong. In Proceedings of the USENIX WOOT.

[74]

Matthias Hein and Maksym Andriushchenko. 2017. Formal guarantees on the robustness of a classifier against adversarial manipulation. In Proceedings of the NeurIPS.

[75]

Dan Hendrycks and Thomas Dietterich. 2019. Benchmarking neural network robustness to common corruptions and perturbations. In Proceedings of the ICLR.

[76]

Dan Hendrycks and Kevin Gimpel. 2016. Early methods for detecting adversarial images. In Proceedings of the ICLR Workshop.

[77]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv:1503.02531 (2017).

[78]

Weiwei Hu and Ying Tan. 2017. Generating adversarial malware examples for black-box attacks based on GAN. arXiv:1702.05983 (2017).

[79]

Ruitong Huang, Bing Xu, Dale Schuurmans, and Csaba Szepesvári. 2015. Learning with a strong adversary. arXiv:1511.03034 (2015).

[80]

Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel. 2017. Adversarial attacks on neural network policies. In Proceedings of the ICLR Workshop.

[81]

Xiaowei Huang, Daniel Kroening, Marta Kwiatkowska, Wenjie Ruan, Youcheng Sun, Emese Thamo, Min Wu, and Xinping Yi. 2018. Safety and trustworthiness of deep neural networks: A survey. arXiv:1812.08342 (2018).

[82]

Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2017. Safety verification of deep neural networks. In Proceedings of the CAV. Springer.

[83]

Peter J. Huber. 2011. Robust statistics. In International Encyclopedia of Statistical Science. Springer.

[84]

Todd Huster, Cho-Yu Jason Chiang, and Ritu Chadha. 2018. Limitations of the Lipschitz constant as a defense against adversarial examples. In Proceedings of the ECML PKDD. Springer.

[85]

Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box adversarial attacks with limited queries and information. In Proceedings of the ICML.

[86]

Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. 2019. Adversarial examples are not bugs, they are features. arXiv:1905.02175 (2019).

[87]

Rauf Izmailov, Shridatt Sugrim, Ritu Chadha, Patrick McDaniel, and Ananthram Swami. 2018. Enablers of adversarial attacks in machine learning. In Proceedings of the MILCOM. IEEE.

[88]

Jason Jo and Yoshua Bengio. 2017. Measuring the tendency of CNNs to learn surface statistical regularities. arXiv:1711.11561 (2017).

[89]

Can Kanbak, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. 2018. Geometric robustness of deep networks: Analysis and improvement. In Proceedings of the CVPR. IEEE.

[90]

Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An efficient SMT solver for verifying deep neural networks. In Proceedings of the CAV. Springer.

[91]

Valentin Khrulkov and Ivan Oseledets. 2018. Art of singular vectors and universal adversarial perturbations. In Proceedings of the CVPR. IEEE.

[92]

Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational Bayes. arXiv:1312.6114 (2013).

[93]

Aleksander Kołcz and Choon Hui Teo. 2009. Feature weighting for improved classifier robustness. In Proceedings of the CEAS.

[94]

Felix Kreuk, Assi Barak, Shir Aviv-Reuven, Moran Baruch, Benny Pinkas, and Joseph Keshet. 2018. Adversarial examples on discrete sequences for beating whole-binary malware detection. arXiv:1802.04528 (2018).

[95]

Dmitry Krotov and John J. Hopfield. 2016. Dense associative memory for pattern recognition. In Proceedings of the NeurIPS.

[96]

Dmitry Krotov and John J. Hopfield. 2017. Dense associative memory is robust to adversarial inputs. Neural Comput. 30 (2017), 3151--3167.

Digital Library

[97]

Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial examples in the physical world. arXiv:1607.02533 (2016).

[98]

Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial machine learning at scale. arXiv:1611.01236 (2016).

[99]

Alex Lamb, Jonathan Binas, Anirudh Goyal, Dmitriy Serdyuk, Sandeep Subramanian, Ioannis Mitliagkas, and Yoshua Bengio. 2018. Fortified networks: Improving the robustness of deep networks by modeling the manifold of hidden representations. arXiv:1804.02485 (2018).

[100]

Hugo Larochelle, Yoshua Bengio, Jérôme Louradour, and Pascal Lamblin. 2009. Exploring strategies for training deep neural networks. J. Mach. Learn, Res. 10, Jan. (2009).

[101]

Pavel Laskov et al. 2014. Practical evasion of a learning-based classifier: A case study. In Proceedings of the S8P. IEEE.

[102]

Hyeungill Lee, Sungyeob Han, and Jungwoo Lee. 2017. Generative adversarial trainer: Defense to adversarial perturbations with GAN. arXiv:1705.03387 (2017).

[103]

Shasha Li, Ajaya Neupane, Sujoy Paul, Chengyu Song, Srikanth V. Krishnamurthy, Amit K. Roy Chowdhury, and Ananthram Swami. 2018. Adversarial perturbations against real-time video classification systems. arXiv:1807.00458 (2018).

[104]

Xin Li and Fuxin Li. 2017. Adversarial examples detection in deep networks with convolutional filter statistics. In Proceedings of the ICCV. IEEE.

[105]

Bin Liang, Hongcheng Li, Miaoqiang Su, Xirong Li, Wenchang Shi, and Xiaofeng Wang. 2017. Detecting adversarial examples in deep networks with adaptive noise reduction. arXiv:1705.08378 (2017).

[106]

Yen-Chen Lin, Zhang-Wei Hong, Yuan-Hong Liao, Meng-Li Shih, Ming-Yu Liu, and Min Sun. 2017. Tactics of adversarial attack on deep reinforcement learning agents. In Proceedings of the IJCAI.

[107]

Hsueh-Ti Derek Liu, Michael Tao, Chun-Liang Li, Derek Nowrouzezahrai, and Alec Jacobson. 2018. Beyond pixel norm-balls: Parametric adversaries using an analytically differentiable renderer. arXiv:1808.02651.

[108]

Qiang Liu, Pan Li, Wentao Zhao, Wei Cai, Shui Yu, and Victor C. M. Leung. 2018. A survey on security threats and defensive techniques of machine learning: A data driven view. IEEE Access 6 (2018), 12103--12117.

[109]

Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. 2016. Delving into transferable adversarial examples and black-box attacks. arXiv:1611.02770 (2016).

[110]

Daniel Lowd and Christopher Meek. 2005. Adversarial learning. In Proceedings of the SIGKDD. ACM.

Digital Library

[111]

Jiajun Lu, Theerasit Issaranon, and David A. Forsyth. 2017. SafetyNet: Detecting and rejecting adversarial examples robustly. In Proceedings of the ICCV. IEEE.

[112]

Jiajun Lu, Hussein Sibai, Evan Fabry, and David Forsyth. 2017. Standard detectors aren’t (currently) fooled by physical adversarial stop signs. arXiv:1710.03337 (2017).

[113]

Yan Luo, Xavier Boix, Gemma Roig, Tomaso Poggio, and Qi Zhao. 2015. Foveation-based mechanisms alleviate adversarial examples. arXiv:1511.06292 (2015).

[114]

Chunchuan Lyu, Kaizhu Huang, and Hai-Ning Liang. 2015. A unified gradient regularization family for adversarial examples. In Proceedings of the ICDM. IEEE.

Digital Library

[115]

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In Proceedings of the ICLR.

[116]

Dongyu Meng and Hao Chen. 2017. Magnet: A two-pronged defense against adversarial examples. In Proceedings of the ACM CCS. ACM.

Digital Library

[117]

Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. 2017. On detecting adversarial perturbations. In Proceedings of the ICLR.

[118]

Matthew Mirman, Timon Gehr, and Martin Vechev. 2018. Differentiable abstract interpretation for provably robust neural networks. In Proceedings of the ICML.

[119]

Tom M. Mitchell et al. 1997. Machine Learning. McGraw Hill, Burr Ridge, IL.

[120]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2017. Universal adversarial perturbations. In Proceedings of the CVPR. IEEE.

[121]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard, and Stefano Soatto. 2017. Analysis of universal adversarial perturbations. arXiv:1705.09554 (2017).

[122]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard, and Stefano Soatto. 2018. Robustness of classifiers to universal perturbations: A geometric perspective. In Proceedings of the ICLR.

[123]

Seyed Mohsen Moosavi Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. DeepFool: A simple and accurate method to fool deep neural networks. In Proceedings of the CVPR. IEEE.

[124]

Nina Narodytska and Shiva Prasad Kasiviswanathan. 2017. Simple black-box adversarial perturbations on deep neural networks. arXiv:1612.06299 (2017).

[125]

Aran Nayebi and Surya Ganguli. 2017. Biologically inspired protection of deep networks from adversarial attacks. arXiv:1703.09202 (2017).

[126]

Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the CVPR. IEEE.

[127]

Nicolas Papernot. 2018. A marauder’s map of security and privacy in machine learning. arXiv:1811.01134 (2018).

[128]

Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. 2016. Transferability in machine learning: From phenomena to black-box attacks using adversarial samples. arXiv:1605.07277 (2016).

[129]

Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the ASIACCS. ACM.

Digital Library

[130]

Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 2016. The limitations of deep learning in adversarial settings. In Proceedings of the EuroS8P. IEEE.

[131]

Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael P. Wellman. 2018. SoK: Security and privacy in machine learning. In Proceedings of the EuroS8P. IEEE.

[132]

Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. 2016. Distillation as a defense to adversarial perturbations against deep neural networks. In Proceedings of the S8P. IEEE.

[133]

Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017. Towards practical verification of machine learning: The case of computer vision systems. arXiv:1712.01785 (2017).

[134]

Sidney Pontes-Filho and Marcus Liwicki. 2018. Bidirectional learning for robust neural networks. arXiv:1805.08006 (2018).

[135]

Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie. 2018. Generative adversarial perturbations. In Proceedings of the CVPR. IEEE.

[136]

Adnan Siraj Rakin, Zhezhi He, Boqing Gong, and Deliang Fan. 2018. Blind pre-processing: A robust defense method against adversarial examples. arXiv:1802.01549 (2018).

[137]

Shakir Mohamed Rezende, Danilo Jimenez and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. arXiv:1401.4082 (2014).

[138]

Ishai Rosenberg, Asaf Shabtai, Lior Rokach, and Yuval Elovici. 2017. Generic black-box end-to-end attack against RNNs and other API calls based malware classifiers. arXiv:1707.05970 (2017).

[139]

Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, and Thomas Hofmann. 2018. Adversarially robust training through structured gradient regularization. arXiv:1805.08736 (2018).

[140]

Andras Rozsa, Manuel Günther, and Terrance E. Boult. 2016. Are accuracy and robustness correlated. In Proceedings of the ICMLA. IEEE.

[141]

Andras Rozsa, Manuel Gunther, and Terrance E. Boult. 2016. Towards robust deep neural networks with BANG. arXiv:1612.00138 (2016).

[142]

Wenjie Ruan, Xiaowei Huang, and Marta Kwiatkowska. 2018. Reachability analysis of deep neural networks with provable guarantees. In Proceedings of the IJCAI.

[143]

Benjamin I. P. Rubinstein, Blaine Nelson, Ling Huang, Anthony D. Joseph, Shing-hon Lau, Satish Rao, Nina Taft, and J. D. Tygar. 2009. Antidote: Understanding and defending against poisoning of anomaly detectors. In Proceedings of the SIGCOMM. ACM.

[144]

Paolo Russu, Ambra Demontis, Battista Biggio, Giorgio Fumera, and Fabio Roli. 2016. Secure kernel machines against evasion attacks. In Proceedings of the AISec. ACM.

Digital Library

[145]

Sara Sabour, Yanshuai Cao, Fartash Faghri, and David J. Fleet. 2015. Adversarial manipulation of deep representations. arXiv:1511.05122 (2015).

[146]

Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, and Pengchuan Zhang. 2019. A convex relaxation barrier to tight robust verification of neural networks. In Proceedings of the ICLR Workshop.

[147]

Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. 2018. Adversarially robust generalization requires more data. In Proceedings of the NeurIPS.

[148]

Uri Shaham, James Garritano, Yutaro Yamada, Ethan Weinberger, Alex Cloninger, Xiuyuan Cheng, and Kelly Stanton. 2018. Defending against adversarial images using basis functions transformations. arXiv:1803.10840 (2018).

[149]

Uri Shaham, Yutaro Yamada, and Sahand Negahban. 2015. Understanding adversarial training: Increasing local stability of neural nets through robust optimization. arXiv:1511.05432 (2015).

[150]

Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. 2016. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the SIGSAC. ACM.

[151]

Yash Sharma and Pin-Yu Chen. 2017. Breaking the Madry defense model with L1-based adversarial examples. arXiv:1710.10733 (2017).

[152]

Ayan Sinha, Zhao Chen, Vijay Badrinarayanan, and Andrew Rabinovich. 2018. Gradient adversarial training of neural networks. arXiv:1806.08028 (2018).

[153]

Aman Sinha, Hongseok Namkoong, and John Duchi. 2018. Certifying some distributional robustness with principled adversarial training. In Proceedings of the ICLR.

[154]

Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. 2018. PixelDefend: Leveraging generative models to understand and defend against adversarial examples. In Proceedings of the ICLR.

[155]

Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon. 2018. Constructing unrestricted adversarial examples with generative models. In Proceedings of the NeurIPS.

[156]

Suvrit Sra, Sebastian Nowozin, and Stephen J. Wright. 2012. Optimization for Machine Learning. The MIT Press.

[157]

Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, and Yupeng Gao. 2018. Is robustness the cost of accuracy?–A comprehensive study on the robustness of 18 deep image classification models. In Proceedings of the ECCV.

[158]

Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. 2019. One pixel attack for fooling deep neural networks. IEEE Trans. Evolut. Comput. 23 (2019), 828--841.

[159]

Zhun Sun, Mete Ozay, and Takayuki Okatani. 2017. HyperNetworks with statistical filtering for defending adversarial examples. arXiv:1711.01791 (2017).

[160]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the NeurIPS.

Digital Library

[161]

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv:1312.6199 (2013).

[162]

Pedro Tabacof and Eduardo Valle. 2016. Exploring the space of adversarial images. In Proceedings of the JCNN.

[163]

Thomas Tanay and Lewis Griffin. 2016. A boundary tilting persepective on the phenomenon of adversarial examples. arXiv:1608.07690 (2016).

[164]

Simen Thys, Wiebe Van Ranst, and Toon Goedemé. 2019. Fooling automated surveillance cameras: Adversarial patches to attack person detection. arXiv:1904.08653 (2019).

[165]

Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. 2017. Ensemble adversarial training: Attacks and defenses. arXiv:1705.07204 (2017).

[166]

Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. 2017. The space of transferable adversarial examples. arXiv:1704.03453 (2017).

[167]

Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. 2019. Robustness may be at odds with accuracy. In Proceedings of the ICLR.

[168]

Leslie G. Valiant. 1984. A theory of the learnable. In Proceedings of the STOC. ACM.

Digital Library

[169]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the NeurIPS.

[170]

Xingxing Wei, Siyuan Liang, Xiaochun Cao, and Jun Zhu. 2018. Transferable adversarial attacks for image and video object detection. arXiv:1811.12641 (2018).

[171]

Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. 2018. Evaluating the robustness of neural networks: An extreme value theory approach. In Proceedings of the ICLR.

[172]

Matthew Wicker, Xiaowei Huang, and Marta Kwiatkowska. 2018. Feature-guided black-box safety testing of deep neural networks. In Proceedings of the TACAS. Springer.

[173]

Eric Wong and J. Zico Kolter. 2018. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Proceedings of the ICML.

[174]

Eric Wong, Frank R. Schmidt, and J. Zico Kolter. 2019. Wasserstein adversarial examples via projected Sinkhorn iterations. arXiv:1902.07906 (2019).

[175]

Min Wu, Matthew Wicker, Wenjie Ruan, Xiaowei Huang, and Marta Kwiatkowska. 2018. A game-based approximate verification of deep neural networks with provable guarantees. arXiv:1807.03571 (2018).

[176]

Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. 2018. Spatially transformed adversarial examples. In Proceedings of the ICLR.

[177]

Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. 2018. Mitigating adversarial effects through randomization. In Proceedings of the ICLR.

[178]

Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. 2017. Adversarial examples for semantic segmentation and object detection. In Proceedings of the CV. IEEE.

[179]

Weilin Xu, Yanjun Qi, and David Evans. 2016. Automatically evading classifiers. In Proceedings of the NDSS.

[180]

Hiromu Yakura and Jun Sakuma. 2018. Robust audio adversarial example for a physical attack. arXiv:1810.11793 (2018).

[181]

Pengfei Yang, Jiangchao Liu, Jianlin Li, Liqian Chen, and Xiaowei Huang. 2019. Analyzing deep neural networks with symbolic propagation: Towards higher precision and faster verification. arXiv:1902.09866 (2019).

[182]

Daniel S. Yeung, Ian Cloete, Daming Shi, and Wing Y Ng. 2010. Sensitivity Analysis for Neural Networks. Springer.

[183]

Chiliang Zhang, Zhimou Yang, and Zuochang Ye. 2018. Detecting adversarial perturbations with saliency. arXiv:1803.08773 (2018).

[184]

Huan Zhang, Hongge Chen, Zhao Song, Duane Boning, Inderjit S. Dhillon, and Cho-Jui Hsieh. 2019. The limitations of adversarial training and the blind-spot attack. In Proceedings of the ICLR.

[185]

Pinlong Zhao, Zhouyu Fu, Qinghua Hu, Jun Wang et al. 2018. Detecting adversarial examples via key-based network. arXiv:1806.00580 (2018).

[186]

Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018. Generating natural adversarial examples. In Proceedings of the ICLR.

Cited By

Azadmanesh MShahgholi Ghahfarokhi BAshouri Talouki M(2025)On the local convergence of ADAM-DPGAN with simultaneous and alternating gradient decent training methodsExpert Systems with Applications10.1016/j.eswa.2024.125646262(125646)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125646
Chan KCheng B(2025)Evoattack: suppressive adversarial attacks against object detection models using evolutionary searchAutomated Software Engineering10.1007/s10515-024-00470-932:1Online publication date: 1-Jun-2025
https://dl.acm.org/doi/10.1007/s10515-024-00470-9
Mei SLian JWang XSu YMa MChau L(2024)A Comprehensive Study on the Robustness of Deep Learning-Based Image Classification and Object Detection in Remote Sensing: Surveying and BenchmarkingJournal of Remote Sensing10.34133/remotesensing.02194Online publication date: 3-Oct-2024
https://doi.org/10.34133/remotesensing.0219
Show More Cited By

Index Terms

Adversarial Examples on Object Recognition: A Comprehensive Survey
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems
Abstract
Over the last few years, the adoption of machine learning in a wide range of domains has been remarkable. Deep learning, in particular, has been extensively used to drive applications and services in specializations such as computer vision, ...
Highlights
- A taxonomy of cybersecurity applications is established.
- Adversarial machine learning is systematically overviewed.
- An extensive, curated list of cybersecurity-related datasets is provided.
- Methods for generating adversarial ...
Resisting Adversarial Examples via Wavelet Extension and Denoising
Smart Computing and Communication
Abstract
It is well known that Deep Neural Networks are vulnerable to adversarial examples. An adversary can inject carefully-crafted perturbations on clean input to manipulate the model output. In this paper, we propose a novel method, WED (Wavelet ...
Assessing transferability of adversarial examples against malware detection classifiers
CF '19: Proceedings of the 16th ACM International Conference on Computing Frontiers

Machine learning (ML) algorithms provide better performance than traditional algorithms in various applications. However, some unknown flaws in ML classifiers make them sensitive to adversarial examples generated by adding small but fooled purposeful ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 53, Issue 3

May 2021

787 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3403423

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2020

Online AM: 07 May 2020

Accepted: 01 March 2020

Revised: 01 January 2020

Received: 01 October 2018

Published in CSUR Volume 53, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

87
Total Citations
View Citations
2,029
Total Downloads

Downloads (Last 12 months)348
Downloads (Last 6 weeks)27

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Azadmanesh MShahgholi Ghahfarokhi BAshouri Talouki M(2025)On the local convergence of ADAM-DPGAN with simultaneous and alternating gradient decent training methodsExpert Systems with Applications10.1016/j.eswa.2024.125646262(125646)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125646
Chan KCheng B(2025)Evoattack: suppressive adversarial attacks against object detection models using evolutionary searchAutomated Software Engineering10.1007/s10515-024-00470-932:1Online publication date: 1-Jun-2025
https://dl.acm.org/doi/10.1007/s10515-024-00470-9
Mei SLian JWang XSu YMa MChau L(2024)A Comprehensive Study on the Robustness of Deep Learning-Based Image Classification and Object Detection in Remote Sensing: Surveying and BenchmarkingJournal of Remote Sensing10.34133/remotesensing.02194Online publication date: 3-Oct-2024
https://doi.org/10.34133/remotesensing.0219
Wozniak ASegura SMazo R(2024)Robustness Assessment of AI-Based 2D Object Detection Systems: A Method and Lessons Learned from Two Industrial CasesElectronics10.3390/electronics1307136813:7(1368)Online publication date: 4-Apr-2024
https://doi.org/10.3390/electronics13071368
Xu LZhai J(2024)DCVAE-adv: A Universal Adversarial Example Generation Method for White and Black Box AttacksTsinghua Science and Technology10.26599/TST.2023.901000429:2(430-446)Online publication date: Apr-2024
https://doi.org/10.26599/TST.2023.9010004
Rangel GCuevas-Tello JNunez-Varela JPuente CSilva-Trujillo A(2024)A Survey on Convolutional Neural Networks and Their Performance Limitations in Image Recognition TasksJournal of Sensors10.1155/2024/27973202024:1Online publication date: 12-Jul-2024
https://doi.org/10.1155/2024/2797320
Kochanthara SSingh TForrai ACleophas L(2024)Safety of Perception Systems for Automated Driving: A Case Study on ApolloACM Transactions on Software Engineering and Methodology10.1145/363196933:3(1-28)Online publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1145/3631969
Xiong PTegegn MSarin JPal SRubin J(2024)It Is All about Data: A Survey on the Effects of Data on Adversarial RobustnessACM Computing Surveys10.1145/362781756:7(1-41)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3627817
Bayer JBecker SMünch DArens M(2024)Network transferability of adversarial patches in real-time object detectionArtificial Intelligence for Security and Defence Applications II10.1117/12.3031501(33)Online publication date: 13-Nov-2024
https://doi.org/10.1117/12.3031501
Hua GTeoh AXiang YJiang H(2024)Unambiguous and High-Fidelity Backdoor Watermarking for Deep Neural NetworksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.325021035:8(11204-11217)Online publication date: Aug-2024
https://doi.org/10.1109/TNNLS.2023.3250210
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents