TY - JOUR
T1 - One Pixel Attack for Fooling Deep Neural Networks
AU - Su, Jiawei
AU - Vargas, Danilo Vasconcellos
AU - Sakurai, Kouichi
N1 - Funding Information:
Manuscript received March 12, 2018; revised June 21, 2018 and October 29, 2018; accepted December 20, 2018. Date of publication January 4, 2019; date of current version October 1, 2019. This work was supported in part by the Collaboration Hubs for International Program of SICORP, in part by the Japan Science and Technology Agency, and in part by the Kyushu University Education and Research Center for Mathematical and Data Science Grant. (Jiawei Su and Danilo Vasconcellos Vargas contributed equally to this work.) (Corresponding author: Jiawei Su.) J. Su and D. V. Vargas are with the Graduate School/Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan (e-mail: jiawei.su@inf.kyushu-u.ac.jp).
Publisher Copyright:
© 1997-2012 IEEE.
PY - 2019/10
Y1 - 2019/10
N2 - Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.
AB - Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.
UR - http://www.scopus.com/inward/record.url?scp=85073072064&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85073072064&partnerID=8YFLogxK
U2 - 10.1109/TEVC.2019.2890858
DO - 10.1109/TEVC.2019.2890858
M3 - Article
AN - SCOPUS:85073072064
SN - 1089-778X
VL - 23
SP - 828
EP - 841
JO - IEEE Transactions on Evolutionary Computation
JF - IEEE Transactions on Evolutionary Computation
IS - 5
M1 - 8601309
ER -