Visualizing One Pixel Attack Using Adversarial Maps

One pixel attack is one of the most puzzling adversarial attacks, in which the position of the attack plays an important role. However, little research has been conducted on the distributions of one pixel attack. In this context, a technique called adversarial maps is proposed, which helps visualize the distributions of one pixel attack for the first time. Adversarial maps consist of pixel adversarial maps and probability adversarial maps, which record the pixel changes and the confidence of the target class in successful attack cases, respectively. Leveraging this technique, one pixel attack distributions and why the position of one pixel attack impacts success rate is explored. Adversarial maps reveal that successful attacks always group as regions and the high saliency areas of saliency maps are more likely to be attacked successfully. Moreover, these observations are further corroborated by a mathematical analysis, demonstrating that adversarial attacks are disturbances in the saliency maps.