Modifying the one-hot encoding technique can enhance the adversarial robustness of the visual model for symbol recognition

Deep learning systems, particularly those used in image classification, are threatened by Adversarial Examples. In contrast, Adversarial Examples do not affect the mammalian visual system. We undertake a comparative analysis of the traditional image multi-classification models and human cognitive frameworks, namely ACT-R and QN-MHP, and find that the One-hot encoded output structure lacks anatomical support. Furthermore, the CLIP model, which uses natural language supervision, closely resembles the human visual cognition process. There is structural information between category labels, and various label errors are not equivalent in most cases. One-hot encoding disregards these distinctions, exacerbating Adversarial Examples’ detrimental impact. We introduce a new direction for the adversarial defense that replaces One-hot encoding with natural language encoding or other encodings that preserve structural information between labels. Experiments and Lipschitz continuity analysis show that this approach can enhance the robustness of the model against Adversarial Samples, especially in scenarios such as visual symbol recognition.