Frequency based Adversarial Patch Localization

In this study, we present a novel approach for detecting adversarial patches in images through a frequency-based analysis. Leveraging the Segment Anything Model (SAM), we initially segment the images, creating distinct regions of interest. Subsequently, we employ a single class Support Vector Machine (SVM) to train on these segments, distinguishing adversarial patches from the natural ones. Our investigation delves into various frequency components, including Discrete Fourier Transform (DFT), Fast Fourier Transform (FFT), and entropy, each operating independently. The results demonstrate the effectiveness of our method in robustly identifying adversarial patches, showcasing the potential of frequency-based analysis for enhancing image security in the realm of adversarial attacks. Additionally, we inpainted the adversarial patch segments using Stable Diffusion thus decreasing the Attack Success Rate (ASR).

Code here