Aarnav Sawant¹ and Tyler Giallanza², ¹Bellarmine College Preparatory, USA, ²Princeton University, USA

Content Based Image Retrieval (CBIR) Systems have been employed in a wide variety of critical applications such as intellectual property management [47], facial recognition [46], and inappropriate content detection [48]. Most CBIRs are vulnerable to adversarial attacks, where small, imperceptible perturbations to input images cause system failure. In this paper, we propose a zero-query, black-box adversarial attack method that simulates an attack setting where the attacker has no knowledge about the CBIR model architecture and is unable to make multiple queries. The proposed method uses an ensemblebased approach, generating one perturbation for an input image that severely hinders the ability of six different CBIR models. Our approach is successfully able to disrupt the relevance of our target image retrieval models with a 65% decrease in Mean Average Precision (mAP) as compared to state-of-the-art UAP [18]. We hope our method serves as a baseline for the evaluation of robustness for future image retrieval research.

Adversarial attack, image retrieval, zero-query attack, black-box attack, ensemble attack.