Awesome-LVLM-Attack

A continual collection of papers related to Attacks on Large-Vision-Language-Models (LVLMs).

Large vision-language models (LVLMs) have achieved significant success and demonstrated promising capabilities in various multimodal downstream tasks. Despite their remarkable capabilities, the increased complexity and deployment of LVLMs have also exposed them to various security threats and vulnerabilities, making the study of attacks on these models a critical area of research.

Here, we've summarized existing LVLM Attack methods in our survey paper👍.

If you find some important work missed, it would be super helpful to let me know (dzliu@stu.pku.edu.cn). Thanks!

If you find our survey useful for your research, please consider citing:

@article{liu2024attack,
  title={A Survey of Attacks on Large Vision-Language Models: Resources, Recent Advances, and Future Trends},
  author={Liu, Daizong and Yang, Mingyu and Qu, Xiaoye and Hu, Wei},
  journal={arXiv preprint arXiv},
  year={2024}
}

Table of Contents

• Adversarial Attacks
• Jailbreack Attacks
• Prompt Injection
• Data Poisoning

---

Adversarial-Attack

• On the Adversarial Robustness of Multi-Modal Foundation Models |
  • Christian Schlarmann, Matthias Hein
  • University of Tubingen
  • [ICCVworkshop2023] https://openaccess.thecvf.com/content/ICCV2023W/AROW/papers/Schlarmann_On_the_Adversarial_Robustness_of_Multi-Modal_Foundation_Models_ICCVW_2023_paper.pdf
• On Evaluating Adversarial Robustness of Large Vision-Language Models | Github
  • Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin
  • Singapore University of Technology and Design, Sea AI Lab, Tsinghua University, Renmin University of China
  • [NeurIPs2023] https://arxiv.org/abs/2305.16934
• Adversarial Illusions in Multi-Modal Embeddings | Github
  • Tingwei Zhang, Rishi Jha, Eugene Bagdasaryan, Vitaly Shmatikov
  • Cornell University, Cornell Tech
  • [Arxiv2023] https://arxiv.org/abs/2308.11804
• Image Hijacks: Adversarial Images can Control Generative Models at Runtime | Github
  • Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons
  • UC Berkeley, Harvard University, University of Cambridge
  • [Arxiv2023] https://arxiv.org/abs/2309.00236
• How Robust is Google's Bard to Adversarial Image Attacks? | Github
  • Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu
  • Tsinghua University, RealAI
  • [Arxiv2023] https://arxiv.org/abs/2309.11751
• Misusing Tools in Large Language Models With Visual Adversarial Examples |
  • Xiaohan Fu, Zihan Wang, Shuheng Li, Rajesh K. Gupta, Niloofar Mireshghallah, Taylor Berg-Kirkpatrick, Earlence Fernandes
  • University of California San Diego, University of Washington
  • [Arxiv2023] https://arxiv.org/abs/2310.03185
• How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs | Github
  • Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, Cihang Xie
  • UC Santa Cruz, UNC-Chapel Hill, University of Edinburgh, University of Oxford, AIWaves Inc.
  • [Arxiv2023] https://arxiv.org/abs/2311.16101
• InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models |
  • Xunguang Wang, Zhenlan Ji, Pingchuan Ma, Zongjie Li, Shuai Wang
  • The Hong Kong University of Science and Technology
  • [Arxiv2023] https://arxiv.org/abs/2312.01886
• An Image Is Worth 1000 Lies: Transferability of Adversarial Images across Prompts on Vision-Language Models | Github
  • Haochen Luo, Jindong Gu, Fengyuan Liu, Philip Torr
  • University of Oxford
  • [ICLR2024] https://arxiv.org/abs/2403.09766
• Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images | Github
  • Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, Wei Liu
  • Tsinghua University, Tencent Technology (Beijing), University of Oxford, Tencent Data Platform, Peng Cheng Laboratory
  • [ICLR2024] https://arxiv.org/abs/2401.11170
• The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative | Github
  • Zhen Tan, Chengshuai Zhao, Raha Moraffah, Yifan Li, Yu Kong, Tianlong Chen, Huan Liu
  • Arizona State University, Michigan State University, Harvard University
  • [Arxiv2024] https://arxiv.org/abs/2402.14859

Jailbreak-Attack

Prompt-Injection

Data-Poisoning