Awesome-LVLM-Attack

A continual collection of papers related to Attacks on Large-Vision-Language-Models (LVLMs).

Large vision-language models (LVLMs) have achieved significant success and demonstrated promising capabilities in various multimodal downstream tasks. Despite their remarkable capabilities, the increased complexity and deployment of LVLMs have also exposed them to various security threats and vulnerabilities, making the study of attacks on these models a critical area of research.

Here, we've summarized existing LVLM Attack methods in our survey paper👍.

If you find some important work missed, it would be super helpful to let me know (dzliu@stu.pku.edu.cn). Thanks!

If you find our survey useful for your research, please consider citing:

@article{liu2024attack,
  title={A Survey of Attacks on Large Vision-Language Models: Resources, Recent Advances, and Future Trends},
  author={Liu, Daizong and Yang, Mingyu and Qu, Xiaoye and Hu, Wei},
  journal={arXiv preprint arXiv},
  year={2024}
}

Table of Contents

• Adversarial Attacks
• Jailbreack Attacks
• Prompt Injection
• Data Poisoning

---

Adversarial-Attack

• ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language | Github
  • Dave Zhenyu Chen, Angel X. Chang, Matthias Nießner
  • Technical University of Munich, Simon Fraser University
  • [ECCV2020] https://arxiv.org/abs/1912.08830
  • A dataset, two-stage approach, proposal-then-selection

Jailbreak-Attack

Prompt-Injection

Data-Poisoning