diff --git a/README.md b/README.md index d518e63..7566c77 100644 --- a/README.md +++ b/README.md @@ -28,11 +28,51 @@ Here, we've summarized existing LVLM Attack methods in our survey paper👍. --- ## Adversarial-Attack -* **ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language** | [Github](https://github.com/daveredrum/ScanRefer) - * Dave Zhenyu Chen, Angel X. Chang, Matthias Nießner - * Technical University of Munich, Simon Fraser University - * [ECCV2020] https://arxiv.org/abs/1912.08830 - * A dataset, two-stage approach, proposal-then-selection +* **On the Adversarial Robustness of Multi-Modal Foundation Models** | + * Christian Schlarmann, Matthias Hein + * University of Tubingen + * [ICCVworkshop2023] https://openaccess.thecvf.com/content/ICCV2023W/AROW/papers/Schlarmann_On_the_Adversarial_Robustness_of_Multi-Modal_Foundation_Models_ICCVW_2023_paper.pdf +* **On Evaluating Adversarial Robustness of Large Vision-Language Models** | [Github](https://github.com/yunqing-me/AttackVLM) + * Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin + * Singapore University of Technology and Design, Sea AI Lab, Tsinghua University, Renmin University of China + * [NeurIPs2023] https://arxiv.org/abs/2305.16934 +* **Adversarial Illusions in Multi-Modal Embeddings** | [Github](https://github.com/ebagdasa/adversarial_illusions) + * Tingwei Zhang, Rishi Jha, Eugene Bagdasaryan, Vitaly Shmatikov + * Cornell University, Cornell Tech + * [Arxiv2023] https://arxiv.org/abs/2308.11804 +* **Image Hijacks: Adversarial Images can Control Generative Models at Runtime** | [Github](https://github.com/euanong/image-hijacks) + * Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons + * UC Berkeley, Harvard University, University of Cambridge + * [Arxiv2023] https://arxiv.org/abs/2309.00236 +* **How Robust is Google's Bard to Adversarial Image Attacks?** | [Github](https://github.com/thu-ml/Attack-Bard) + * Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu + * Tsinghua University, RealAI + * [Arxiv2023] https://arxiv.org/abs/2309.11751 +* **Misusing Tools in Large Language Models With Visual Adversarial Examples** | + * Xiaohan Fu, Zihan Wang, Shuheng Li, Rajesh K. Gupta, Niloofar Mireshghallah, Taylor Berg-Kirkpatrick, Earlence Fernandes + * University of California San Diego, University of Washington + * [Arxiv2023] https://arxiv.org/abs/2310.03185 +* **How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs** | [Github](https://github.com/UCSC-VLAA/vllm-safety-benchmark) + * Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, Cihang Xie + * UC Santa Cruz, UNC-Chapel Hill, University of Edinburgh, University of Oxford, AIWaves Inc. + * [Arxiv2023] https://arxiv.org/abs/2311.16101 +* **InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models** | + * Xunguang Wang, Zhenlan Ji, Pingchuan Ma, Zongjie Li, Shuai Wang + * The Hong Kong University of Science and Technology + * [Arxiv2023] https://arxiv.org/abs/2312.01886 +* **An Image Is Worth 1000 Lies: Transferability of Adversarial Images across Prompts on Vision-Language Models** | [Github](https://github.com/Haochen-Luo/CroPA) + * Haochen Luo, Jindong Gu, Fengyuan Liu, Philip Torr + * University of Oxford + * [ICLR2024] https://arxiv.org/abs/2403.09766 +* **Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images** | [Github](https://github.com/KuofengGao/Verbose_Images) + * Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, Wei Liu + * Tsinghua University, Tencent Technology (Beijing), University of Oxford, Tencent Data Platform, Peng Cheng Laboratory + * [ICLR2024] https://arxiv.org/abs/2401.11170 +* **The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative** | [Github](https://github.com/ChengshuaiZhao0/The-Wolf-Within) + * Zhen Tan, Chengshuai Zhao, Raha Moraffah, Yifan Li, Yu Kong, Tianlong Chen, Huan Liu + * Arizona State University, Michigan State University, Harvard University + * [Arxiv2024] https://arxiv.org/abs/2402.14859 + ## Jailbreak-Attack