From 158a88d8b531e38dc75ec2dedf5693aef494c679 Mon Sep 17 00:00:00 2001 From: Daizong Liu Date: Fri, 31 May 2024 18:09:16 +0800 Subject: [PATCH] update 5.31.2024 --- README.md | 38 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 7566c77..71d57df 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,10 @@ Here, we've summarized existing LVLM Attack methods in our survey paper👍. * Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin * Singapore University of Technology and Design, Sea AI Lab, Tsinghua University, Renmin University of China * [NeurIPs2023] https://arxiv.org/abs/2305.16934 +* **VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models** | [Github](https://github.com/ericyinyzy/VLAttack) + * Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du, Jinguo Zhu, Han Liu, Jinghui Chen, Ting Wang, Fenglong Ma + * The Pennsylvania State University, Zhejiang University, Xi’an Jiaotong University, Dalian University of Technology, Stony Brook University + * [NeurIPs2023] [https://arxiv.org/abs/2312.03777](https://arxiv.org/abs/2310.04655) * **Adversarial Illusions in Multi-Modal Embeddings** | [Github](https://github.com/ebagdasa/adversarial_illusions) * Tingwei Zhang, Rishi Jha, Eugene Bagdasaryan, Vitaly Shmatikov * Cornell University, Cornell Tech @@ -56,10 +60,18 @@ Here, we've summarized existing LVLM Attack methods in our survey paper👍. * Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, Cihang Xie * UC Santa Cruz, UNC-Chapel Hill, University of Edinburgh, University of Oxford, AIWaves Inc. * [Arxiv2023] https://arxiv.org/abs/2311.16101 +* **On the Robustness of Large Multimodal Models Against Image Adversarial Attacks** | + * Xuanming Cui, Alejandro Aparcedo, Young Kyun Jang, Ser-Nam Lim + * University of Central Florida + * [Arxiv2023] https://arxiv.org/abs/2312.03777 * **InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models** | * Xunguang Wang, Zhenlan Ji, Pingchuan Ma, Zongjie Li, Shuai Wang * The Hong Kong University of Science and Technology * [Arxiv2023] https://arxiv.org/abs/2312.01886 +* **OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization** | + * Dongchen Han, Xiaojun Jia, Yang Bai, Jindong Gu, Yang Liu, Xiaochun Cao + * Sun Yat-sen University, Nanyang Technological University, Tsinghua University, University of Oxford + * [Arxiv2023] https://arxiv.org/abs/2312.04403 * **An Image Is Worth 1000 Lies: Transferability of Adversarial Images across Prompts on Vision-Language Models** | [Github](https://github.com/Haochen-Luo/CroPA) * Haochen Luo, Jindong Gu, Fengyuan Liu, Philip Torr * University of Oxford @@ -68,12 +80,34 @@ Here, we've summarized existing LVLM Attack methods in our survey paper👍. * Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, Wei Liu * Tsinghua University, Tencent Technology (Beijing), University of Oxford, Tencent Data Platform, Peng Cheng Laboratory * [ICLR2024] https://arxiv.org/abs/2401.11170 +* **Adversarial Robustness for Visual Grounding of Multimodal Large Language Models** | + * Kuofeng Gao, Yang Bai, Jiawang Bai, Yong Yang, Shu-Tao Xia + * Tsinghua University, Tencent Security Platform, Peng Cheng Laboratory + * [ICLRworkshop2024] https://arxiv.org/abs/2405.09981 +* **Transferable Multimodal Attack on Vision-Language Pre-training Models** | + * Haodi Wang, Kai Dong, Zhilei Zhu, Haotong Qin, Aishan Liu, Xiaolin Fang, Jiakai Wang, Xianglong Liu + * Southeast University, Data Space Research Institute of Hefei Comprehensive National Science Centre, Beihang University, Southeast University, Zhongguancun Laboratory + * [S&P2024] https://www.computer.org/csdl/proceedings-article/sp/2024/313000a102/1Ub239H4xyg +* **On the Safety Concerns of Deploying LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities** | + * Xiyang Wu, Ruiqi Xian, Tianrui Guan, Jing Liang, Souradip Chakraborty, Fuxiao Liu, Brian Sadler, Dinesh Manocha, Amrit Singh Bedi + * University of Maryland, Army Research Laboratory, University of Central Florida + * [Arxiv2024] https://arxiv.org/abs/2402.10340 * **The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative** | [Github](https://github.com/ChengshuaiZhao0/The-Wolf-Within) * Zhen Tan, Chengshuai Zhao, Raha Moraffah, Yifan Li, Yu Kong, Tianlong Chen, Huan Liu * Arizona State University, Michigan State University, Harvard University * [Arxiv2024] https://arxiv.org/abs/2402.14859 - - +* **Stop Reasoning! When Multimodal LLMs with Chain-of-Thought Reasoning Meets Adversarial Images** | + * Zefeng Wang, Zhen Han, Shuo Chen, Fan Xue, Zifeng Ding, Xun Xiao, Volker Tresp, Philip Torr, Jindong Gu + * Technical University of Munich, Ludwig Maximilian University of Munich, Huawei Munich Research Center, University of Oxford + * [Arxiv2024] https://arxiv.org/abs/2402.14899 +* **AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Adversarial Visual-Instructions** | + * Hao Zhang, Wenqi Shao, Hong Liu, Yongqiang Ma, Ping Luo, Yu Qiao, Kaipeng Zhang + * Xi’an Jiaotong University, Shanghai Artificial Intelligence Laboratory, Osaka University + * [Arxiv2024] https://arxiv.org/abs/2403.09346 +* **Efficiently Adversarial Examples Generation for Visual-Language Models under Targeted Transfer Scenarios using Diffusion Models** | + * Qi Guo, Shanmin Pang, Xiaojun Jia, Qing Guo + * Xi’an Jiaotong University, Nanyang Technological University, Center for Frontier AI Research + * [Arxiv2024] https://arxiv.org/abs/2404.10335 ## Jailbreak-Attack