Yinan Zhong (Zhejiang University), Qianhao Miao (Zhejiang University), Yanjiao Chen (Zhejiang University), Jiangyi Deng (Zhejiang University), Yushi Cheng (Zhejiang University), Wenyuan Xu (Zhejiang University)

Large Language Models (LLMs) have been integrated into many applications (e.g., web agents) to perform more sophisticated tasks. However, LLM-empowered applications are vulnerable to Indirect Prompt Injection (IPI) attacks, where instructions are injected via untrustworthy external data sources. This paper presents Rennervate, a defense framework to detect and prevent IPI attacks. Rennervate leverages attention features to detect the covert injection at a fine-grained token level, enabling precise sanitization that neutralizes IPI attacks while maintaining LLM functionalities. Specifically, the token-level detector is materialized with a 2-step attentive pooling mechanism, which aggregates attention heads and response tokens for IPI detection and sanitization. Moreover, we establish a fine-grained IPI dataset, FIPI, to be open-sourced to support further research. Extensive experiments verify that Rennervate outperforms 15 commercial and academic IPI defense methods, achieving high precision on 5 LLMs and 6 datasets. We also demonstrate that Rennervate is transferable to unseen attacks and robust against adaptive adversaries.

View More Papers

CRISP: An Efficient Cryptographic Framework for ML Inference Against...

Xiaoyu Fang (Beijing University of Posts and Telecommunications), Shihui Zheng (Beijing University of Posts and Telecommunications), Lize Gu (Beijing University of Posts and Telecommunications)

Read More

TENSURE: Fuzzing Sparse Tensor Compilers (Registered Report)

Kabilan Mahathevan (Department of Computer Science, Virginia Tech, Blacksburg), Yining Zhang (Department of Computer Science, Virginia Tech, Blacksburg), Muhammad Ali Gulzar (Department of Computer Science, Virginia Tech, Blacksburg), Kirshanthan Sundararajah (Department of Computer Science, Virginia Tech, Blacksburg)

Read More

Position Paper: Towards Ubiquitous and Automated User Privacy Configuration

Song Liao (Texas Tech University), Jingwen Yan (Clemson University), Yichen Liu (University of Illinois Urbana-Champaign), David Kotz (Dartmouth College), Luyi Xing (University of Illinois Urbana-Champaign), Long Cheng (Clemson University)

Read More