Amrita Roy Chowdhury (University of Michigan, Ann Arbor), David Glukhov (University of Toronto and Vector Institute), Divyam Anshumaan (University of Wisconsin-Madison), Prasad Chalasani (Langroid Incorporated), Nicholas Papernot (University of Toronto and Vector Institute), Somesh Jha (University of Wisconsin-Madison), Mihir Bellare (University of California, San Diego)

The rise of large language models (LLMs) has introduced new privacy challenges, particularly during inference where sensitive information in prompts may be exposed to proprietary LLM APIs. In this paper, we address the problem of formally protecting the sensitive information contained in a prompt while maintaining response quality. To this end, first, we introduce a cryptographically inspired notion of a prompt sanitizer which transforms an input prompt to protect its sensitive tokens. Second, we propose Prϵϵmpt, a novel system that implements a prompt sanitizer, focusing on the sensitive information that can be derived solely from the individual tokens. Prϵϵmpt categorizes sensitive tokens into two types: (1) those where the LLM’s response depends solely on the format (such as SSNs, credit card numbers), for which we use format-preserving encryption (FPE); and (2) those where the response depends on specific values, (such as age, salary) for which we apply metric differential privacy (mDP). Our evaluation demonstrates that Prϵϵmpt is a practical method to achieve meaningful privacy guarantees, while maintaining high utility compared to unsanitized prompts, and outperforming prior methods.

View More Papers

Lightweight Internet Bandwidth Allocation and Isolation with Fractional Fair...

Marc Wyss (ETH Zurich), Yih-Chun Hu (University of Illinois at Urbana-Champaign), Vincent Lenders (University of Luxembourg), Roland Meier (armasuisse), Adrian Perrig (ETH Zurich)

Read More

vSim: Semantics-Aware Value Extraction for Efficient Binary Code Similarity...

Huaijin Wang (The Ohio State University), Zhiqiang Lin (The Ohio State University)

Read More

SoK: Cryptographic Authenticated Dictionaries

Harjasleen Malvai (University of Illinois, Urbana-Champaign), Francesca Falzon (ETH Zürich), Andrew Zitek-Estrada (EPFL), Sarah Meiklejohn (University College London), Joseph Bonneau (NYU)

Read More