Zihang Xiang (KAUST), Tianhao Wang (University of Virginia), Cheng-Long Wang (KAUST), Di Wang (KAUST)

We investigate the application of differential privacy in hyper-parameter tuning, a process involving selecting the best run from several candidates. Unlike many private learning algorithms, including the prevalent DP-SGD, the privacy implications of selecting the best are often overlooked. While recent works propose a generic private selection solution for the tuning process, an open question persists: is such privacy upper bound tight?

This paper provides both empirical and theoretical examinations of this question. Initially, we provide studies affirming the current privacy analysis for private selection is indeed tight in general. However, when we specifically study the hyper-parameter tuning problem in a white-box setting, such tightness no longer holds. This is first demonstrated by applying privacy audit on the tuning process. Our findings underscore a substantial gap between the current theoretical privacy bound and the empirical privacy leakage derived even under strong audit setups.

This gap motivates our subsequent theoretical investigations, which provide improved privacy upper bound for private hyper-parameter tuning due to its distinct properties. Our improved bound leads to better utility. Our analysis also demonstrates broader applicability compared to prior analyses, which are limited to specific parameter configurations. Overall, we contribute to a better understanding of how privacy degrades due to selection.

View More Papers

G-Prove: Gossip-Based Provenance for Scalable Detection of Cross-Domain Flow...

Moustapha Awwalou Diouf (SnT, University of Luxembourg), Maimouna Tamah Diao (SnT, University of Luxembourg), El-hacen Diallo (SnT, University of Luxembourg), Samuel Ouya (Cheikh Hamidou KANE Digital University), Jacques Klein (SnT, University of Luxembourg), Tegawendé F. Bissyandé (SnT, University of Luxembourg)

Read More

Beyond Jailbreak: Unveiling Risks in LLM Applications Arising from...

Yunyi Zhang (Tsinghua University), Shibo Cui (Tsinghua University), Baojun Liu (Tsinghua University), Jingkai Yu (Tsinghua University), Min Zhang (National University of Defense Technology), Fan Shi (National University of Defense Technology), Han Zheng (TrustAl Pte. Ltd.)

Read More

SAGA: A Security Architecture for Governing AI Agentic Systems

Georgios Syros (Northeastern University), Anshuman Suri (Northeastern University), Jacob Ginesin (Northeastern University), Cristina Nita-Rotaru (Northeastern University), Alina Oprea (Northeastern University)

Read More