Eden Luzon (Ben-Gurion University, Institute of Software Systems and Security), Guy Amit (Ben-Gurion University, Institute of Software Systems and Security), Roy Weiss (Ben-Gurion University, Institute of Software Systems and Security), Torsten Krauß (University of Würzburg), Alexandra Dmitrienko (University of Würzburg), Yisroel Mirsky (Ben-Gurion University, Institute of Software Systems and Security)

Neural networks are often trained on proprietary datasets, making them attractive attack targets. We present a novel dataset extraction method leveraging an innovative training-time backdoor attack, allowing a malicious federated learning (FL) server to systematically and deterministically extract complete client training samples through a simple indexing process. Unlike prior techniques, our approach guarantees exact data recovery rather than probabilistic reconstructions or hallucinations, provides precise control over which samples are memorized and how many, and shows high capacity and robustness. Infected models output data samples when they receive a pattern-based index trigger, enabling systematic extraction of meaningful patches from each client’s local data without disrupting global model utility. To address small model output sizes, we extract patches and then recombined them.

The attack requires only a minor modification to the training code that can easily evade detection during client-side verification. Hence, this vulnerability represents a realistic FL supply-chain threat, where a malicious server can distribute modified training code to clients and later recover private data from their updates. Evaluations across classifiers, segmentation models, and large language models demonstrate that thousands of sensitive training samples can be recovered from client models with minimal impact on task performance, and a client's entire dataset can be stolen after multiple FL rounds. For instance, a medical segmentation dataset can be extracted with only a 3% utility drop. These findings expose a critical privacy vulnerability in FL systems, emphasizing the need for stronger integrity and transparency in distributed training pipelines.

View More Papers

From Matrix to Metrics: Introducing and Applying a Configuration...

Tobias Länge (SECUSO, Karlsruhe Institute of Technology, Karlsruhe, Germany), Fabian Lucas Ballreich (SECUSO, Karlsruhe Institute of Technology, Karlsruhe, Germany), Anne Hennig (SECUSO, Karlsruhe Institute of Technology, Karlsruhe, Germany), Peter Mayer (SECUSO, Karlsruhe Institute of Technology, Karlsruhe, Germany), Melanie Volkamer (SECUSO, Karlsruhe Institute of Technology, Karlsruhe, Germany)

Read More

PriMod4AI: Lifecycle-Aware Privacy Threat Modeling for AI Systems using...

Gautam Savaliya (Deggendorf Institute of Technology, Germany), Robert Aufschlager (Deggendorf Institute of Technology, Germany), Abhishek Subedi (Deggendorf Institute of Technology, Germany), Michael Heigl (Deggendorf Institute of Technology, Germany), Martin Schramm (Deggendorf Institute of Technology, Germany)

Read More

PathProb: Probabilistic Inference and Path Scoring for Enhanced and...

Yingqian Hao (Computer Network Information Center, Chinese Academy of Sciences; University of Chinese Academy of Sciences), Hui Zou (Computer Network Information Center, Chinese Academy of Sciences; University of Chinese Academy of Sciences), Lu Zhou (Computer Network Information Center, Chinese Academy of Sciences; University of Chinese Academy of Sciences), Yuxuan Chen (Computer Network Information Center, Chinese…

Read More