Guofu Liao (Shenzhen University), Taotao Wang (Shenzhen University), Shengli Zhang (Shenzhen University), Jiqun Zhang (Shenzhen University), Long Shi (Nanjing University of Science and Technology), Dacheng Tao (Nanyang Technological University)

Fine-tuning large language models (LLMs) is crucial for adapting them to specific tasks, yet it remains computationally demanding and raises concerns about correctness and privacy, particularly in untrusted environments. Although parameter-efficient methods like Low-Rank Adaptation (LoRA) significantly reduce resource requirements, ensuring the security and verifiability of fine-tuning under zero-knowledge constraints remains an unresolved challenge. To address this, we introduce VeriLoRA, the first framework to integrate LoRA fine-tuning with zero-knowledge proofs (ZKPs), achieving provable security and correctness. VeriLoRA employs advanced cryptographic techniques---such as lookup arguments, sumcheck protocols, and polynomial commitments---to verify both arithmetic and non-arithmetic operations in Transformer-based architectures. The framework provides end-to-end verifiability for forward propagation, backward propagation, and parameter updates during LoRA fine-tuning, while safeguarding the privacy of model parameters and training data. Leveraging GPU-based implementations, VeriLoRA demonstrates practicality and efficiency through experimental validation on open-source LLMs like LLaMA, scaling up to 13 billion parameters. By combining parameter-efficient fine-tuning with ZKPs, VeriLoRA bridges a critical gap, enabling secure and trustworthy deployment of LLMs in sensitive or untrusted environments.

View More Papers

CAT: Can Trust be Predicted with Context-Awareness in Dynamic...

Jie Wang (State Key Laboratory of Integrated Services Networks, School of Cyber Engineering, Xidian University), Zheng Yan (State Key Laboratory of Integrated Services Networks, School of Cyber Engineering, Xidian University and Hangzhou Institute of Technology, Xidian University), Jiahe Lan (State Key Laboratory of Integrated Services Networks, School of Cyber Engineering, Xidian University), Xuyan Li (Hangzhou…

Read More

Automated Code Annotation with LLMs for Establishing TEE Boundaries

Varun Gadey (University of Würzburg), Melanie Melanie Gotz (University of Würzburg), Christoph Sendner (University of Würzburg), Sampo Sovio (Huawei Technologies), Alexandra Dmitrienko (University of Wuerzburg)

Read More

Validity Is Not Enough: Uncovering the Security Pitfall in...

Di Zhai (Beijing Jiaotong University), Jiashuo Zhang (Peking University), Jianbo Gao (Beijing Jiaotong University), Tianhao Liu (Beijing Jiaotong University), Tao Zhang (Beijing Jiaotong University), Jian Wang (Beijing Jiaotong University), Jiqiang Liu (Beijing Jiaotong University)

Read More