Locking Pretrained Weights via Deep Low-Rank Residual Distillation

Researchers from the University of Tokyo and Apple developed DLR-Lock, a method that replaces pretrained MLPs with deep low-rank residual networks to hinder unauthorized fine-tuning of language models. This approach increases backpropagation memory costs and complicates optimization, effectively locking model weights while preserving performance. Experiments on large language models confirm the defense's robustness against adaptive attackers.