LoRA (Low-Rank Adaptation)
Parameter-efficient fine-tuning technique freezing base weights and inserting trainable low-rank decomposition matrices.
Mechanism & Definition
A PEFT method that freezes foundational LLM layer weight matrices $W_0 \in \mathbb{R}^{d \times k}$ and injects trainable rank decomposition matrices $A$ and $B$ such that $\Delta W = B A$. Reduces trainable parameters by up to 99.9% while retaining full adaptation performance.