π Step 9: AI/LLM#305 / 350
Low-Rank Adaptation
Low-Rank Adaptation (LoRA)
πOne-line summary
An efficient fine-tuning method that adjusts only a small set of low-rank matrices instead of all parameters.
π‘Easy explanation
A fine-tuning trick that adjusts only a tiny fraction of the model's parameters. It needs so little GPU memory that even individuals can do it.
β¨Example
μ 체 vs LoRA νμΈνλ
μ 체 νμ΅
LoRA β¨
π© = μ‘°μ μμ (μ 체μ ~1%)