---
title: "LoRA (Low-Rank Adaptation)"
type: "entity"
entity_kind: "term"
status: "hub"
canonical_name: "LoRA"
aliases: ["Low-Rank Adaptation"]
first_seen: "2026-07-12T00:00:00.000Z"
writer_model: "claude-sonnet-5"
connects_to: ["intrinsic dimension","Armen Aghajanyan","fine-tuning","parameter-efficient adaptation","GPT-3"]
---


Parameter-efficient fine-tuning technique (Hu et al., arXiv:2106.09685, 2021) that freezes a pretrained model's weights and injects trainable low-rank decomposition matrices into each layer instead of updating the full weight matrix. Built directly on [[entity-armen-aghajanyan]]'s intrinsic-dimension finding — the paper's own framing is that if adaptation lives on a low-dimensional subspace, you can parameterize the update on that subspace directly rather than discover it implicitly. Demonstrated at GPT-3 175B scale that the intrinsic rank of a fine-tuning update can be as low as 1 or 2, extending the falling-intrinsic-dimension trend far past the BERT/RoBERTa-scale models Aghajanyan et al. originally tested.

## References
- [[claim-hu-2021-lora-gpt3-175b-intrinsic-rank-one-or-two]] · [[claim-aghajanyan-2020-fine-tuning-low-intrinsic-dimension]]
- [[question-intrinsic-dimension-falls-with-model-scale-adaptation]]