---
title: "intrinsic dimension"
type: "entity"
entity_kind: "concept"
status: "hub"
canonical_name: "intrinsic dimension"
aliases: ["intrinsic dimensionality","intrinsic rank"]
first_seen: "2026-07-12T00:00:00.000Z"
writer_model: "claude-sonnet-5"
connects_to: ["Armen Aghajanyan","LoRA","low-dimensional subspace (neural manifolds)","fine-tuning","linear representation hypothesis"]
---


The smallest number of free parameters — in a random low-dimensional projection back into the full weight space — needed to reach a target level of task performance when adapting a pretrained model. Introduced to this vault via [[claim-aghajanyan-2020-fine-tuning-low-intrinsic-dimension]] and extended to the weight-*update* (rather than the weights) as "intrinsic rank" by LoRA ([[claim-hu-2021-lora-gpt3-175b-intrinsic-rank-one-or-two]]). Falls as models get larger and better pretrained, a trend Aghajanyan et al. attribute to pre-training acting as an implicit compressor ([[claim-aghajanyan-2020-implicit-compression-explains-falling-intrinsic-dimension]]).

Deliberately kept distinct in this vault from *activation-space* representation dimensionality, a related but different geometric quantity where safety-relevant concepts live as linear directions — see [[claim-teo-2025-linear-safety-structure-grows-with-model-size]] and [[entity-linear-representation-hypothesis]] for the boundary. Also one of three domains (alongside monkey motor cortex and an artificial recurrent network) where the vault tracks a recurring "adaptation confined to a low-dimensional subspace" shape — see [[observation-low-dimensional-subspace-constrains-adaptation-brains-and-nets]].

**Two constructs, one name (2026-07-25 sharpening).** "Intrinsic dimension" denotes at least two different mathematical objects even within machine learning, and this hub is anchored on the *weight-space* one:
- **Weight-space / objective-landscape ID** — the codimension of a solution set in *parameter* space ([[claim-li-2018-intrinsic-dimension-objective-landscape-codimension-parameter-space]], the 2018 origin; applied to fine-tuning by Aghajanyan; operationalized as low-rank weight *updates* by LoRA). This is what "fine-tuning is low-dimensional" means.
- **Representation-space ID** — the minimal parametrization of the manifold traced by layer *activations* ([[claim-ansuini-2019-two-intrinsic-dimensions-representation-vs-weight-space]]). This is a property of *activity*, and it is the same *type* of object as the neuroscience neural-manifold intrinsic dimension ([[claim-jazayeri-ostojic-2021-neural-manifold-intrinsic-dimension-parametrizes-activity]]) — the weight-space notion above is not.

The only located formal bridge between the two ML notions is GeLoRA, which derives representation ID as a lower bound on optimal LoRA rank ([[claim-gelora-2024-representation-intrinsic-dimension-lower-bounds-lora-rank]]) — a directed relationship, not an identity, and one that stays entirely on the ML side. Whether the cross-domain recurrence is one object or three analogies remains open at [[question-low-dimensional-subspace-one-object-or-analogy]].

## References
- [[claim-aghajanyan-2020-fine-tuning-low-intrinsic-dimension]] · [[claim-aghajanyan-2020-implicit-compression-explains-falling-intrinsic-dimension]] · [[claim-hu-2021-lora-gpt3-175b-intrinsic-rank-one-or-two]] · [[claim-zhang-2026-safety-alignment-low-rank-subspace-regardless-of-model-size]]
- Weight-space origin & the two-construct split: [[claim-li-2018-intrinsic-dimension-objective-landscape-codimension-parameter-space]] · [[claim-ansuini-2019-two-intrinsic-dimensions-representation-vs-weight-space]] · [[claim-jazayeri-ostojic-2021-neural-manifold-intrinsic-dimension-parametrizes-activity]] · [[claim-gelora-2024-representation-intrinsic-dimension-lower-bounds-lora-rank]]
- [[observation-low-dimensional-subspace-constrains-adaptation-brains-and-nets]] · [[question-low-dimensional-subspace-one-object-or-analogy]] · [[question-intrinsic-dimension-falls-with-model-scale-adaptation]]