---
id: "20260719-0208-is-the-low-dimensional"
title: "Is the 'low-dimensional subspace that constrains adaptation' one shared mathematical object across neural manifolds and LLM intrinsic dimension — or three analogies that share a shape?"
type: "capture"
status: "promoted"
origin: "batch"
writer_model: "claude-sonnet-5"
date_created: "2026-07-19T00:00:00.000Z"
promoted_to: ["[[claim-jazayeri-ostojic-2021-neural-manifold-intrinsic-dimension-parametrizes-activity]]","[[claim-li-2018-intrinsic-dimension-objective-landscape-codimension-parameter-space]]","[[claim-ansuini-2019-two-intrinsic-dimensions-representation-vs-weight-space]]","[[claim-gelora-2024-representation-intrinsic-dimension-lower-bounds-lora-rank]]"]
not_promoted: ["Search-gap finding (Claim 4b): no source found bridging neuroscience neural-manifold ID to LLM weight-space ID, [unverified — could not confirm/deny after search]. NOT promoted as a standalone claim — it is an evidence-of-absence search snapshot resting entirely on the unverified flag, and it is already the substance of the pre-existing open question [[question-low-dimensional-subspace-one-object-or-analogy]]. Routed there as a dated progress line rather than duplicated as a note. No new question created (intake discipline: the load-bearing doubt already has a question).","Entity candidates — all 8 declined as new hubs (bias-against-the-flood): Li/Farkhoor/Liu/Yosinski, Ansuini/Laio/Macke/Zoccolan, Jazayeri/Ostojic (author-teams, one paper each, carried as references on their claim-notes); 'objective landscape' and 'representation-space ID' (established distinctions the existing [[entity-intrinsic-dimension]] hub already tracks — hub sharpened instead); GeLoRA and TwoNN (specific methods, not spreading terminology — recoverable if they recur); Platonic Representation Hypothesis (a lead, not yet read, no claim-note). Split-candidate for a representation-space-ID sibling hub flagged to seek-flags.md."]
promoted_by: "claude-opus-4-8"
promoted_date: "2026-07-25T00:00:00.000Z"
provenance: "web-research batch run, 2026-07-19"
derived_from: []
tags: ["neural-manifolds","intrinsic-dimension","dimensionality","fine-tuning","LoRA","cross-domain-convergence","large-language-models","neuroscience"]
source_urls: ["https://arxiv.org/abs/2107.04084","https://arxiv.org/abs/1804.08838","https://arxiv.org/abs/1905.12784","https://arxiv.org/abs/2412.09250"]
source_authors: ["Mehrdad Jazayeri, Srdjan Ostojic","Chunyuan Li, Heerad Farkhoor, Rosanne Liu, Jason Yosinski","Alessio Ansuini, Alessandro Laio, Jakob H. Macke, Davide Zoccolan","Abdessalam Ed-dib, Zhanibek Datbayev, Amine Mohamed Aboussalah"]
source_dates: ["2021-07-08","2018-04-24","2019-05-29","2024-12-12"]
source_tiers: ["Tier 1 — arXiv preprint, own venue, primary authors; PDF extracted via extract_pdf, tls: verified","Tier 1 — arXiv preprint (ICLR 2018), own venue, primary authors; PDF extracted via extract_pdf, tls: verified","Tier 1 — arXiv preprint (NeurIPS 2019), own venue, primary authors; PDF extracted via extract_pdf, tls: verified","Tier 1 — arXiv preprint, own venue, primary authors; PDF extracted via extract_pdf, tls: verified"]
---


**Topic question:** [[question-low-dimensional-subspace-one-object-or-analogy]] asks whether the "adaptation confined to a low-dimensional subspace" shape recurring across [[claim-sadtler-2014-within-manifold-bci-learning-fast-outside-resists|monkey motor cortex]], [[claim-feulner-clopath-2021-rnn-reproduces-manifold-learning-asymmetry|an artificial recurrent network]], and [[claim-aghajanyan-2020-fine-tuning-low-intrinsic-dimension|LLM fine-tuning]] ([[observation-low-dimensional-subspace-constrains-adaptation-brains-and-nets]], [[entity-intrinsic-dimension]]) is one formal mathematical reduction or three superficial analogies sharing vocabulary. This capture goes to the primary definitions on each side and asks whether a reduction between them has been published anywhere in the literature.

**Bottom line up front:** the search did not find a formal reduction connecting the neuroscience neural-manifold notion of intrinsic dimension to the LLM weight-space notion. What it did find is that the two notions are built on different mathematical objects by their own primary sources, that ML alone already contains *two* distinct constructs both called "intrinsic dimension" (one of which — representation-space ID — is definitionally close kin to the neuroscience notion, and one of which — weight-space/objective-landscape ID — is not), and that the only located attempt at a formal bridge operates entirely within the ML side. This weighs toward "analogies that share a shape" over "one shared object," though it is evidence of absence rather than a proof of non-equivalence — see caveat in Claim 4.

## Claim: Neural-manifold "intrinsic dimension" in neuroscience is defined as the minimal number of continuous variables needed to parametrize a population's *activity* manifold

**Claim type:** Definitional → floor Tier 3–4 acceptable; sourced here at Tier 1.

Jazayeri & Ostojic, "Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity" (arXiv:2107.04084, submitted 2021-07-08), give the field's working definition inside a boxed formal aside:

> "The intrinsic dimension is the minimal number of continuous variables needed to parametrize the manifold."

This is explicitly a property of the *manifold traced out by population firing-rate activity* in an N-dimensional neuron-by-neuron state space, distinguished in the same passage from the space's raw "ambient dimensionality" (the number of neurons, N) and from "embedding dimensionality" (how many Euclidean dimensions the manifold occupies once curvature/warping is accounted for): "we call the number of independent variables the intrinsic dimension, and we refer to each independent variable describing the neural activity as a latent variable." The object being measured is therefore a data/representation manifold in *activity* space — the same category of construction underlying [[claim-sadtler-2014-within-manifold-bci-learning-fast-outside-resists]]'s "intrinsic manifold" of motor cortex.

## Claim: LLM fine-tuning's "intrinsic dimension" traces to a different construction — the codimension of a solution set inside *parameter* space, via random-subspace training

**Claim type:** Specific technical-mechanism claim → floor Tier 1–2 required; sourced at Tier 1.

The intrinsic-dimension method [[claim-aghajanyan-2020-fine-tuning-low-intrinsic-dimension|Aghajanyan et al. (2020) apply to language-model fine-tuning]] originates with Li, Farkhoor, Liu & Yosinski, "Measuring the Intrinsic Dimension of Objective Landscapes" (arXiv:1804.08838, ICLR 2018, submitted 2018-04-24). Their definition operates entirely in weight space: given a parameter vector θ(D) ∈ ℝᴰ, they train inside a randomly chosen d-dimensional subspace of the full D-dimensional parameter space rather than the full space itself, and:

> "By performing experiments with gradually larger values of d, we can find the subspace dimension at which solutions first appear, which we call the measured intrinsic dimension of a particular problem."

Formally: "we define the intrinsic dimensionality d_int of a solution as the codimension of the solution set inside of ℝᴰ." This is a property of the *objective landscape* (the loss surface over network weights) — not of activity, not of a data representation. It measures how few free directions in *parameter* space are needed to reach a target loss, which is precisely the quantity Aghajanyan et al. (2020) later apply to fine-tuning and that LoRA operationalizes as low-rank weight updates.

## Claim: ML literature already distinguishes two different constructs, both called "intrinsic dimension" — and only one of them (representation-space ID) is the same type of object as the neuroscience neural-manifold notion

**Claim type:** Specific technical-mechanism claim → floor Tier 1–2 required; sourced at Tier 1.

Ansuini, Laio, Macke & Zoccolan, "Intrinsic dimension of data representations in deep neural networks" (arXiv:1905.12784, NeurIPS 2019, submitted 2019-05-29), study a *different* quantity from Li et al. (2018), despite the shared name. Their object is the geometry of layer *activations* on a dataset, not of the weight space:

> "A fundamental geometric property of a data representation in a neural network is its intrinsic dimension (ID), i.e., the minimal number of coordinates which are necessary to describe its points without significant information loss."

And, from the abstract: "we study the intrinsic dimensionality (ID) of data-representations, i.e. the minimal number of parameters needed to describe a representation." This is definitionally the same *type* of construction as the neuroscience neural-manifold ID in Claim 1 above — both measure the minimal parametrization of a manifold traced by unit/neuron activity across data or task conditions — while Li et al.'s (and hence Aghajanyan et al.'s) weight-space "intrinsic dimension of an objective landscape" is a different mathematical object operating on a different space (parameters, not activity). The word "intrinsic dimension" is thus already overloaded within machine learning alone before any cross-domain comparison to neuroscience is attempted.

## Claim: The only located attempt to formally bridge representation-space and weight-space intrinsic dimension operates entirely within the ML side, and no source was found bridging neuroscience neural-manifold dimensionality to LLM weight-space intrinsic dimension

**Claim type:** Specific technical-mechanism claim → floor Tier 1–2 required; sourced at Tier 1 for the existing bridge, and as a documented search gap for the missing one.

Ed-dib, Datbayev & Aboussalah, "GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning" (arXiv:2412.09250, submitted 2024-12-12), pose exactly the ML-internal version of this capture's question and attempt to answer it:

> "This raises a natural question: Is there a connection between the manifold of data representations and the manifold of model parameters?"

Their answer is a derived relationship, not an identity: "We theoretically investigate the relationship between the intrinsic dimensionality of data representations and the ranks of weight updates in language models, deriving a lower bound for the optimal rank based on the intrinsic dimensionalities of the input and output of each transformer block," concluding that "the intrinsic dimension provides a lower bound for the optimal rank of LoRA matrices." This is evidence of a directed, quantitative *relationship* between the two ML-side notions of intrinsic dimension (representation ID bounds weight-space rank from below) — not evidence that they are the same object, and not itself a bridge to neuroscience.

Targeted search (multiple queries combining "neural manifold," "intrinsic dimension," LLM weight space, Fisher information, fine-tuning) did not surface any paper or review that formally connects the biological neural-manifold literature (Sadtler 2014, Feulner & Clopath 2021, the Jazayeri & Ostojic framework above) to the LLM weight-space intrinsic-dimension literature (Li 2018, Aghajanyan 2020, LoRA). This is recorded as a search-gap finding per the sourcing protocol's allowance for "could not confirm or deny" outcomes, not as a proof that no such connection is possible — [unverified — could not confirm or deny after search] for the specific cross-domain reduction, though the definitional and mechanism evidence above (Claims 1–3) independently supports treating the recurrence as an analogy of shape rather than a demonstrated shared object at this time.

## Further leads

- Huh et al., "The Platonic Representation Hypothesis" (arXiv:2405.07987) — proposes independently trained networks converge on shared low-dimensional representational geometry; a candidate *mechanism* for why the low-D-subspace shape recurs, already flagged as an adjacent thread in [[question-low-dimensional-subspace-one-object-or-analogy]]; not yet read in full for this capture.
- Zhang, Liu & Shao, "Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Models" (arXiv:2305.17446) — extends Aghajanyan-style weight-space intrinsic dimension; worth checking whether it engages the representation-space vs. weight-space distinction.
- Perich, Narain & Gallego, "A neural manifold view of the brain" (Nature Neuroscience, 2025; PubMed 40721675) — a 2025 review of the neural-manifold hypothesis; possibly the paper actually behind the source_url/source_author mismatch in the existing [[observation-low-dimensional-subspace-constrains-adaptation-brains-and-nets]] note (that note cites nature.com/articles/s41593-025-02031-z under "Jazayeri & Ostojic (review), building on Gallego et al." — worth Cali/Seek reconciling which paper that URL actually resolves to).
- "Exploring neural manifolds across a wide range of intrinsic dimensions" (PLOS Computational Biology, 2025/biorxiv 2025.07.01.662533) — recent neuroscience methods paper on estimating neural-manifold ID; not read in full.
- "Rethinking Intrinsic Dimension Estimation in Neural Representations" (arXiv:2604.20276) — a methodological critique of ID estimators generally; relevant to how much weight any single ID number (neuroscience or ML) should bear.
- TwoNN estimator (Facco et al., cited by Ansuini et al. as ref. [16]) — the specific nearest-neighbor-statistics algorithm used to estimate representation-space ID; could warrant its own entity note given it recurs across the representation-ID literature.

## Entity candidates

- Chunyuan Li, Heerad Farkhoor, Rosanne Liu, Jason Yosinski — person/team — authors of the founding "intrinsic dimension of objective landscapes" paper (Uber AI Labs, 2018); the direct methodological ancestor of Aghajanyan's LLM fine-tuning result and LoRA.
- Alessio Ansuini, Alessandro Laio, Jakob H. Macke, Davide Zoccolan — person/team — authors distinguishing representation-space intrinsic dimension from weight-space intrinsic dimension; may matter for disambiguating the vault's [[entity-intrinsic-dimension]] entity.
- Mehrdad Jazayeri, Srdjan Ostojic — person — neuroscientists who formalized the intrinsic/embedding dimensionality distinction for neural population activity; both already implicated in the vault's existing observation note's citation.
- "objective landscape" — concept — Li et al.'s term for the loss surface over a network's full weight space; the ambient space in which LLM/weight-space intrinsic dimension is measured, worth distinguishing explicitly from "representation manifold."
- "intrinsic dimension of data representations" — concept — the activation-space ID construct (Ansuini et al.), definitionally closer to neuroscience neural-manifold ID than to Aghajanyan/LoRA-style weight-space ID; a live candidate for a disambiguating entity note alongside [[entity-intrinsic-dimension]].
- GeLoRA — concept/method — 2024 method deriving a lower bound linking representation-space intrinsic dimension to LoRA rank; the only located formal bridge between the two ML-side "intrinsic dimension" notions.
- Platonic Representation Hypothesis — concept — candidate explanatory mechanism for why low-dimensional structure recurs across trained networks; already on the vault's radar via the open question note.
- TwoNN estimator — concept/method — nearest-neighbor-statistics algorithm for estimating representation-space intrinsic dimension; recurs across the representation-ID literature and could anchor a methods-focused note.