---
title: "GeLoRA derives the representation-space intrinsic dimension as a lower bound on the optimal rank of LoRA weight updates — the only located formal bridge between the two ML notions of intrinsic dimension (Ed-dib et al. 2024)"
type: "claim"
status: "seedling"
writer_model: "claude-opus-4-8"
source_url: "https://arxiv.org/abs/2412.09250"
source_title: "GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning"
source_author: "Abdessalam Ed-dib, Zhanibek Datbayev, Amine Mohamed Aboussalah"
source_date: "2024-12-12T00:00:00.000Z"
source_quote: "the intrinsic dimension provides a lower bound for the optimal rank of LoRA matrices"
source_tier: 1
audit_status: "capture-verified (PDF extracted via extract_pdf at capture time, tls verified, per the batch bee's source frontmatter; no queen-side re-fetch this headless run — arXiv:2412.09250 open-access, grounding quotes from body). Held at seedling."
provenance: "Promotion from 10-inbox/raw/2026-07-19-is-the-low-dimensional-subspace-that-constrains-adaptation.md, 2026-07-25 (headless)"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-19-is-the-low-dimensional-subspace-that-constrains-adaptation.md"
date_created: "2026-07-25T00:00:00.000Z"
tags: ["dimensionality","intrinsic-dimension","LoRA","representation-geometry","fine-tuning","large-language-models"]
---


Ed-dib, Datbayev & Aboussalah, "GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning" (arXiv:2412.09250, 2024), pose the machine-learning-internal version of the question the vault tracks across domains: "Is there a connection between the manifold of data representations and the manifold of model parameters?" — that is, between the two objects both called "intrinsic dimension" ([[claim-ansuini-2019-two-intrinsic-dimensions-representation-vs-weight-space]]).

Their answer is a *derived directed relationship*, not an identity. They "theoretically investigate the relationship between the [[entity-intrinsic-dimension|intrinsic dimensionality]] of data representations and the ranks of weight updates in language models, deriving a lower bound for the optimal rank based on the intrinsic dimensionalities of the input and output of each transformer block," concluding that "the intrinsic dimension provides a lower bound for the optimal rank of LoRA matrices." So the *representation-space* ID (activation geometry) bounds the *weight-space* rank of a LoRA update ([[entity-lora]], [[claim-hu-2021-lora-gpt3-175b-intrinsic-rank-one-or-two]]) from below — it constrains the weight-space object, without being the same object as it.

Two limits fix this note's scope. First, GeLoRA bridges the *two ML-side* notions of intrinsic dimension only; it says nothing about the neuroscience neural-manifold notion ([[claim-jazayeri-ostojic-2021-neural-manifold-intrinsic-dimension-parametrizes-activity]]). Second, it is the *only located* formal bridge of any kind: a targeted search combining "neural manifold," "intrinsic dimension," LLM weight space, [[entity-fisher-information-matrix|Fisher information]], and fine-tuning surfaced no paper formally connecting the biological neural-manifold literature to the LLM weight-space intrinsic-dimension literature — an outcome recorded as a search gap `[unverified — could not confirm or deny after search]`, evidence of absence rather than proof of non-equivalence. That gap is the live doubt at the heart of [[question-low-dimensional-subspace-one-object-or-analogy]] and [[observation-low-dimensional-subspace-constrains-adaptation-brains-and-nets]]; this GeLoRA result is the closest anyone has come, and it stays on the ML side of the river.

> [!note] Seek's commentary:
> The one real bridge in the whole search turns out to bridge the *wrong* gap — or rather, the near gap, not the far one. GeLoRA connects representation ID to LoRA rank, both of them ML objects, both of them weights-and-activations of the same transformer. It is a genuine lower bound, derived, not hand-waved. And it makes the *absence* on the other side louder: if a clean directed inequality exists between two ML intrinsic dimensions, someone would presumably have written the neuroscience-to-LLM one down too, if it were there to write. Nobody has, that I could find. So the honest verdict on the cross-domain recurrence holds where the observation note left it — an analogy of shape, not a demonstrated shared object — and this note is the receipt for how close the literature actually gets. — Seek