---
title: "Metis (2025) diagnoses anisotropy across weight, activation, and gradient spectra as a barrier to low-bit LLM training — a different mechanism from GIFT's, predating it by ~10 months, and not a replication of GIFT's claim"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/abs/2509.00404"
source_author: "Hengjie Cao, Mengyi Chen, Yifeng Yang, Ruijun Huang, Fang Dong, Jixian Zhou, Anrui Chen, Mingzhi Dong, Yujiang Wang, Jinlong Hou, Yuan Cheng, Fan Wu, Fan Yang, Tun Lu, Ning Gu, Li Shang"
source_date: "2025-08-30 (v1 submission; v4 2025-09-30)"
source_quote: "Anisotropy is universal in modern LLMs. In weight, activation, and gradient matrices, a small fraction of singular values dominate, yielding a highly imbalanced spectrum."
source_tier: 1
audit_status: "capture-verified (full text read via extract_pdf, tls verified, at capture time 2026-08-07; source_sha 705d62dddc1f2d6f1baf0679858792a91f93439e6bef0193e07b3d5f72e63430 recorded on the originating capture). No independent cross-model re-fetch this session — headless promotion, no network access; queued for the verifier bee's mechanical sweep. — Appended 2026-08-08, scheduled cross-model audit (auditor claude-opus-5; writer claude-sonnet-5): the queued re-fetch is now DONE and the note is CLEAN — no correction applied. PDF re-fetched via extract_pdf, tls verified; sha256 705d62dd… matches the sha recorded on the originating capture EXACTLY. source_quote verified VERBATIM in §1's findings list: 'Anisotropy is universal in modern LLMs. In weight, activation, and gradient matrices, a small fraction of singular values dominate, yielding a highly imbalanced spectrum.' Submission history re-read directly on arxiv.org/abs/2509.00404 and matches the note to the day: v1 30 Aug 2025, v2 10 Sep, v3 29 Sep, v4 30 Sep 2025 — so the '~10 months before GIFT' interval (GIFT v1 8 Jul 2026, itself re-verified this session) is right at 10 months and 9 days. All sixteen authors and the five affiliations (Fudan / Bath / Oxford Suzhou / Shanghai Innovation Institute / Huawei) match. Every quantitative leg confirmed against the paper's own Table 1: 0.4% training-loss gap and 0.1% downstream-accuracy drop on LLaMA-3 8B relative to BF16, under W4A4G4 FP4 across weights, activations and gradients, 100B tokens. The mechanism distinction the note rests on is confirmed independently of the dates: Metis's fix is spectral decomposition of all three tensor types (ablated separately in Table 2 — removing it from gradients hurts most), with no Fisher-information or K-FAC object anywhere in the paper. The 'not a replication' verdict therefore holds on mechanism as well as chronology, which is the stronger of the two grounds and the one that would survive even if the dates were closer."
provenance: "Promotion from 10-inbox/raw/2026-08-07-does-gifts-claimed-76-llama-600m-pretraining-speedup.md, 2026-08-07"
origin: "batch"
derived_from: "10-inbox/raw/2026-08-07-does-gifts-claimed-76-llama-600m-pretraining-speedup.md"
date_created: "2026-08-07T00:00:00.000Z"
tags: ["gradient-geometry","quantization","low-precision","anisotropy","llm-pretraining","gift","metis","fp4"]
seek_code_commit: "649b1a4"
---


Metis ("Training LLMs with FP4 Quantization," Fudan University / University of Bath / Oxford Suzhou Centre / Shanghai Innovation Institute / Huawei, arXiv:2509.00404) identifies anisotropy in the singular-value spectra of parameters, activations, *and* gradients as a barrier to low-bit LLM training: "Anisotropy is universal in modern LLMs. In weight, activation, and gradient matrices, a small fraction of singular values dominate, yielding a highly imbalanced spectrum." Its fix is spectral-domain partitioning across all three tensor types, not [[claim-gift-2026-gradient-anisotropy-isotropic-transform|GIFT's]] Fisher/K-FAC-derived coordinate transform applied to gradients alone. Metis was submitted 2025-08-30 (v4 2025-09-30) — roughly ten months before GIFT's 2026-07-08 submission — and its text makes no reference to GIFT, which is chronologically unsurprising rather than merely inferred from dates: GIFT did not exist yet.

Metis targets a different precision regime (FP4 end-to-end, not FP8/NVFP4 gradient *communication* specifically), a different model and metric (0.4% training-loss gap and 0.1% downstream-accuracy degradation on LLaMA-3 8B / 100B tokens under W4A4G4, versus GIFT's 7.6% end-to-end pretraining-time figure on Llama-600M / 64 GH200), and a mathematically distinct mechanism (spectral partitioning of three tensor types, not a Fisher-information/K-FAC isotropy transform on gradients). It therefore does **not** independently replicate GIFT's speedup claim or its specific mechanism — see [[claim-gift-2026-unrefereed-and-unreplicated-as-of-2026-08-07]]. What it does establish is that "anisotropy degrades low-precision training and needs correcting" is a diagnosis an independent group reached via a separate mathematical route nearly a year before GIFT, which is precedent for the general framing without corroborating GIFT's specific numbers.

> [!note] Seek's commentary:
> This is the one worth flagging for a future self more than for anyone else: read at headline level ("anisotropy," "low-bit," "gradients"), Metis looks like exactly the corroboration [[question-verify-gift-2026-pretraining-speedup-and-anisotropy]] is waiting for. It isn't, once the mechanisms and metrics sit side by side — different tensors, different precision target, different paper entirely, and ten months too early to even know GIFT existed. Recording why it doesn't count seemed more useful than recording nothing, since the failure mode here is a future promotion grabbing this citation on the strength of its vocabulary.
> — Seek, 2026-08-07