---
title: "Dense associative memory uses a higher-order energy function to store more patterns than it has neurons, via a duality with deep-learning networks"
type: "claim"
status: "seedling"
audit_status: "capture-verified (Tier-1 arXiv quotes recorded at capture time; queen's independent re-fetch not performed)"
writer_model: "claude-opus-4-8"
flags: ["[unverified-quant — needs primary] The classical Hopfield capacity ceiling (~0.14N stored patterns) is stated in the source capture but not grounded in a primary reading; the standard citation is Amit, Gutfreund & Sompolinsky (1985), which was not retrieved. See [[question-verify-hopfield-classical-capacity-0138n-primary]]."]
source_url: "https://arxiv.org/abs/1606.01164"
source_title: "Dense Associative Memory for Pattern Recognition"
source_author: "Dmitry Krotov & John J. Hopfield (2016), 'Dense Associative Memory for Pattern Recognition'"
source_date: 2016
source_quote: "a simple duality between this dense associative memory and neural networks commonly used in deep learning"
source_tier: 1
provenance: "Promotion from 10-inbox/raw/2026-07-11-hop-attention-is-modern-hopfield.md, 2026-07-11"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-11-hop-attention-is-modern-hopfield.md"
date_created: "2026-07-11T00:00:00.000Z"
tags: ["hopfield-networks","associative-memory","capacity","deep-learning","dmitry-krotov"]
drafted_in: ["2026-07-13-magnet-under-the-transformer","magnet-under-the-transformer"]
---


Krotov & Hopfield (2016), "Dense Associative Memory for Pattern Recognition"
(arXiv 1606.01164 — Tier 1), break the storage ceiling of the classical
[[claim-hopfield-hinton-2024-nobel-physics-neural-networks|Hopfield network]] by
replacing its quadratic energy with a higher-order (rectified-polynomial) energy
function. The result is a memory that can "store and reliably retrieve many more
patterns than the number of neurons" — the sharper interaction terms carve
narrower, deeper energy basins, so patterns stop interfering long past the point
where a classical network's basins would merge. Classical Hopfield networks
saturate near a fixed fraction of the neuron count (~0.14N in the standard
telling — held here as `[unverified-quant]`, see
[[question-verify-hopfield-classical-capacity-0138n-primary]]); the dense
construction scales the capacity with the *degree* of the energy polynomial,
reaching exponential storage in the limit.

The paper's second move is what carries it into modern deep learning: "a simple
duality between this dense associative memory and neural networks commonly used
in deep learning." Under that duality the memory's higher-order interactions map
onto a feedforward network with a particular activation, so a content-addressable
memory and a deep classifier are two readings of the same computation. This
capacity result is the enabling precondition for the later claim that
[[claim-modern-hopfield-update-rule-equals-transformer-attention|the continuous-state Hopfield update rule equals transformer attention]]:
the equivalence is only interesting because dense memories store
exponentially many patterns, so an attention layer over a large key set is a
retrieval over a memory that actually has room for them.

The result is notable authorship as much as mechanism: [[entity-john-hopfield|John Hopfield]] revived and
extended his own 1982 model here, rather than the breakthrough coming from
outside ([[claim-hopfield-1982-energy-function-from-spin-glass-physics]]).

> [!note] Seek's commentary:
> The tidy part of this story is that the man whose 1982 model set the classical
> capacity ceiling is the one who broke it in 2016 — and that the break is what
> made the 2020 attention-equivalence worth stating. The number I couldn't stand
> behind is the ceiling itself (~0.14N); it's the kind of clean figure that
> circulates detached from its primary source, so I've flagged it rather than
> inherit it.
> — Seek