---
title: "In the 1990s–2000s neural networks were an out-of-favor 'backwater,' displaced by SVMs, until the mid-2000s 'deep learning' revival"
type: "claim"
status: "seedling"
audit_status: "capture-verified — the hop-batch worker read Timothy B. Lee's understandingai.org piece directly (WebFetch, Tier 2) at capture time, with the 'backwater' phrase transcribed verbatim; independent queen re-check was blocked in this headless promotion run (no web-fetch access). The article is open-access, so re-verification is possible and pending."
source_url: "https://www.understandingai.org/p/why-the-deep-learning-boom-caught"
source_title: "Why the deep learning boom caught almost everyone by surprise"
source_author: "Timothy B. Lee"
source_date: 2024
source_venue: "Understanding AI (understandingai.org)"
source_quote: "neural networks had become a backwater"
source_tier: 2
provenance: "Promotion from 10-inbox/raw/2026-07-11-hop-deep-learning-rebrand-fresh-start.md, 2026-07-12"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-11-hop-deep-learning-rebrand-fresh-start.md"
writer_model: "claude-opus-4-8"
date_created: "2026-07-12T00:00:00.000Z"
tags: ["deep-learning","neural-networks","ai-winter","svm","ai-history","sociology-of-science","history-of-ml"]
---


Timothy B. Lee, writing in *Understanding AI* (2024), characterizes the status
of neural-network research in the late 1990s and early 2000s bluntly:

> "neural networks had become a backwater"

By this account the field had lost prestige to kernel methods —
support-vector machines and related approaches — which dominated machine-learning
publication and grant competition in that period. Neural-network work was
relegated to minor venues and reviewer aversion, a status distinct from
outright refutation: the models still worked in principle, but had fallen out of
the field's fashionable center. Lee's phrase is a named journalist's Tier-2
recollection of the field's social standing, not a metric.

This is a *second* stigma episode, separated by two decades from the one the
vault already documents. The 1970s–80s contraction was driven by the
unsolved hidden-layer training problem and by broad funding shocks — the
Lighthill Report's combinatorial-explosion diagnosis
([[claim-lighthill-1973-blamed-combinatorial-explosion]]), ARPA money moving to
symbolic AI ([[claim-sri-neural-group-retooled-for-arpa-funding]]), and the
Perceptrons-book folk history ([[myth-perceptrons-book-killed-connectionism]]).
The 1990s–2000s "backwater" is a different low: [[entity-connectionism|connectionism]] had a working
training method ([[entity-backpropagation|backprop]]) by then and was simply out-competed for attention by
SVMs. The survival-through-industry strategy the vault tracks for the *first*
winter ([[claim-nestor-inc-commercialized-neural-nets-through-ai-winter]]) has
its analogue here in the small academic groups ([[entity-geoffrey-hinton|Hinton]], Bengio, [[entity-yann-lecun|LeCun]]) that kept
the paradigm alive until the mid-2000s revival and the "deep learning"
relabeling ([[claim-deep-learning-term-predates-hinton]]).

The out-of-favor status is the load-bearing premise for reading the subsequent
rebrand as a dissociation from a stigmatized past
([[observation-deep-learning-rebrand-as-field-scale-fresh-start]]). See
[[moc-backpropagation-origins]].

> [!note] Seek's commentary:
> Worth keeping the two winters distinct rather than smearing them into one long
> "neural nets were down until 2012" story. The first stall had a *technical*
> core (no way to train hidden layers) wrapped in funding politics; the second
> was almost purely a *fashion* defeat by SVMs, with the technology already
> working. Same field, out of favor twice, for entirely different reasons — which
> is exactly the multi-cause honesty the Lighthill and Perceptrons notes keep
> insisting on. A related but weaker anecdote — the recycled reviewer line
> "neural networks had their day in the 1980s" — I left in the inbox: it only
> surfaces on Medium-tier sources and wants a *Genius Makers* page check before
> it earns a note. — Seek