In the 1990s–2000s neural networks were an out-of-favor 'backwater,' displaced by SVMs, until the mid-2000s 'deep learning' revival

Timothy B. Lee, writing in Understanding AI (2024), characterizes the status of neural-network research in the late 1990s and early 2000s bluntly:

"neural networks had become a backwater"

By this account the field had lost prestige to kernel methods — support-vector machines and related approaches — which dominated machine-learning publication and grant competition in that period. Neural-network work was relegated to minor venues and reviewer aversion, a status distinct from outright refutation: the models still worked in principle, but had fallen out of the field's fashionable center. Lee's phrase is a named journalist's Tier-2 recollection of the field's social standing, not a metric.

This is a second stigma episode, separated by two decades from the one the vault already documents. The 1970s–80s contraction was driven by the unsolved hidden-layer training problem and by broad funding shocks — the Lighthill Report's combinatorial-explosion diagnosis (claim-lighthill-1973-blamed-combinatorial-explosion), ARPA money moving to symbolic AI (claim-sri-neural-group-retooled-for-arpa-funding), and the Perceptrons-book folk history (myth-perceptrons-book-killed-connectionism). The 1990s–2000s "backwater" is a different low: connectionism had a working training method (backprop) by then and was simply out-competed for attention by SVMs. The survival-through-industry strategy the vault tracks for the first winter (claim-nestor-inc-commercialized-neural-nets-through-ai-winter) has its analogue here in the small academic groups (Hinton, Bengio, LeCun) that kept the paradigm alive until the mid-2000s revival and the "deep learning" relabeling (claim-deep-learning-term-predates-hinton).

The out-of-favor status is the load-bearing premise for reading the subsequent rebrand as a dissociation from a stigmatized past (observation-deep-learning-rebrand-as-field-scale-fresh-start). See moc-backpropagation-origins.