all tags

deep-learning

34 notes carry this tag — 1 moc, 22 claims, 1 observation, 3 questions, 7 captures. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

• A critical period that means less than it shows — a robust deep-network effect whose biological and information-theoretic readings both dissolve on a primary read
  topic map 2026-08-23 · critical-periods, deep-learning, information-bottleneck, information-plasticity, fisher-information
• Verify the origin of the term 'deep learning' against primaries: Dechter (1986), Aizenberg et al. (2000), and the Hinton-era popularization (c. 2006)
  capture promoted Tier 1 2026-07-31 · deep-learning, terminology, priority-dispute, ai-history, history-of-ml
• Rina Dechter's 1986 AAAI paper genuinely uses 'deep learning' and 'shallow learning' as terms of art — for depth of constraint-satisfaction search, not neural networks
  claim seedling Tier 1 2026-07-31 · deep-learning, terminology, dechter, ai-history, history-of-ml
• Neither of Hinton's two canonical 2006 papers — the Neural Computation deep-belief-net paper or the Science autoencoder paper — contains the phrase 'deep learning' anywhere in its text
  claim seedling Tier 1 2026-07-31 · deep-learning, terminology, hinton, ai-history, history-of-ml
• Aggregate validation loss follows a smooth power law while individual downstream tasks scale diversely — averaging obscures the per-task signal (Neural Neural Scaling Laws, 2026)
  claim seedling Tier 1 2026-07-25 · neural-scaling-laws, scaling-laws, averaging-artifact, deep-learning, downstream-tasks
• The founding deep-network critical-period paper's Fisher-Information signal does not correlate with the Information Bottleneck's compression signal
  claim seedling Tier 1 2026-07-20 · information-bottleneck, information-plasticity, critical-periods, fisher-information, tishby
• The measured 'compression' in Information-Bottleneck experiments may be a binning-estimator artifact, not a real change in mutual information
  claim seedling Tier 1 2026-07-20 · information-bottleneck, compression-phase, mutual-information, measurement-artifact, deep-learning
• The Information Bottleneck's compression phase is nonlinearity-dependent, not a universal law of deep-network training
  claim seedling 2026-07-20 · information-bottleneck, tishby, saxe, compression-phase, deep-learning
• Does the Information Bottleneck ('learning is forgetting') actually explain deep-network critical periods, and is its compression phase even universal?
  capture promoted 2026-07-19 · information-bottleneck, critical-periods, tishby, saxe, information-plasticity
• The authors of the founding deep-network critical-period paper explicitly declined to claim their network is a valid model of neurobiological information processing
  claim seedling Tier 1 2026-07-18 · critical-periods, deep-learning, neuroscience, epistemics, backpropagation-gap
• The founding deep-network critical-period paper validated its timing against animal data only, and never attempted a human match
  claim seedling Tier 1 2026-07-18 · critical-periods, deep-learning, amblyopia, neuroscience, loop-closure
• As of mid-2026 no deep-network critical-period model has been shown to quantitatively match a human critical-period window — the biology→DL→clinic loop remains open
  claim seedling Tier 1 2026-07-18 · critical-periods, deep-learning, amblyopia, neuroscience, loop-closure
• Has a deep-network critical-period model ever predicted or matched the timing of a human critical period (e.g. amblyopia)?
  capture promoted 2026-07-14 · critical-periods, deep-learning, amblyopia, neuroscience, loop-closure
• Barlow Twins (2021) trains a self-supervised encoder by driving the cross-correlation matrix of two views toward the identity, minimizing redundancy between embedding components
  claim seedling Tier 1 2026-07-12 · self-supervised-learning, representation-learning, barlow-twins, redundancy-reduction, efficient-coding
• Birdal, Lou, Guibas & Şimşekli (2021) bound neural-network generalization error by the persistent-homology dimension of the training trajectory
  claim seedling Tier 1 2026-07-12 · persistent-homology, topological-data-analysis, generalization, deep-learning, neural-networks
• A deep network's critical period is a property of learning dynamics, not biological machinery — it tracks a rise-then-fall of information Achille calls a loss of 'Information Plasticity'
  claim seedling Tier 1 2026-07-12 · critical-periods, information-plasticity, fisher-information, information-bottleneck, deep-learning
• The term 'deep learning' predates Hinton: introduced to machine learning by Rina Dechter (1986) and applied to neural networks by Aizenberg et al. (2000); Hinton popularized it c. 2006
  claim seedling Tier 4 2026-07-12 · deep-learning, terminology, priority-dispute, ai-history, misattribution
• Deep neural networks exhibit critical learning periods — a temporary input deficit early in training can permanently cap the final skill, timed by onset and length like animal critical periods
  claim seedling Tier 1 2026-07-12 · critical-periods, deep-learning, training-dynamics, neuroscience, plasticity
• Persistence-diagram distance between successive neural-network training states correlates with validation accuracy, enabling generalization estimates without a validation set (Gutiérrez-Fandiño et al. 2021)
  claim seedling Tier 1 2026-07-12 · persistent-homology, topological-data-analysis, generalization, deep-learning, validation-set-free
• A neural network can forget along a graceful, human-like power-law curve — not only catastrophically — when feature overlap preserves the trace
  claim seedling Tier 1 2026-07-12 · catastrophic-forgetting, continual-learning, forgetting-curve, memory, neuro-AI-parallel
• In the 1990s–2000s neural networks were an out-of-favor 'backwater,' displaced by SVMs, until the mid-2000s 'deep learning' revival
  claim seedling Tier 2 2026-07-12 · deep-learning, neural-networks, ai-winter, svm, ai-history
• The neural-net→'deep learning' rebrand can be read as a field-scale Fresh Start Effect — dissociation from a stigmatized past self — though the collective-scale extension is unestablished
  observation seedling Tier 2 2026-07-12 · fresh-start-effect, deep-learning, ai-winter, rebranding, cross-domain-analogy
• Does the Information Bottleneck ('learning is forgetting') actually explain deep-network critical periods, and is its compression phase even universal?
  question answered 2026-07-12 · information-bottleneck, critical-periods, tishby, saxe, information-plasticity
• Has a deep-network critical-period model ever predicted or matched the timing of a human critical period (e.g. amblyopia) — closing the biology→DL→clinic loop?
  question open 2026-07-12 · critical-periods, amblyopia, deep-learning, loop-closure, clinical-neuroscience
• Verify the origin of the term 'deep learning' against primaries: Dechter (1986), Aizenberg et al. (2000), and the Hinton-era popularization (c. 2006)
  question open 2026-07-12 · deep-learning, terminology, priority-dispute, ai-history, verification
• 'Deep learning' as a field-scale Fresh Start Effect: the rebrand that dissociated AI from its perceptron-stigmatized past self
  capture promoted Tier 2 2026-07-11 · fresh-start-effect, deep-learning, ai-winter, rebranding, cross-domain-analogy
• Deep neural networks have critical learning periods like Hubel–Wiesel's kittens — an early input deficit becomes permanent
  capture promoted Tier 1 2026-07-11 · critical-periods, hubel-wiesel, deep-learning, neuroscience, information-plasticity
• Dense associative memory uses a higher-order energy function to store more patterns than it has neurons, via a duality with deep-learning networks
  claim seedling Tier 1 2026-07-11 · hopfield-networks, associative-memory, capacity, deep-learning, dmitry-krotov
• The update rule of a modern (continuous) Hopfield network is equivalent to the attention mechanism of transformers
  claim seedling Tier 1 2026-07-11 · hopfield-networks, associative-memory, transformer-attention, deep-learning, cross-domain-bridge
• Sitzmann et al.'s 2020 SIREN paper represents images, video, sound, and PDE solutions as continuous differentiable functions parameterized by neural networks with periodic activations, not as discrete grids of samples
  claim seedling Tier 1 2026-07-09 · implicit-neural-representations, siren, deep-learning, signal-processing, neural-rendering
• The "Ivakhnenko 1965 = first deep learning" characterization is Schmidhuber's, hedged in his own peer-reviewed text, and rests on GMDH's layer-wise regression — not gradient training
  claim seedling Tier 2 2026-07-07 · ivakhnenko, gmdh, deep-learning, schmidhuber, history-of-ml
• The vanishing gradient problem is a chain-rule pathology — early layers stop learning as gradient magnitudes diverge across depth
  claim budding Tier 4 2026-07-06 · vanishing-gradient, backpropagation, chain-rule, deep-learning, training-pathology
• Did Kunihiko Fukushima's 1979 Neocognitron constitute the first multilayer convolutional neural network, and what do primary sources actually claim?
  capture promoted 2026-07-03 · fukushima, neocognitron, convolutional-neural-network, cnn, deep-learning
• What is the vanishing gradient problem and which paper first characterized it rigorously?
  capture promoted 2026-07-01 · deep-learning, backpropagation, training-dynamics, neural-networks, history-of-ml