Neural tangent kernel
PulseAugur coverage of Neural tangent kernel — every cluster mentioning Neural tangent kernel across labs, papers, and developer communities, ranked by signal.
- 2026-05-13 research_milestone Publication of a paper introducing a force-aware Neural Tangent Kernel for active learning of MLIPs. source
6 day(s) with sentiment data
-
New method analyzes generalization in nonlinear least-squares models
Researchers have developed a new method to understand how nonlinear least-squares models generalize. Their approach uses on-average algorithmic stability to derive error bounds for local minimizers. These bounds are lin…
-
Deep Neural Networks Achieve Optimal Generalization Rates
Two new papers submitted to arXiv analyze the generalization performance of gradient descent methods in deep neural networks. The research establishes minimax-optimal rates for excess population risk in deep ReLU networ…
-
NTK theory extended to neural network classification
Researchers have extended the Neural Tangent Kernel (NTK) theory to classification tasks, previously a limitation to regression losses. They identified conditions, including parameter-space regularization or non-degener…
-
New theory explains neural network training speed
Researchers have developed a new theoretical framework to better understand the optimization dynamics of over-parameterized neural networks. This framework, centered around the Neural Tangent Kernel (NTK), introduces co…
-
New optimization technique boosts accuracy for complex physics neural networks
Researchers have developed a new optimization technique called SOAP+GN to improve the accuracy of physics-informed neural networks (PINNs) when dealing with complex, coupled multiphysics systems. This method addresses a…
-
New method enhances neural network uncertainty estimation
Researchers have developed a new method to improve Bayesian Last Layers (BLLs) for estimating uncertainty in neural networks. Their approach leverages a projection of Neural Tangent Kernel (NTK) features to account for …
-
GLU structures accelerate LLM optimization by reshaping NTK spectrum
Researchers have investigated why Gated Linear Units (GLU) are superior to non-GLU structures in large language models. Their analysis in the neural tangent kernel regime indicates that GLU reshapes the NTK spectrum, re…
-
New Math Framework Explains Transformer Training Dynamics
A new paper introduces a mathematical framework for understanding how Transformers train, particularly in the mean-field regime where both depth and width approach infinity. Unlike ResNets which can be modeled by ODEs, …
-
New framework enables scalable, robust active learning for MLIPs
Researchers have developed a new active learning framework for machine-learning interatomic potentials (MLIPs) that addresses scalability and robustness challenges. This framework utilizes a force-aware Neural Tangent K…
-
Paper explores preconditioned gradient descent's impact on neural network learning regimes
This paper investigates how preconditioned gradient descent (PGD) methods, like Gauss-Newton, influence spectral bias and the phenomenon of grokking in neural networks. Researchers propose that PGD can mitigate spectral…
-
New research explains why Zeroth-Order Optimization scales to LLMs
Two new papers explore zeroth-order (ZO) optimization for fine-tuning large language models (LLMs). The first paper introduces a kernel perspective, showing that the approximation error depends on output size rather tha…
-
New theories explore how pre-training and sparse connectivity enhance deep learning generalization
Three new papers explore the theoretical underpinnings of generalization in deep learning. One paper identifies pre-training as a critical factor for weak-to-strong generalization, demonstrating its emergence through a …