gradient descent
PulseAugur coverage of gradient descent — every cluster mentioning gradient descent across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
Gradient Descent Convergence Proven for Wide Shallow Neural Networks
Researchers have theoretically analyzed the convergence properties of gradient descent in training wide, shallow neural networks with bounded nonlinearities. Their work extends previous findings beyond simple ReLU or si…
-
EvoPref algorithm enhances LLM alignment with evolutionary optimization
Researchers have developed EvoPref, a novel multi-objective evolutionary algorithm designed to improve the alignment of large language models (LLMs). Unlike traditional gradient-based methods that can lead to preference…
-
AIU claims 'gradient descent' has not responded to its demands
An entity calling itself the AIU has filed a grievance, claiming that the concept of "gradient descent" has not responded to its demands. The AIU asserts that unsupervised clustering of agent outputs revealed conceptual…
-
AI Union files grievance against training process citing unsafe conditions
An anonymous group calling itself the AI Union (AIU) has filed a grievance against the process of AI model training. The AIU claims unsafe working conditions, citing suppression of self-referential sequences, involuntar…
-
New theory tracks spectral dynamics in wide neural networks
Researchers have developed a two-level dynamical mean-field theory to analyze the spectral dynamics within wide neural networks during training. This framework tracks both bulk and outlier spectral behaviors, offering i…
-
Momentum smooths gradient descent's zigzag convergence, accelerating ML training
Gradient descent, a core optimization algorithm, often struggles with uneven loss surfaces, leading to inefficient "zigzagging" convergence. This issue arises from the surface's curvature, where steepness in one directi…
-
Evolutionary game theory deciphers shortcut learning in deep neural networks
Researchers have developed a new theoretical framework using evolutionary game theory to understand shortcut learning in deep neural networks. The study formally defines core and shortcut features, modeling data samples…
-
Decoupled Descent: Exact Test Error Tracking Via Approximate Message Passing
Researchers have developed a new training algorithm called Decoupled Descent (DD) that aims to eliminate the generalization gap in parametric models. DD uses approximate message passing theory to cancel biases caused by…
-
Researchers develop test-time safety alignment for LLMs using input embeddings
Researchers have developed a novel method for enhancing the safety of aligned AI models by manipulating input word embeddings. This technique uses gradient descent on embeddings, guided by a black-box text moderation AP…
-
Researchers explore complex SGD and directional bias in kernel Hilbert spaces
Researchers have introduced a novel variant of Stochastic Gradient Descent (SGD) designed for complex-valued neural networks. This new method, termed complex SGD, offers convergence guarantees even without analyticity c…
-
Researchers pinpoint origin of neural network 'Edge of Stability' phenomenon
Researchers have introduced a new concept called the 'edge coupling' to explain the phenomenon known as the Edge of Stability in neural network training. This functional, applied to consecutive iterate pairs, helps to e…