SGD+Momentum

New principle optimizes AI model training by aligning gradients and updates

Researchers have introduced a new principle called Greedy Alignment for selecting and tuning optimizer hyperparameters in machine learning. This principle treats optimizers as causal filters that map gradients to update…