Christoph Reisinger

Paper details uniform scaling limits in AdamW-trained transformers

Researchers have published a paper detailing uniform scaling limits in transformers trained with the AdamW optimizer. The study models hidden-state dynamics as an interacting particle system, demonstrating convergence t…