New AdamO optimizer enhances stability and performance in offline RL

Researchers have introduced AdamO, a novel optimizer designed to enhance stability in offline reinforcement learning. This new optimizer addresses the issue of 'collapse,' where errors in temporal-difference updates can lead to extreme and unusable Q-values. AdamO incorporates orthogonality constraints to prevent the amplification of TD errors, theoretically guaranteeing task safety while maintaining the continuous-time dissipative dynamics of Adam. Empirical results show that AdamO improves stability and performance across various offline RL benchmarks when integrated with existing baselines. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Introduces a new optimizer that improves stability and performance in offline reinforcement learning tasks.

RANK_REASON This is a research paper detailing a new optimization technique for a specific machine learning problem. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• AdamO
• arXiv
• offline reinforcement learning
• TD updates
• Adam
• Q-values

• paper
• other