SA-DPO

Researchers propose structure-aware consistency for LLM preference learning

Researchers have identified a theoretical inconsistency in popular preference learning methods like Direct Preference Optimization (DPO) used for aligning Large Language Models (LLMs). The study proposes a new framework…