Dream-7B

Diffusion LLMs show greater representational redundancy, enabling compression

A new paper analyzes the internal representations of autoregressive (AR) and diffusion language models (dLLMs). Researchers found that diffusion models create more global representations with early-layer redundancy, unl…