Zyphra AI has released ZAYA1-8B, a Mixture of Experts (MoE) language model with 760 million active parameters and 8.4 billion total parameters. Trained on AMD hardware, this model demonstrates competitive performance against larger models on math and coding benchmarks, utilizing innovations like Compressed Convolutional Attention and an MLP-based router. ZAYA1-8B is available under an Apache 2.0 license and as a serverless endpoint, offering efficient deployment for on-device applications and lower latency inference. AI
Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →
IMPACT Offers a more efficient alternative for reasoning tasks, potentially lowering inference costs and enabling on-device LLM applications.
RANK_REASON Release of a new open-weight language model with novel architecture and training infrastructure. [lever_c_demoted from research: ic=1 ai=1.0]
