SpecMD

Apple researchers unveil SpecMD for faster MoE model inference

Apple's machine learning research team has published a paper detailing SpecMD, a new framework for evaluating Mixture-of-Experts (MoE) model caching policies. Their experiments show that traditional caching assumptions …