Qwen 3.6 model hits 110 tokens/sec on consumer GPUs via llama.cpp

The open-weight model Qwen 3.6, in its 35 billion parameter version, has achieved an impressive 110 tokens per second inference speed on consumer GPUs with 12GB of VRAM. This performance was enabled by a specialized variant of llama.cpp, referred to as ik_llama.cpp, and specific quantization techniques. Additionally, a 27 billion parameter version of Qwen 3.6 has been successfully deployed locally using llama.cpp's server configuration, providing a practical example for self-hosted AI applications. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Accelerates the accessibility and practicality of running powerful LLMs on local hardware, reducing reliance on cloud services.

RANK_REASON The cluster details benchmark results and practical deployment examples for open-weight models on consumer hardware, focusing on performance optimizations. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• Claude Code
• consumer GPUs
• GitHub Copilot
• ik_llama.cpp
• llama.cpp
• Qwen 3.6
• Pi

• model release
• infra