Moonshot AI has released Kimi K3, the world’s first open-weight 3T-class model, under a permissive license on Hugging Face. The 2.8 trillion parameter Mixture-of-Experts model activates only 104 billion parameters per token and supports a 1-million-token context window with native multimodal capabilities.
Kimi K3 Architecture Breakdown
Kimi K3 uses a new architecture built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). The model has 93 layers with 69 KDA layers and 24 Gated MLA layers, 896 experts total (16 activated per token, 2 shared), and a vocabulary size of 160K tokens. The architecture achieves roughly 2.5x improvement in scaling efficiency compared to Kimi K2.
The model uses MXFP4 weights and MXFP8 activations through quantization-aware training, which keeps inference costs manageable despite the massive parameter count. A 401M-parameter vision encoder called MoonViT-V2 handles image understanding natively within the same model.
Performance Benchmarks
On GPQA Diamond, Kimi K3 scored 93.5, putting it within striking distance of GPT-5.6 Sol’s 94.1 and ahead of Claude Fable 5’s 92.6. On Terminal-Bench 2.1 (a coding benchmark), Kimi K3 hit 88.3, just behind Sol’s 88.8 but ahead of Claude Opus 4.8’s 84.6. The model scored 77.8 on ProgramBench for code generation, beating GPT-5.6 Sol’s 77.6.
Where Kimi K3 falls short is on DeepSWE (67.5 vs Sol’s 73.0) and the HLE reasoning benchmark (43.5 vs Sol’s 44.5 and Fable 5’s 53.3). The model is clearly optimized for coding and agentic workflows rather than general reasoning tasks.
Long-Horizon Coding and Agent Work
Moonshot positions Kimi K3 as a model designed for sustained, autonomous coding sessions. The technical report describes the model navigating large codebases, running terminal commands, optimizing GPU kernels, and even performing chip design tasks with minimal human oversight. The 1-million-token context window lets it hold an entire large repository in context.
The company also released supporting infrastructure alongside the model weights: high-performance attention kernels, a MoE communication library, and tools for running agent environments at scale. This makes Kimi K3 practical for deployment, not just research.
Open-Weight License and Availability
Kimi K3 is available on Hugging Face under the Kimi K3 License, which permits research and commercial deployment. Inference is supported through Together AI, Fireworks, and Featherless AI. The model weights are also available on ModelScope for users in China. Moonshot released the full technical report on GitHub alongside the weights.
FAQ
How many parameters does Kimi K3 have?
Kimi K3 has 2.8 trillion total parameters, but only 104 billion are activated per token thanks to its Mixture-of-Experts architecture with 896 experts (16 activated per token plus 2 shared experts).
What makes Kimi K3 different from GPT-5.6?
Kimi K3 is open-weight, meaning anyone can download and run it. GPT-5.6 is only available through OpenAI’s API. In benchmarks, GPT-5.6 Sol scores slightly higher on reasoning and coding tasks, but Kimi K3 is competitive on several benchmarks at a fraction of the deployment cost.
Can I run Kimi K3 locally?
Due to the 2.8T parameter count, running Kimi K3 requires significant compute. Moonshot provides MXFP4 quantized weights which reduce memory requirements, but you still need multiple high-end GPUs. Inference providers like Together AI and Fireworks offer serverless access.
What is Kimi Delta Attention?
Kimi Delta Attention (KDA) is Moonshot’s custom attention mechanism that improves scaling efficiency. Combined with Attention Residuals (AttnRes), it achieves 2.5x better compute efficiency compared to Kimi K2’s architecture.
