Run the Kimi K3 Architecture Locally with a 0.18B Test Model
A 0.18B-parameter Kimi K3 derivative that preserves the core layer types in a roughly 700 MB checkpoint for local architecture testing and development.
What you get from it
Kimi-K3-0.18B is a tiny, MIT-licensed derivative of Moonshot AI’s 2.8T-parameter Kimi K3. It reduces the model to four layers, eight experts and 0.18B total parameters, with roughly 0.10B activated per token. The single model.safetensors checkpoint is about 0.69 GB, making the architecture accessible on ordinary development hardware.
What it preserves
The miniature model retains Kimi K3’s main layer types: dense and MoE feed-forward blocks, KDA linear attention, and an MLA full-attention layer. That makes it useful for testing loaders, integrations, architecture support and compression workflows without downloading or serving the full 2.8T model.
Best use cases
- Validate Transformers compatibility and custom Kimi K3 model code
- Prototype inference, quantization or conversion workflows
- Inspect the MoE and attention implementation on limited hardware
- Build development tests around the Kimi K3 architecture
Loading caveat
The example loader requires trust_remote_code=True, which executes repository-provided Python code. Review the model repository and pin a revision before running it in a sensitive environment. The model is not currently hosted by a Hugging Face inference provider.
Discussion
Share practical experience, questions, or warnings with the community.
Sign in to join the discussion and vote on comments.
Sign in