Local-running Kimi K3 hardware barrier? Starts with 64 GPUs, 45kW electricity use—players build it themselves and get eliminated

Moonshot AI published Kimi K3 this week, with 2.8 trillion total parameters to become the largest open-weight model in history. The complete weights are expected to be uploaded by July 27. The official deployment guide states that running it on your own requires at least 64 accelerators; for single-device players, there are two options: use the K2 family or call the official API directly.
（Background: Moonshot AI is about to launch Kimi K3! It features 2.8 trillion parameters and a million-long text context, and its performance is close to Anthropic Opus 4.8）
（Background info: Can my computer run AI models locally? CanIRun.ai helps you analyze it quickly）

Table of contents

Toggle

• What is MoE?
• Official threshold: starts at 64 GPUs
• The previous generation could still fit in one computer

Earlier this week, Moonshot AI released Kimi K3, with total parameters reaching 2.8 trillion, the largest open-weight model in history. Just to store these 2.8 trillion parameters on disk, using the official native 4-bit format (MXFP4) would require 1.4TB to 1.5TB of space—before you even start inference. For most people, the barrier is extremely, extremely high.

Moonshot AI says the full open-weight release is expected to be uploaded by July 27, using a Modified-MIT license that allows self-hosting. As of the time of this writing, the full weights and community quantized versions (e.g., GGUF) have not appeared yet.

What is MoE?

Kimi K3 uses the MoE (Mixture of Experts) architecture. In simple terms, the model maintains 896 “expert” subnetworks internally; for each token processed, the system selects only 16 routed experts plus shared expert computation, so it doesn’t compute everything. This is also why the number of parameters actually “active” is only 50 billion to 60 billion.

The model also adopts a new architecture called Kimi Delta Attention (KDA), along with Attention Residuals. According to the official self-evaluation, in million-token long-context settings, decoding speed can be improved by up to 6.3 times. The context window can reach 1 million tokens, and it also natively supports visual input.

Official threshold: starts at 64 GPUs

Moonshot AI’s official deployment guide states that self-hosting K3 requires at least 64 accelerators. The native MXFP4 weights are about 1.5TB, which is 3.7 times the size of DeepSeek-R1 671B (around 407GB at Q4 quantization). If you simply convert the number of GPUs needed to fit the weights:

• H100 (80GB): about 19 GPUs
• H200 (141GB): about 11 GPUs
• B200 (192GB): about 8 GPUs

And this still doesn’t include KV cache, meaning the memory the model uses to remember the dialogue context. The official KV cache requirement for a 1 million-token context hasn’t been published yet, but it will only be higher, not lower.

On pricing: a single H100 80GB chip costs about $25k to $33k (PCIe), while the SXM version is over $35k to $40k. An 8-card H100 server costs over $300k. Based on the official 64-GPU threshold, that equals 8 eight-card servers, meaning hardware procurement alone exceeds $2.4 million, or more than NT$70 million.

In terms of power consumption: each H100 is about 700W, so 64 GPUs would be about 45kW—far beyond the capacity of a typical household power supply, making it a data-center level electricity requirement.

The previous generation could still fit in one computer

Compared with the previous generation K2 family (1 trillion parameters, about 32 billion activated parameters), it’s even more noticeable. The community’s 1.8-bit extreme quantization for K2 requires about 247GB to run fully, and a single 24GB consumer graphics card combined with system memory offloading can make it work. K2.5 needs over 380GB of unified memory, and the Q2 version of K2.6 needs about 350GB.

A previous-generation Mac Studio with 512GB storage could barely run it, while K3 directly pushes the threshold back to data-center level. For retail users who want to use it, just downgrade to the K2 family or call the official API directly.