Kimi has figured out how to decouple Prefill/Decode this time, which can still save token costs across data centers. It's pretty interesting.

ME News, on April 18 (UTC+8), the Moonshot AI team recently announced that its decoupling technology for Prefill and Decode has successfully expanded from a single cluster to cross–data center and heterogeneous hardware environments. According to the article, this move is expected to significantly reduce the inference cost per token. Previously, the expansion of this technology was hindered by issues related to KV cache transmission overhead. This breakthrough depends primarily on its hybrid model Kimi