Kimi has figured out how to decouple Prefill/Decode this time, which can still save inference costs across data centers. There's some value in that.

ME News message. On April 18 (UTC+8), the Moonshot AI team recently announced that its decoupling technology for Prefill and Decode has successfully expanded from a single cluster to cross-data-center and heterogeneous hardware environments. According to the article, this is expected to significantly reduce inference costs per token. Previously, the expansion of this technology was hindered by KV cache transmission overhead issues. The key to this breakthrough lies in its hybrid model Kimi