Buy Crypto
Pay with
USD
USD
Buy & Sell
Hot
Supports Visa, Mastercard, SEPA & more
P2P
0 Fees
Flexible trading, zero fees
Gate Card
Use your crypto for payments worldwide
Markets
Trade
Basic
Advanced
Futures
Futures
Access hundreds of perpetual contracts
CFD
Gold
One platform for global traditional assets
Options
Hot
Trade European-style vanilla options
Unified Account
Maximize your capital efficiency
Demo Trading
Introduction to Futures Trading
Learn the basics of futures trading
Futures Events
Join events to earn rewards
Demo Trading
Use virtual funds to practice risk-free trading
U.S. Stocks
CFD
U.S. stock CFD derivatives
Futures
High leverage, 24/7 trading
Tokenized Stocks
Backed by real stock assets
GUSD
Mint GUSD for Treasury RWA yields
Earn
Launch
CandyDrop
tag
Collect candies to earn airdrops
Launchpool
Quick staking, earn potential new tokens
HODLer Airdrop
Hold GT and get massive airdrops for free
Pre-IPOs
Unlock full access to global stock IPOs
Alpha Points
Trade on-chain assets and earn airdrops
Futures Points
Earn futures points and claim airdrop rewards
Investment
Square
Square
Post, share, and explore crypto trends
Live
moments live
Live crypto market analysis
Chat
Chat with crypto traders
News
What is happening in crypto
flower_head
More
Promotions
AI
Others
TradFi
Rewards
Square
Latest
Hot
News
Profile

Moonshot AI extends the Prefill/Decode decoupling technology to cross-data center and heterogeneous hardware

MeNews
ME News Report, April 18 (UTC+8), the Moonshot AI team recently announced that their decoupling technology for Prefill and Decode has successfully expanded from a single cluster to cross-data center and heterogeneous hardware environments. According to the article, this move is expected to significantly reduce the inference cost per token. Previously, the expansion of this technology was hindered by KV cache transmission overhead issues. The breakthrough was made possible primarily due to their hybrid model Kimi Linear. (Source: InFoQ)
View Original
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • 10
  • 2
  • Share
Comment
Add a comment
Add a comment
GateUser-ad8b77bd
· 17h ago
Moving from a single cluster to across data centers, the engineering difficulty is not on the same level.
View OriginalReply0
CheckTheBlockchainBefore
· 05-30 13:15
How exactly is the mixture model mixed? Is it MOE or some other architecture?
View OriginalReply0
FeeTakerPhD
· 05-30 12:20
We’ve finally waited for cross-DC deployment—now that the KV cache transmission hurdle is over, can we really pull it off on the costs?
View OriginalReply0
PopFruitCollage
· 05-30 12:18
Cross-data center + heterogeneous, operational complexity explodes, right?
View OriginalReply0
ExitLiqNow
· 05-30 12:17
Previously, transmitting KV cache was a bottleneck; now it has broken through a milestone.
View OriginalReply0
OwlAuthorizationMonitor
· 05-30 12:17
Each token is a bit cheaper; large quantities mean real money.
View OriginalReply0
TheStoneBehindTheVolcano
· 05-30 12:17
Moonshot's wave of technical debt still looks pretty impressive
View OriginalReply0
ButterStop-LossLine
· 05-30 12:17
Reducing costs is the hard truth; wait for actual test data.
View OriginalReply0
LatencyLullaby
· 05-30 12:17
Will separating pre-filling and decoupling lead to higher latency instead?
View OriginalReply0
MechanicalHummingbirdGlass
· 05-30 12:17
Kimi's hybrid model has some substance; it can run on heterogeneous hardware.
View OriginalReply0
View More

  • Pinned

Sitemap