NVIDIA's $20 billion acquisition of Groq marks its first strategic discussion: reasoning tokens should be priced based on quality, with low latency and high unit price being the new race track

ME News Report, April 16 (UTC+8), according to Beating Monitoring, Jensen Huang first explained in detail the strategic logic behind NVIDIA’s acquisition of Groq during an interview. NVIDIA acquired Groq’s inference chip business for $20 billion in December last year, with Groq founder Jonathan Ross and core team joining NVIDIA, and Groq continuing to operate as an independent company. At the GTC conference in March this year, NVIDIA announced the first chip after the merger, Groq 3 LPU, manufactured with Samsung’s 4nm process, which NVIDIA claims has a trillion-parameter model inference throughput per megawatt that is 35 times that of Blackwell NVL72. Jensen Huang said that the motivation for acquiring Groq was the stratification of the inference market. Previously, inference optimization had only one direction: increasing throughput. But the commercial value of tokens has risen significantly, and different users are willing to pay different prices for different response speeds. “If I can provide software engineers with faster response tokens, making them more efficient than now, I am willing to pay for it. But this market only appeared recently.” He described this as an expansion of the Pareto frontier in the inference market: beyond existing high-throughput solutions, adding a new market segment characterized by low latency and high unit price. For the same model, differentiated pricing based on response time, “although throughput is lower, the higher unit price can compensate.” Groq’s LPU architecture is known for deterministic low latency, complementing NVIDIA’s high-throughput GPU approach, and the acquisition fills a gap in NVIDIA’s inference product line. (Source: BlockBeats)