Perplexity Open-Source Inference Base PPLX-Garden, Bypassing NVIDIA Network Tax for Multi-GPU Ultra-Fast Communication

ME AI News: according to Data Observation Beating monitoring, the search engine giant Perplexity AI has officially open-sourced a high-performance inference infrastructure toolkit for production use, pplx-garden. The project’s core is a self-developed Rust high-performance point-to-point communication library, fabric-lib (also known as TransferEngine), designed to break the hardware lock-in imposed by NVIDIA’s proprietary exclusive communication protocol. It helps developers achieve ultra-fast operation of trillion-parameter large models on heterogeneous multi-GPU clusters without having to purchase expensive dedicated network switches.

Traditional distributed large-model inference relies heavily on NVIDIA’s exclusive high-speed communication networks, which results in extremely high hardware deployment costs and supply-chain lock-in. fabric-lib achieves hardware-level decoupling: it not only perfectly adapts to NVIDIA ConnectX-7 network cards, but also natively supports Amazon’s inexpensive AWS EFA standard Ethernet cards, pushing network bandwidth between multiple cards directly up to 400 Gbps. To address the physical shortcomings of out-of-order transmission in AWS EFA, Perplexity pioneered an ImmCounter counter-synchronization mechanism, enabling efficient “zero-copy” data flow without requiring any hard assumptions about packet order. The communication library includes a data-distribution algorithm specifically designed for mixture-of-experts (MoE) models, overlapping GPU data reception with matrix computations to greatly maximize the available compute capacity in the decoding phase.

In actual production, the engineering benefits brought by pplx-garden are extremely significant. In a decoupled inference architecture, the network library enables rapid scheduling of key-value cache between the Prefill node and the Decoder node. In asynchronous reinforcement learning training, it takes only 1.3 秒 to complete weight synchronization and distribution for trillion-parameter models. To solve the computation latency during the tokenization stage, pplx-garden also provides an open-source tokenizer, pplx-unigram, rewritten in Rust, reducing CPU usage by 5 to 6 times and eliminating performance bottlenecks caused by reordering and vector models during tokenization.

(Source: BlockBeats)