llama.cpp officially supports WebGPU, with inference memory usage on the browser side dropping by over 30%

ME AI According to Beating Monitoring, the official WebGPU backend for llama.cpp and ggml has been officially released, enabling GGUF-format large models to run directly in the browser with local GPU acceleration. The new backend breaks free from reliance on specific native clients or complex WebAssembly architectures, achieving privacy-preserving, on-device inference with no data leaving the device—opening up a zero-configuration entry point to local compute power for the web ecosystem.

A related paper published on May 20 states that the WebGPU backend introduces static memory planning and an efficient model-loading mechanism, reducing runtime GPU memory overhead on the web by 29% to 33% compared with existing frameworks. On mainstream GPU devices such as Intel, Apple, and NVIDIA, average decoding throughput improves by 45% to 69%.

The web demo is based on the open-source library wllama. Recent underlying optimizations completed at the foundation level have achieved better GPU memory control than described in the paper. llama.cpp can also be natively compiled via Google’s C++ WebGPU implementation Dawn, providing a benchmarking baseline for performance comparisons between Vulkan and WebGPU.

(Source: BlockBeats)