Google releases the eighth-generation TPU, with training and inference now separated into two independent chips

ME News message. On April 22 (UTC+8), according to Beating Monitoring, Google CEO Sundar Pichai unveiled the eighth-generation TPU at Cloud Next 2026, for the first time splitting training and inference into two separate chips. TPU 8t is designed for training. A single super node can connect to 9,600 TPUs, providing 121 ExaFlops of computing power and 2PB of shared high-bandwidth memory; its processing performance is 3 times that of the previous Ironwood generation, with energy efficiency improved by up to 2 times. Inter-chip interconnect bandwidth has doubled, and together with the newly launched Virgo network topology, it can form a single logical cluster with up to 1,000,000 chips, enabling near-linear scaling. Google says the goal is to shorten the development cycle of cutting-edge models from several months to a few weeks.

TPU 8i is designed for inference. A single pod connects to 1,152 TPUs, equipped with 288GB of high-bandwidth memory and 384MB of on-chip SRAM, which is 3 times that of Ironwood, to keep active model data on the chip as much as possible. The newly introduced Boardfly network topology significantly reduces latency. Google claims that at the same cost, it can serve nearly 2 times the number of customers, aiming to support millions of agents running simultaneously.

Both chips are hosted on Google’s self-developed Arm architecture Axion CPU, paired with fourth-generation liquid cooling and heat dissipation. It is planned to be officially supplied later in 2026 on the Google Cloud AI Hypercomputer platform, alongside NVIDIA GPU instances. (Source: BlockBeats)