Google releases the eighth-generation TPU, with training and inference now first separated into two independent chips

ME News Update, April 22 (UTC+8): According to Beating monitoring, Google CEO Sundar Pichai unveiled the eighth-generation TPU at Cloud Next 2026, marking the first time that training and inference are split into two separate chips. TPU 8t is designed for training. A single supernode can connect to 9,600 TPUs, providing 121 ExaFlops of computing power and 2PB of shared high-bandwidth memory. Its processing performance is 3 times that of the previous Ironwood generation, with energy efficiency improving by up to 2 times. Inter-chip interconnect bandwidth has doubled, and together with the newly launched Virgo network topology, up to 1 million chips can be configured into a single logical cluster, enabling near-linear scaling. Google says its goal is to shorten the development cycle of frontier models from several months to a few weeks.

TPU 8i is designed for inference. A single pod connects to 1,152 TPUs and is equipped with 288GB of high-bandwidth memory and 384MB of on-chip SRAM—the latter is 3 times that of Ironwood—so that active model data can be kept on the chip as much as possible. The newly introduced Boardfly network topology significantly reduces latency. Google claims that at the same cost, it can serve nearly 2 times the number of customers, aiming to support millions of agents running simultaneously.

Both chips are hosted on Google’s in-house Arm-based Axion CPU architecture, paired with fourth-generation liquid cooling. They are planned to be officially supplied later in 2026 on the Google Cloud AI Hypercomputer platform, alongside NVIDIA GPU instances. (Source: BlockBeats)