"Former Google TPU Architect: The Real Bottleneck of AI Isn't Computing Power"

In this two-hour interview, Reiner Pope step-by-step explained the physics behind training and inference on the blackboard. His insights are crucial for understanding the AI industry chain—especially chips, memory, and interconnected devices.
But the original text is very complex, and ordinary readers may find it exhausting to read.
So, without changing any of Reiner's original meaning, I will do two things:
First, rephrase it in plain language.
Second, extract key points from an investment perspective.
The article is divided into three parts: the current situation, the underlying principles, and how it will impact various industries.
1. Summarize in one sentence:
Reiner's core judgment in this lecture is that the real bottleneck of AI is not computing power but the speed of data transfer. This bottleneck cannot be solved in the short term.
If you only want to remember one thing, it’s this sentence. Almost all subsequent industry implications are derived from this.
Why is this important? Because where the money flows in the entire AI industry chain—who profits and who takes the share—depends on "where the bottleneck is."
If the bottleneck is computing power, then GPU manufacturers are the absolute winners;
if the bottleneck is data transfer, then the money will be taken by another group of companies—HBM memory, inter-rack connectivity, cables, switches, liquid cooling, power supplies.
Reiner's clear answer is: the bottleneck is the latter.
This can be directly seen from the capital expenditure structure of major companies—industry estimates suggest that about half of their spending this year is on memory.
2. Computing power is sufficient; what’s missing are "transporters"
To understand why computing power is not lacking but memory is, let’s use an analogy.
Imagine a GPU as a super accountant who is very good at calculations. Given a stack of ledgers (model parameters), he can finish calculations quickly.
The problem is: the ledgers are not in his hand but stored in a warehouse.
Every time he needs to do calculations, someone has to move the ledger from the warehouse to his desk, and after he finishes, put it back.
There are two time factors here:
Calculation time: how fast he can compute
Transport time: how slow it is to move the ledger back and forth
As usual, the article is long, so let’s skip directly to the main points.