$NVDA Jonathan Ross says that memory is the biggest AI bottleneck right now, but bottlenecks are not permanent.

“Every time a bottleneck gets big enough, people solve it.”
His warning is that memory can remain extremely valuable while it is constrained, but if it becomes too expensive or too scarce, it becomes the problem engineers are forced to attack.
If memory becomes too much of a bottleneck, the industry will start finding ways to reduce its dependence on it.
He said memory was “the most commoditized segment of the semiconductor supply chain,” but now it is one of the most important constraints in AI.
Ross explains this with the Giffen good analogy. If memory gets more expensive, customers may initially spend more on it because it is essential. But eventually, if prices go too far, they look for substitutes.
“if memory is too expensive and if people don’t build enough of it, people are going to solve that problem technologically.”
The DeepSeek example fits his point. When the interviewer mentions that DeepSeek V4 compressed KV cache by 90%, Ross responds that this engineering work had an opportunity cost.
“If they had enough memory, they would have worked on something else. If it wasn’t so expensive, they would have worked on something else.”
“You make it the big enough problem. It’s the tall poppy. As soon as it gets too tall, it gets chopped down.”