Many people are worried that storage will end up like the fiber optics of 2000, because excessive infrastructure building leads to eventual blowups. But I think storage and fiber optics still have several fundamental physical differences:

1) The “linearity” of Maxwell’s equations means that one optical path can be reused across multiple frequencies without interfering; therefore, the bandwidth of a single fiber can be infinitely large.
2) Even if the same frequency is used, fiber optics can still improve communication efficiency through purely mathematical methods like modulation and coding.
But the problem with HBM now is completely not “efficiency,” but capacity. A Rubin GPU with 500GB of VRAM needs HBM chips equal to 2x the number of 250GB Blackwell—that’s an unchangeable mathematical fact.
Besides, for every additional bit in VRAM, it requires another piece of physical space, which also wastes another wafer. This is different from what exists in the virtual world: as long as energy conservation is satisfied, you can create freely—so the additional photons are different.
Therefore, likening storage to fiber optics isn’t appropriate. Even if it will eventually become “excessive,” the difficulty of that excess and the time it takes to arrive will be far stronger than, and later than, the infrastructure bubble of the internet in 2000.
Even if we develop “byte reuse” for storage or context compression algorithms to reduce demand, this would again involve Jevons’ paradox.
Hard!