Something that I expect will become much more obvious over the next few generations of computer hardware is that the square cube law applies to computation as well.
You can pack a lot of memory and compute into a volume, but the ability to move energy, heat, and data in and out of it is proportional to its surface area, and those scale at very different rates.
Much of the reason why AI inference compute is so easy to scale is precisely because it aligns with this well; a byte loaded in for a weight in a matrix can be reused thousands of times. Compute scales better than bandwidth.
Eventually you do hit a limit on energy and thermals, so compute only scales up to a point as well, but there's a large constant factor gap between flows of constant power or high-entropy heat and flows of organized, coordinated data.
Historically, the algorithms people use are skewed heavily toward quasi-linear algorithms, with superlinear ones being considered "slow". That doesn't mean they're useless though. We've likely under-explored and neglected such algorithms heavily.
If you have a chip where ops/s of compute exceeds bytes/s of bandwidth by multiple orders of magnitude, then that's a huge speedup for traditionally "slow" algorithms. Many of these algorithms will likely become much more valuable. Matrix multiplication alone is an obvious example, being O(n鲁).