What's next for AI?
@ajassy thinks the biggest wave is just getting started in the workloads companies run every day, and bringing down the cost of inference is a big part of unlocking it.
@ajassy thinks the biggest wave is just getting started in the workloads companies run every day, and bringing down the cost of inference is a big part of unlocking it.
AI demand is a barbell, with the labs on one end and enterprise cost-cutting on the other. Amazon CEO Andy Jassy says the biggest market sits in the middle, and inference is still "an order of magnitude too expensive" to reach it.
"On one end of the barbell are the AI labs who are consuming gobs and gobs of compute, unlike anything I've seen. And then a few runaway apps, ChatGPT, Claude Code, things like that."
"On the other end of the barbell are enterprises who are largely using AI for cost avoidance and productivity, so automating customer service or fraud or business process automation."
"In the middle of the barbell are the current production workloads of enterprises, many of which are using AWS and using inference today, but I would say the overwhelming majority of which still aren't in any pervasive way."
"And that I think is gonna be the largest absolute segment of AI usage."
"The cost of inference has come down quite a bit in the last few years, but it's still an order of magnitude too expensive, in my opinion, for it to be used as pervasively as possible."
"If you're asking enterprises to rewrite their applications that use CPU today with AI accelerators... the amount of incremental benefit they have to get is a much higher bar to want to rewrite those apps. But if you can take the cost of inference down, that bar is much lower, and all applications are gonna be better and smarter and more capable."
@ajassyView quoted post on X
