Every dollar in AI right now is chasing the same thing: a bigger model, in a bigger data centre, drawing more power. I think the next trillion is going the other way.
I'm Usamah Zaheer, and I work on ML inference optimisation at Arm. My job is making models run inside power budgets that would be a rounding error in a data centre. From down here, the industry's obsession with scale looks less like the destination and more like phase one.
Cloud AI is phase one, not the end state
The AI industry is currently being built around scarcity. GPUs are scarce. Inference is expensive. Power is expensive. Running a frontier model takes infrastructure that maybe a dozen organisations on earth can finance. So naturally, almost all the attention and almost all the capital flows to whoever is building the largest clusters: NVIDIA, the hyperscalers, and the energy, networking and memory companies feeding them.
That's real. I'm not arguing it isn't.
But technology gets genuinely interesting at the moment something expensive becomes cheap and ubiquitous. Computers did it. Bandwidth did it. Storage did it. Cameras did it. An image sensor went from being a product to being a component you drop into anything for a few dollars, and that unlocked far more value than the camera industry ever captured for itself.
So the question isn't whether inference gets cheaper. It will. The question is what happens when useful intelligence costs approximately nothing, and I don't think that's priced in anywhere.
The constraint is the product
There are workloads where a round trip to the cloud is simply not on the menu.
Latency you can't negotiate with. A robot can't wait on an API call to decide whether it's about to hit someone. A car can't ship camera frames to a data centre before deciding whether to brake. Physics sets the deadline, not your SLA.
Cost that scales the wrong way. A million security cameras streaming continuous video upstream on the off chance something interesting happens is an absurd way to spend money. For the overwhelming majority of those frames, nothing interesting happens.
Connectivity you can't assume. Factory basements, tunnels, farms, ships, operating theatres. Any system whose intelligence evaporates when the network does isn't intelligent, it's a thin client with good marketing.
So a huge share of inference moves to where the data is generated. And when it does, the optimisation problem inverts. The winner stops being whoever trains the biggest model and starts being whoever can take useful intelligence and fit it inside brutal constraints. Ten watts. Two watts. Five hundred milliwatts. Two megabytes of SRAM. No network. Milliseconds of budget.
That is a completely different engineering problem, and having spent years on that side of it, I'd say it's badly underestimated. The gap between what works in a notebook and what runs on a 4-watt chip is where most of the actual work lives.
The model is one layer of a very tall stack
People discuss models as though the model is the product. It isn't. A model has to run somewhere, and between a PyTorch graph and a piece of silicon sits an enormous amount of machinery: quantization, compilers, runtimes, kernels, memory planning, accelerators, system software, power management, packaging.
Every percentage point anywhere in that stack compounds once you multiply it across billions of devices. Make a model 2x smaller and it matters. Cut memory bandwidth by 30% and it matters. Move an operator from CPU to NPU at a fraction of the power and it matters. Build a compiler that makes an entire class of models viable on cheaper silicon and you haven't won an incremental improvement. You've caused a market to exist that didn't before.
Which is why I suspect some of the most valuable AI companies of the next decade will look boring next to the model labs. Runtimes. Compilers. Inference infrastructure. Specialised silicon. Memory systems. Developer tooling. Almost nobody outside the field will ever learn their names, and everything else will sit on top of them.
Robotics makes the argument for me
Robotics is where this stops being arguable.
A useful general-purpose robot runs perception, speech, planning, localisation and control continuously. Not on request, not when a user taps something. You can't route every one of those decisions through a metered cloud model forever and still have a business. I spent nearly two years putting vision-language models on real robots at Dyson, and the binding constraint was never model quality. It was latency, power, and what happens the moment the network isn't there.
Now assume humanoids, autonomous vehicles, industrial robots and drones actually scale. That's hundreds of millions of machines each running local inference all day, every day. The same logic already reached phones. Apple, Qualcomm, MediaTek and Arm are all building silicon around on-device AI, and it won't stop there. Eventually we'll stop saying "AI chip" the way we stopped saying "maths coprocessor." It'll just be a processor.
Good enough, cheap and instant beats smart
Here's the part I think gets missed most often.
The world does not need every task solved by a trillion-parameter model. If a 500M parameter model does the job locally at essentially zero marginal cost, that is frequently worth more commercially than access to the most intelligent model on earth. Good enough plus cheap plus instant has won an enormous number of markets, and it's going to win a lot of these. It's the same instinct behind the cost-aware routing we built into agent systems: match the compute to the complexity, and stop paying frontier prices for trivial work.
Right now we treat inference as precious. I think it ends up looking like electricity: barely worth thinking about, available everywhere, noticed only when it's absent. Your glasses understand what you're looking at. Your car understands the road. Your camera understands what it's seeing. Almost all of it happening silently, continuously, locally, at a daily volume of operations that's genuinely hard to picture. And every one of those operations runs through somebody's hardware and somebody's software.
That's the opportunity.
The bottleneck always moves
Every technology cycle has a bottleneck, and the bottleneck is where the value concentrates. Today it's access to high-end compute, which is why NVIDIA has captured what it has.
Bottlenecks move. As models get more efficient and silicon catches up, the binding constraint shifts from training to deployment: how do you fit a model onto constrained hardware, make it fast enough, run it without destroying battery life, and ship it across hundreds of different hardware targets without rewriting it every time? Whoever answers those questions owns a genuinely valuable part of the stack.
Usamah Zaheer works on ML inference optimisation at Arm, on a software stack that reaches billions of devices, after deploying perception models on robots at Dyson and researching CNNs for satellite imagery at the University of Leicester. The pattern across all of it is the same: the hard engineering, and increasingly the value, shows up the moment you leave the cloud behind.
Not all of this becomes a standalone business. Plenty of it gets commoditised, or absorbed into a larger platform, or quietly disappears. But somewhere in that stack are the companies that end up foundational to how computing works, the way Arm became foundational to mobile without most people ever noticing.
The first stage of AI was: who can build the smartest model? The second is: who can build enough compute to run them? The third, I think, is: who can make intelligence cheap enough to put everywhere? Because the end state probably isn't a handful of enormous models sitting in a handful of enormous buildings. It's billions of machines quietly running intelligence all around us. And whoever makes that possible captures a great deal of what AI creates.
That's the race I care about. And it's the one I'm building my career inside.