After a 1,000,000x AI Compute Leap: What Jeff Dean Sees Next

Jeff Dean helped found Google Brain, and today he's a research director at Google DeepMind. In a recent talk, he took on a claim that sounds like pure marketing but turns out to have real technical teeth: AI computing power has grown a million-fold over the past decade. Not doubled. Not a hundred times. A million. So the question becomes — where did all that come from, and where does it go next?

Let's start with what's surprising. You might assume this is just Moore's Law doing its thing — transistors doubling every couple of years. But Moore's Law only accounts for maybe a hundred-fold of improvement across this whole period. That leaves a factor of ten thousand coming from somewhere else. And that "somewhere else" is really three stories running in parallel.

The first is hardware specialization. Here's the core problem: a general-purpose CPU is genuinely bad at matrix multiplication, which is the thing deep learning does constantly. GPUs, with their massively parallel cores, gave you a ten-to-a-hundred-fold speedup right away. But GPUs were originally built for graphics — they're general-purpose accelerators wearing a different hat. So starting around 2016, Google designed TPUs, chips built specifically for the matrix math that neural networks live on. More aggressive optimization, and a lot better energy efficiency than GPUs for that particular job.

The second story is distributed training systems. When you train a modern large language model, you're not using one chip or ten — you might be coordinating thousands, even tens of thousands of accelerators at the same time. And that's a brutal engineering problem. How do you split the model across all those chips — do you slice it by layers, or slice the math inside each layer? How do you keep everyone's gradients in sync when they all need to share updates? And critically, how do you make sure that when one node fails — and at that scale, something is always failing — the entire multi-week training run doesn't just die? Google's answer here is systems like Pathways and the Jax and XLA compiler stack.

The third story is architecture efficiency — the models themselves getting smarter about how they use compute. The Transformer, for instance, is far more parallelizable than the older recurrent networks it replaced, so it maps onto all that hardware much better. Then there are tricks like Flash Attention, which reorganizes how the attention mechanism reads and writes memory, so you can train on much longer sequences without spending more compute. And mixed-precision training — using smaller number formats like sixteen-bit floats — lets you pack more parameters into the same memory.

Now here's the key insight: these three tracks multiply together. Roughly a hundred-fold from Moore's Law, hundreds-to-thousands-fold from specialized hardware, another few hundred from software and architecture. Stack those on top of each other and you land in million-fold territory. That's the compounding. Training a large model today happens at a completely different efficiency level than it did ten years ago — not because of any single breakthrough, but because three curves bent upward at once.

But Dean's talk isn't really a victory lap about big numbers. What he's actually excited about is what this scale of compute unlocks — scientific problems that were flatly impossible before and are now becoming tractable.

The clearest example is protein structure prediction. AlphaFold2 cracked the shape of proteins, and that was huge. But Dean points past it to the harder problems waiting on the other side: protein dynamics — not just the final folded shape, but the actual pathway of how it folds — how proteins interact with small molecules, and designing entirely new proteins from scratch. Every one of those needs even more compute than AlphaFold did.

Then there's climate modeling. Earth's climate is this massive tangle of coupled physical equations, and traditional supercomputer models are limited in resolution by how much compute you can throw at them. AI models like Google's GraphCast can now run higher-resolution forecasts in less time — and on a lot of accuracy measures, they've started beating the classic numerical methods. And in medicine and genomics: predicting disease risk from someone's genome, predicting treatment outcomes from health records. These are problems where the scale of compute you can bring directly sets the accuracy you can reach.

So where does it go from here? Dean flags a shift in mindset. The old model was: train one enormous model, then spend a fixed amount of compute every time you use it. The new idea is to allocate compute dynamically, based on how hard the actual problem is.

One direction is Mixture of Experts. Picture a model with lots of specialized sub-networks — experts — but for any given piece of input, only a small handful actually switch on. So the total knowledge stored in the model can be gigantic, while the compute you spend on each token stays modest. You scale up what the model knows without scaling up what it costs to run.

The other direction is giving models thinking time. Instead of blurting out an answer in one pass, you let the model spend more reasoning steps on genuinely hard problems — chains of thought, searching through possibilities. This is exactly the territory that OpenAI's o1 and o3, and Google's Gemini Thinking models, are exploring right now.

Now, if you're an engineer building AI applications, there's a quieter message tucked inside all this. The democratization of compute lags way behind the frontier. The scale the big labs are playing with today won't reach an ordinary developer for another three to five years. And the flip side of that is genuinely useful to internalize: applications you'd build today will cost dramatically less to run in a few years. Things that feel too expensive to even attempt right now are going to quietly become viable. Meanwhile, because compute is scarce, the craft of getting better results from less of it stays valuable — quantization, distillation, fine-tuning small models for specific jobs. That work isn't going out of style anytime soon.

So let me leave you with three things to hold onto. First: the million-fold isn't hype — it's the real compounded product of three separate tracks, hardware, software, and model architecture, all multiplying together. Second: the payoff isn't bragging rights, it's scientific problems — protein dynamics, climate, genomics — that simply weren't reachable before. And third: the next phase isn't just about building bigger, it's about spending compute more intelligently — routing it to the experts and the problems that actually need it. And the reason Dean's read on this is worth trusting is simple. He didn't just watch this happen. He helped design the TPUs, he built the software stacks, he worked on AlphaFold. When he predicts where this goes next, he's describing the things he's been building all along.