Table of Contents
In July 2024, DeepMind announced that AlphaProof had solved 4 of 6 problems from that year’s International Mathematical Olympiad (IMO) — including the hardest problem, which only 5 human competitors solved. This wasn’t another LLM benchmark improvement. It was an architectural shift.
TL;DR
AlphaProof couples a Gemini-based language model with AlphaZero reinforcement learning, operating in the Lean formal proof language where every step is machine-verified. It produces complete, checkable proofs — not probable answers. After reaching IMO silver-medal level, its successor AlphaProof Nexus solved 9 open Erdős problems and proved 44 of 492 OEIS conjectures.
What it is
AlphaProof is an automated theorem-proving system built on a “neurosymbolic” architecture: a neural network provides statistical intuition, while a symbolic system (Lean) enforces rigorous verification. Its lineage traces back to AlphaZero — the RL system that mastered chess, Go, and shogi — now applied to mathematics.
Why it matters
Language models have a fundamental problem with math: no self-verification. They generate plausible-looking reasoning steps with no mechanism to catch errors mid-proof. AlphaProof changes this by translating mathematical problems into Lean, where every inference step is automatically checked — it either holds or it doesn’t, with no ambiguity. This gives reinforcement learning a clean reward signal, enabling genuine learning from failure.
How it works
Language model layer: A pre-trained Gemini-based model translates natural-language problems into Lean and generates candidate proof steps.
RL layer: An AlphaZero-based engine searches proof paths. Each Lean step is verified in real time; successful proofs strengthen the language model’s strategy for similar problem structures. Difficulty increases iteratively.
The key property is verifiability: AlphaProof’s output isn’t “probably correct” — it’s a complete proof that anyone can recheck step-by-step in Lean.
IMO 2024 performance
AlphaProof solved 4 of 6 problems: 2 algebra, 1 number theory, 1 geometry (the hardest, solved by only 5 humans). Under IMO scoring (7 points per problem), this equals 28 points — silver-medal level. Note that it took far longer than the 4.5-hour human exam window; accuracy was the goal, not speed.
AlphaProof Nexus
The follow-up system extended the approach to open mathematical problems: 9 of 353 open Erdős conjectures solved, and 44 of 492 OEIS conjectures proved. These are genuine contributions to mathematics, not benchmark numbers.
Comparison with other AI reasoning approaches
| Approach | Verification | Output guarantee |
|---|---|---|
| LLMs (o1, o3) | None — relies on generation quality | None |
| AlphaProof | Lean formal verification | Complete, checkable proof |
| CAS (Mathematica) | Arithmetic verification | Limited to computational problems |
Summary
AlphaProof’s significance isn’t that AI can now do math. It’s that the neurosymbolic architecture — neural intuition + formal verification — gives AI genuine error-correction in structured reasoning. Formal languages describe more than mathematics: software specifications, protocol design, security properties. If this framework transfers, the implications for software reliability extend well beyond solving IMO problems.
References
Answers come from this article only. Click any prompt below or open the chat at the bottom right.
🇺🇸 English
In July of 2024, DeepMind dropped an announcement that sounded like just another AI milestone — but it wasn't. Their system, AlphaProof, had solved four out of six problems from that year's International Mathematical Olympiad. And not just the easy ones. It cracked the single hardest problem on the exam — one that only five human competitors on the entire planet managed to solve. Here's why this matters: it wasn't a better language model. It was a fundamentally different kind of machine.
So let's talk about what AlphaProof actually is. At its core, it's an automated theorem-proving system built on what researchers call a "neurosymbolic" architecture. That's a fancy word for a simple idea — you bolt two very different things together. On one side, you have a neural network that provides intuition, a statistical gut feeling for where a proof might go. On the other side, you have a symbolic system — in this case a formal proof language called Lean — that acts as a ruthless, unforgiving referee. Every single step gets checked. And the lineage here is worth noting: AlphaProof descends from AlphaZero, the same reinforcement learning system that mastered chess, Go, and shogi. Now that same engine has been pointed at mathematics.
Now, why does this architecture matter so much? Here's the thing about ordinary language models and math — they have a deep, structural flaw. They can't check their own work. A large language model generates reasoning that looks plausible, step after step, but there's no internal mechanism to catch a mistake halfway through a proof. It just keeps confidently marching forward, error and all. AlphaProof sidesteps this entirely. It translates the math problem into Lean, and in Lean, every inference either holds or it doesn't. There's no "sounds about right." It's binary. And that turns out to be the secret ingredient, because a clean yes-or-no signal is exactly what reinforcement learning needs to genuinely learn from failure.
Let me walk you through how it actually runs. There are two layers working together. The first is the language model layer — a Gemini-based model that reads the problem in plain human language and translates it into Lean, then proposes candidate steps for the proof. The second is the reinforcement learning layer, the AlphaZero engine. It takes those candidates and searches through possible proof paths, and as it goes, every Lean step gets verified in real time. When a proof succeeds, that success flows back and strengthens the language model's instincts for similar problems down the road. Then the difficulty ratchets up, and it does it all again, harder. And the payoff of this whole design is one word: verifiability. AlphaProof doesn't hand you an answer that's "probably correct." It hands you a complete proof that any mathematician can pick up and recheck, line by line, in Lean.
Back to that Olympiad performance for a second, because the details are striking. Of the four problems it solved, two were algebra, one was number theory, and one was geometry — and that geometry problem was the brutal one, the one only five humans on Earth got. Under standard IMO scoring, seven points per problem, that adds up to twenty-eight points. Silver-medal territory. Now, one honest caveat — it did not do this in the four-and-a-half-hour window that human competitors get. It took far longer. But speed was never the point here. Accuracy was.
And AlphaProof didn't stop at competition math. Its successor, AlphaProof Nexus, turned toward genuinely open problems — the kind mathematicians actually don't have answers to yet. It solved nine out of three hundred fifty-three open Erdős conjectures, and it proved forty-four out of nearly five hundred conjectures from the OEIS, the online encyclopedia of integer sequences. These aren't benchmark scores. These are real, new contributions to mathematics.
It helps to put this side by side with other approaches. Take today's reasoning language models — the o1s and o3s of the world. They have no verification at all; whatever they output is only as good as the generation itself, with no guarantee behind it. Then there are classical computer algebra systems like Mathematica — those do verify their arithmetic, but they're boxed into purely computational problems. AlphaProof sits in a different category: full formal verification through Lean, and the output is a complete, checkable proof. That combination is what nobody else has.
So let me leave you with the three things that really matter here. First — the breakthrough isn't that an AI can do math. It's that the neurosymbolic design, neural intuition fused with formal verification, finally gives an AI real error-correction inside structured reasoning. That's the deep shift. Second — the results are concrete and verifiable, from silver-medal Olympiad performance to actual proofs of previously open conjectures, all of it recheckable by any human. And third, the part worth sitting with — formal languages describe a lot more than mathematics. They describe software specifications, network protocols, security properties. If this framework transfers to those domains, then the story here was never really about winning a math competition. It was about a path toward software you can actually prove is correct. And that's a much bigger prize.
🇹🇼 中文
大部分人對「AI 解數學題」的直覺是這樣的:語言模型在多步推導的中間會跳步、會胡說八道,算出錯的答案卻一臉自信。DeepMind 最新的 AlphaProof Nexus,展示了一條繞過這個問題的路——它去挑戰匈牙利傳奇數學家 Paul Erdős 留下的開放難題,這些題目,幾十年來沒有任何人解得開。
而真正有意思的地方,不在於模型變得多聰明,而在於它外面套的那一圈迴圈,我們叫它 harness。
先講結果。Erdős 生前留下超過一千道開放問題丟給世界去解。這次 AlphaProof Nexus 挑了其中大約三百五十道,最後解出了九道。官方也很誠實,失敗率大約百分之九十五點七,每一題的成本大概幾百美元。聽起來失敗率很高,但你要記得,這些是數十年來無人破解的老問題,能啃下九道,已經相當驚人。而突破的核心,就是把一個不可靠的 AI,放進一個嚴謹的迴圈裡,最後逼它產出可以被電腦逐步檢查的形式化證明。
那為什麼不能直接問 AI 助理就好?因為你直接叫它證這些題,它多半解不出來——它會 hallucinate、會把東西掰出來,講得煞有其事,但完全站不住腳。
DeepMind 的做法是強迫它用 Lean。Lean 是一種形式化的數學語言,任何一步推導都能被電腦自動判定合不合法,要嘛過、要嘛不過,沒有模糊地帶。但這一點本身其實不新鮮,現在大家都在這樣做。真正新的東西,在 Lean 外面那一圈。
我來把整個流程講一遍。第一步,人先出題:一位數學家把問題、還有「要證明什麼」寫成 Lean,但證明本身留空。第二步,AI agent 去試著填那段證明,而它通常會失敗,因為題目真的太難了。第三步,另一個 AI 負責批改,它不只說「這不行」,還會說出「為什麼不行」。第四步是關鍵——一個很便宜的 judge AI 上場比大小。它讀兩份先前的解答,挑出比較好的那一份。注意,這兩份很可能都是錯的,但它就是挑出那個「稍微沒那麼爛」的。
這裡的精髓,是把每一份解答都當成一名棋手,給它一個 ELO 分數。順帶一提,Elo 評分系統也是匈牙利人 Arpad Elo 發明的。有時候人也會餵進幾份解答一起評。於是每一份證明,都有了分數。
然後就重新開始——但不是從零開始,而是從「分數最高的那份爛解」接著往下推。這就變成一場錦標賽,一輪一輪跑下去,直到 Lean 驗證器說「這份沒問題」,我們就拿到了一份形式化證明。
你可以把這整個流程想像成一個閉環:出題、嘗試、批改、評分、驗證,只要驗證沒過,就回到分數最高的那份爛解重跑,過了就收工。這套設計之所以厲害,是因為它從一開始就接受核心 AI 會說謊、會胡扯,但只要讓它跑夠多次,外面套一個好裁判加上錦標賽機制,最後還是能擠出一個可靠的結果。用不可靠的零件,組出一個可靠的系統。
這其實代表 AI 的故事變了。過去幾年,敘事一直是「把模型做得更聰明」。這次不一樣:你不一定要讓模型更聰明,而是要把它外面的 harness 收得更緊。給它一個好裁判,讓它跑上一千次,它就會慢慢磨出極難問題的正解。智慧不只在模型裡,而是在環繞它的那個迴圈裡。對工程師來說,這是一個值得記下來的轉向:面對會出錯的模型,與其死磕「換更強的模型」,不如去設計外圍的驗證、評分跟重試機制。很多時候,可靠性是在迴圈裡長出來的,不是在權重裡。
不過這技術也不是萬能的,有兩個限制主流報導不太提。第一,選題有偏差。Erdős 的開放問題其實有大約一千兩百道,這次只挑了三百五十道,很可能是「比較容易寫成 Lean」的那個子集。這算問題嗎?其實還好,總要從某個地方開始,但你要知道,這不是「全部題目」的成績。第二,也更值得留意——換成比較小的模型去跑,解出的題目數是零,一題都沒有。也就是說,核心仍然需要一個夠強的大模型,光有迴圈、沒有夠力的引擎,一樣跑不出來。這一點其實很有啟發:很多 benchmark 上,又快又便宜的小模型看起來只落後前沿模型幾個百分點,但一旦真的拿去解硬問題,差距往往比帳面上大得多。
最後換個角度看這個成績,看看這條一直在移動的門檻。四年前,大家說 GPT-3 連加法都算不可靠。兩年前,說它連高中競賽題都解不穩。一年前,說它連數學奧林匹克金牌都拿不到。到了今天,大家說它連五十年沒人解開的難題都解不穩。你看,門檻一路被往上推。用「論文第一定律」來看:別只盯著現在在哪裡,想想再過兩篇論文之後會在哪裡。
好,收尾幫你抓三個重點。第一,這次真正的突破不是模型更聰明,而是外面那圈 harness 收得更緊——judge 加錦標賽加 Lean 驗證,用不可靠的零件組出可靠的系統。第二,迴圈很重要,但引擎也不能少,換小模型直接掛零,說明夠強的大模型仍是地基。第三,這九道題是有選題偏差、也有成本的成績,但放在「數十年無解」這個尺度上看,它確實是相當驚人的一步。而下一步會走到哪,值得我們持續盯著。
Tags
Related Articles
AlphaFold's Nobel Prize: When AI Starts to Decode the Language of Life
AlphaFold's protein structure predictions earned the 2024 Nobel Prize in Chemistry. Here's what the MSA + Transformer architecture actually does and why it matters.
AlphaFold: The AI That Solved Biology's 50-Year Problem and Won a Nobel Prize
AlphaFold solved the protein folding problem in 2020 at near-experimental accuracy, earning Demis Hassabis and John Jumper the 2024 Nobel Prize in Chemistry. Its database now contains 200M+ protein structures, actively accelerating drug development and materials science.
Demis Hassabis: Why I Love Hard Questions — The Core of a Research Philosophy
Hassabis's preference for 'hard questions' isn't a personality quirk — it's a research strategy: choose problems that unlock large amounts of downstream value when solved, not problems easy enough to publish quickly. This strategy is the core reason DeepMind keeps breaking through at the scientific frontier.