Machine-Checkable Proof Certificate vs Human Expert Review (2026): What a Lean Certificate Actually Proves
OpenAI shipped ten decade-open proofs with Lean certificates for under $2,000. Days earlier, Lean's kernel accepted a proof of False. What a certificate proves.
Use the certificate for the interior of the argument and the human for its boundary — and never let either one claim the other's job. The August 2026 evidence makes this unusually concrete. Ten decade-open results with Lean certificates for under $2,000 is a genuine collapse in the cost of the thing that used to be expensive: checking every step of a long argument. That work is now seconds of CPU, repeatable by anyone, forever. Nothing in human review competes with that, and pretending otherwise is nostalgia. But two facts from the same week set the limit precisely. Lean's kernel accepted an axiom-free proof of False for about a week, and the main independent re-implementation accepted it too — a certificate relocates trust to the checker, it does not remove it. And as of 2 August the openai/ten-proofs repository had 269 stars, 24 forks, and zero issues or pull requests filed against it. Nobody had audited anything. 'Checkable' is a property; 'checked' is an event, and the two are being conflated in almost every summary of the story. The deeper limit is not a bug and will not be patched. A certificate proves the formal statement follows from the axioms. It says nothing about whether that formal statement is the conjecture anyone meant. This is why Gary Marcus's objection survived the certificates rather than being answered by them: a 249-page paper with no account of the verification process, the human role, or the failures leaves the one gap machines cannot close. Itai Sher's version is sharper still — without the problems that were attempted and failed, there is no denominator, and a hit rate of unknown size is not a result. Our recommendation for teams making this call on ordinary work — AI-generated code, migrations, financial models, security fixes — is a division of labour, not a choice. Push everything mechanisable into machine checks and run them on every change, because that is where the marginal cost is zero and human attention is worst spent. Then concentrate your scarce expert time on exactly two questions the checker structurally cannot answer: is this the right statement, and what is the blast radius if it is wrong. The failure mode of the next few years is not teams that refuse to use certificates. It is teams that read a green check and stop asking what it was checking.
Detailed Comparison
A side-by-side analysis of key factors to help you make the right choice.
| Factor | Machine-Checkable Proof CertificateRecommended | Human Expert Review | Winner |
|---|---|---|---|
| What is actually guaranteed | That the formal statement follows from the stated axioms, mechanically and exhaustively. No step is skipped, no 'clearly' is taken on faith, and the checker has no reputation to protect. | That a qualified person believed the argument — and, critically, that the thing being proved is the thing anyone cared about. Human review covers the question as well as the answer. | |
| Marginal cost per check | Effectively zero. Once the certificate exists, checking it is seconds of CPU and can run in CI on every commit, by anyone, forever. | Weeks to months of scarce expert attention, and it does not get cheaper the second time. Re-verification means paying the full cost again. | |
| Where the trust sits | Relocated, not removed. You now trust the kernel — a piece of software with bugs, as Lean's own July 2026 postmortem documents in detail. | Distributed across named people with reputations at stake and domain intuition the checker does not have. Diffuse, slow, but not a single point of failure. | |
| The formalization gap | Unaddressed by construction. A certificate proves the formal statement, never that the formalization matches the informal conjecture people meant. This is the gap Gary Marcus was still asking about after the certificates shipped. | This is precisely what a reviewer is good at. Reading the informal statement next to the formal one, and objecting when they diverge, is human work with no mechanical substitute. | |
| Scalability | Unlimited and parallel. Ten thousand certificates check as easily as one, and the checking can be delegated to anyone with the binary. | Hard-capped by the number of people qualified to review the specific claim — often a single-digit number in a specialised field, and they have day jobs. | |
| Failure mode | Silent and systemic. One kernel bug quietly validates everything checked through it, and you get no warning. In July 2026 that failure lasted about a week across two independent implementations. | Noisy and local. One reviewer misses one thing in one paper. Bad, but bounded — the error does not propagate to every other result in the field. | |
| Speed of correction | Hours. The Lean soundness bug went from minimal reproduction to pushed fix in about an hour, and a tagged release the same day. Software defects have software response times. | Years. A flawed result that survives review can stand in the literature for a long time, and retraction depends on someone caring enough to re-derive it. | |
| Cost to set up | High and front-loaded. Formalising a statement in Lean is the actual work — often harder than the informal proof — and it demands a skill set most teams do not have. | None beyond what already exists. Journals, review boards and senior engineers are infrastructure you are already paying for. | |
| Total Score | 3/ 8 | 2/ 8 | 3 ties |
Key Statistics
Real data from verified industry sources to support your decision.
OpenAI
Leonardo de Moura, Lean
Leonardo de Moura, Lean
leanprover/lean4 issue #14576
GitHub, openai/ten-proofs
Gary Marcus
All statistics come from verified third-party sources. Source, year, and direct link are shown on each metric.
When to Choose Each Option
Clear guidance based on your specific situation and needs.
Choose Machine-Checkable Proof Certificate when...
- The claim is fully formalisable — a proof, a protocol, a compiler pass, a type-safety or cryptographic property — so that a mechanical check covers the entire assertion rather than a fragment of it.
- The same claim will be re-verified many times, on every commit or every release. This is where the near-zero marginal cost compounds and human review never can.
- No qualified reviewer is reachable, or the pool is small enough that review becomes a scheduling problem rather than a technical one.
- You need assurance that does not depend on trusting the producer — an outside auditor, a customer or a regulator can run the checker themselves without taking your word for anything.
Choose Human Expert Review when...
- The hard part is whether you are solving the right problem at all. A certificate cannot tell you the formal statement matches your intent; only a person reading both can.
- The claim resists formalisation — architecture decisions, threat models, design trade-offs, anything where 'correct' depends on context that never enters the formal statement.
- The result must carry weight with people: a board, a customer, a court or a scientific community. Consensus among named experts is a social artifact, and a machine cannot manufacture it.
- Formalising would cost more than the decision is worth. For a one-off, low-blast-radius claim, an afternoon of senior review is the cheaper instrument.
Our Recommendation
Use the certificate for the interior of the argument and the human for its boundary — and never let either one claim the other's job. The August 2026 evidence makes this unusually concrete. Ten decade-open results with Lean certificates for under $2,000 is a genuine collapse in the cost of the thing that used to be expensive: checking every step of a long argument. That work is now seconds of CPU, repeatable by anyone, forever. Nothing in human review competes with that, and pretending otherwise is nostalgia. But two facts from the same week set the limit precisely. Lean's kernel accepted an axiom-free proof of False for about a week, and the main independent re-implementation accepted it too — a certificate relocates trust to the checker, it does not remove it. And as of 2 August the openai/ten-proofs repository had 269 stars, 24 forks, and zero issues or pull requests filed against it. Nobody had audited anything. 'Checkable' is a property; 'checked' is an event, and the two are being conflated in almost every summary of the story. The deeper limit is not a bug and will not be patched. A certificate proves the formal statement follows from the axioms. It says nothing about whether that formal statement is the conjecture anyone meant. This is why Gary Marcus's objection survived the certificates rather than being answered by them: a 249-page paper with no account of the verification process, the human role, or the failures leaves the one gap machines cannot close. Itai Sher's version is sharper still — without the problems that were attempted and failed, there is no denominator, and a hit rate of unknown size is not a result. Our recommendation for teams making this call on ordinary work — AI-generated code, migrations, financial models, security fixes — is a division of labour, not a choice. Push everything mechanisable into machine checks and run them on every change, because that is where the marginal cost is zero and human attention is worst spent. Then concentrate your scarce expert time on exactly two questions the checker structurally cannot answer: is this the right statement, and what is the blast radius if it is wrong. The failure mode of the next few years is not teams that refuse to use certificates. It is teams that read a green check and stop asking what it was checking.
Frequently Asked Questions
Common questions about this comparison answered.
Need help deciding?
Book a free 30-minute consultation and we'll help you determine the best approach for your specific project.