Executive briefing · Leopold Aschenbrenner · June 2024
Situational AwarenessThe Decade Ahead
A 165-page argument that AGI arrives around 2027, superintelligence follows within about a year, and the U.S. national-security state ends up running the endgame. Here it is on one page.
Bottom line up front
The thesis in five lines
- Count the orders of magnitude. Compute, algorithmic efficiency and "unhobbling" trends make AGI by 2027 "strikingly plausible": models that do the work of an AI researcher. I
- Then it accelerates. Hundreds of millions of AGI copies automating AI research could compress a decade of progress into under a year, reaching superintelligence by about 2030. II
- The build-out is industrial. By 2030: a $1T+ training cluster drawing 100 GW, about $8T a year of total AI investment, and power demand equal to all current U.S. electricity output. IIIa
- The labs aren't ready. Security is "random startup" grade, and nobody yet knows how to reliably control AI smarter than us. IIIb IIIc
- So it becomes national security. Superintelligence is militarily decisive. The free world must keep its lead, and a U.S.-led government "Project" takes over around 2027/28. IIId IV
How to read this. This page summarizes the author's argument and estimates as published in June 2024. It doesn't endorse them or forecast independently. Text in quotation marks is verbatim. Each chapter header links to the original.
The argument as a chain
Six links, each with a weak point the author names himself
- 1Trends hold → AGI ~2027Breaks if: the data wall isn't cracked with synthetic data, self-play or RL
- 2AGI automates AI research → superintelligence ~2028–30Breaks if: compute for experiments bottlenecks automated researchers
- 3Industry builds the clustersBreaks if: power and permitting stall. "Where do I find 10GW?"
- 4Secrets and weights stay in U.S. handsBreaks if: state actors steal them. One lab self-rates at security level 0
- 5We can control what we buildBreaks if: alignment fails mid-takeoff, when the first serious failure may be catastrophic
- 6Government steps in: The ProjectOpen question: will it be competent? The author calls this under-examined
The forecast on one axis
2024–2030, as the essay projects it
Collected from all chapters. The today marker shows where today sits on the author's 2024 timeline. It doesn't say which of these milestones have actually happened.
From GPT-4 to AGI: Counting the OOMs
"AGI by 2027 is strikingly plausible."
GPT-2 to GPT-4 took AI from preschooler to smart high-schooler in four years. Extend the same trend lines (compute, algorithmic efficiency, unhobbling) and you get another jump that size by 2027: models that do the work of an AI researcher or engineer.
Detail & quotes
Unhobbling: chatbot → agent
- RLHF: a small RLHF'd model matched a non-RLHF'd model more than 100× larger on rater preference.
- Scaffolding, tools, post-training: 5–30× effective compute (Epoch survey).
- Test-time compute: +4 OOMs at inference ≈ +3 OOMs of pre-training.
- Agentic tasks (METR): ~5% base model → ~20% GPT-4 at launch → ~40% mid-2024.
- SWE-Bench: ~2% for GPT-4 → 14–23% with Devin-style scaffolding.
Benchmarks
- MMLU (2020): ~90% three years later.
- GPQA (PhD-level): Claude 3 Opus ~60% vs. in-domain PhDs ~80%, "within one or two generations."
- MATH: post-training alone took GPT-4 from ~50% to 72%.
- Progress runs at about 3× the pace of child development.
The big uncertainty: the data wall
- Llama 3 trained on 15T+ tokens. Common Crawl is ~30T after simple deduplication.
- Repeating data stops working: after ~16 epochs, "returns diminish extremely fast to nil."
- The author expects labs to get past it (synthetic data, self-play, RL) but says there's a real chance progress stalls.
"We are on course for AGI by 2027."
"In 2027, a leading AI lab will be able to train a GPT-4-level model in a minute."
"We've been racing through the OOMs this decade; after the early 2030s, we will face a slow slog."
From AGI to Superintelligence: the Intelligence Explosion
"AI progress won't stop at human-level."
Once models can do AI research, labs can run them by the hundred million, around the clock. A decade of algorithmic progress (5+ OOMs, about a GPT-2→GPT-4 jump) gets compressed into a year or less. That's the leap from human-level to vastly superhuman.
Bottlenecks the author tests, and his verdict
| Bottleneck | Verdict |
|---|---|
| Limited compute for experiments | Most important still beatable, at least 10× effectiveness gain |
| Complementarities / long tail | Delay full automation may take an extra year or two |
| Inherent limits to algorithms | Unlikely today's architectures look inefficient |
| Ideas get harder to find | Outweighed by a 100,000× bigger research force |
| Diminishing returns | Survivable a one-time boost still yields many OOMs |
What superintelligence would bring
- ⚔Military: a decisive, overwhelming advantage, including new classes of weapons
- ⚗Science: roughly a 20th century's worth of progress in under a decade
- ⚙Robotics: solved as an ML problem, at most a few years' delay
- ⚖Power: whoever controls it could overpower pre-superintelligence forces
Detail & quotes
The arithmetic
- Assumes a human "thinks" at about 100 tokens per minute.
- You can trade copies for speed, for example ~1M researchers at ~100× human speed.
- Today's algorithmic progress is ~0.5 OOM/yr, so 5+ OOMs normally takes a decade.
- Automated research compresses that into under a year.
Mainline sequence
- 2026/27: proto-automated engineer, 1.5–2× speedup.
- 2027/28: proto-automated researchers do over 90% of the work, 3×+.
- 2028/29: 10×+ pace, superintelligence.
- From fully automated researchers to vastly superhuman: "a year, or a few years at most."
"the first ultraintelligent machine is the last invention that man need ever make" I. J. Good, quoted
"one of the most volatile, tense, dangerous, and wildest periods ever in human history"
Racing to the Trillion-Dollar Cluster
"The most extraordinary techno-capital acceleration has been set in motion."
The largest training cluster grows about 10× every two years. By 2030 a single cluster costs more than $1T and draws 100 GW, over 20% of today's U.S. electricity. The binding constraint is power, not chips.
The cluster ladder: largest single training clusterBar length: power draw (log scale, each tick 10×)
| 2024 | ~2026 | ~2028 | ~2030 | |
|---|---|---|---|---|
| Accelerators shipped (H100-eq.) | 5–10M | 10s of M | ~100M | 100s of M |
| Power, % of U.S. electricity | 1–2% | 5% | 20% | 100% |
| Chips, % of leading-edge TSMC output | 5–10% | ~25% | ~100% | 4× capacity |
The author's read
- Power is solvable. 10 GW needs only a few percent of U.S. gas output, and 100 GW of gas plants costs ~$100B. The obstacles are permitting, regulation and climate commitments, so deregulation is a national-security priority.
- Chips: dozens of ~$20B "Gigafabs," with over $1T of capex. Advanced packaging (CoWoS) and HBM memory are the near-term bottlenecks.
- Location is the strategic choice: build in America, not in Gulf autocracies.
Detail & quotes
"Where do I find 10GW?" the question every AI CEO is asking
"The power constraint can, must, and will be solved."
"The only question is whether they will be built in America."
"We're going to drive the AGI datacenters to the Middle East, under the thumb of brutal, capricious autocrats." the risk if the U.S. doesn't build
"Brace for the G-forces."
Electricity math: total U.S. production is ~4,250 TWh/yr. A 10 GW cluster running all year uses ~87.6 TWh, and 100 GW uses ~876 TWh (~20%). Utilities now project 4.7% growth over five years, up from 2.6%.
Lock Down the Labs: Security for AGI
"The nation's leading AI labs treat security as an afterthought."
Two assets matter: model weights, and the algorithmic secrets being invented right now. Both are effectively open to state actors. The window to fix the second is 12–24 months, and fixing the first takes years of lead time.
The security gapGoogle DeepMind's Frontier Safety Framework security levels, as the essay describes them
What must be protected
"A large file of numbers." Stealing it skips the whole training effort. Stolen automated-researcher weights let an adversary run its own intelligence explosion.
The post-LLM methods that get past the data wall. Worth an estimated 10–100× in compute efficiency, like owning a much bigger cluster. Protect these first: they're being developed now.
Evidence cited
- A Chinese national copied Google AI source code into Apple Notes, exported it to PDF and uploaded it. He was caught only after other mistakes.
- ByteDance reportedly recruited authors of the Gemini paper with senior-level offers.
- Russian hackers stole Microsoft executives' emails. The OPM breach exposed 22M security-clearance files.
- Thousands of people have access, there's little vetting or compartmentalization, and secrets get discussed "at SF parties."
Prescription
- Air-gapped training and inference datacenters with military-grade physical security
- Confidential compute and hardware encryption, with a scrutinized supply chain (defense in depth, since encryption can be side-channeled)
- Researchers working from SCIFs
- Extensive vetting, integrity testing, information siloing
- Multi-key sign-off to run code. Few external dependencies
- Ongoing NSA-style penetration testing
- Government involvement: only the state can defend against state actors
On cost: strict security slows labs down. The author treats this as a tragedy of the commons and says a ~10% slowdown is worth paying.
Quotes
"Basically nothing else we do—on national competition, and on AI safety—will matter if we don't fix this, soon."
"AI lab security isn't much better than ‘random startup security.’"
"We are not on track"
Superalignment
"Reliably controlling AI systems much smarter than we are is an unsolved technical problem."
Today's main technique, RLHF, depends on humans understanding what the model is doing. That stops working once systems are superhuman, and in an intelligence explosion the crossover could take less than a year. The author calls it solvable, but says it could go badly wrong in a rushed takeoff.
Optimism, split
Cautiously to strongly optimistic. There's lots of low-hanging fruit, and deep learning has generalized more benignly than feared.
Deeply worried. Very few people work on it, labs won't make costly trade-offs, and the world will "stumble" into the explosion.
Why the explosion makes it harder
- Speed: under a year from "RLHF works" to "RLHF fails"
- Stakes: the first serious failure may already be catastrophic
- Trust: humans can't verify superhuman work
- Alien architectures: reasoning may stop being human-readable
- Volatility: arms-race pressure and ambiguous evidence
Research bets
"Superdefense": defense in depthOuter layer first
- Security air-gapped clusters, guard against self-exfiltration
- Monitoring trusted weaker models watch outputs, reasoning and network activity
- Capability limits strip or unlearn bio and chem from training
- Training restrictions prefer imitation over long-horizon RL, never train against the monitors
- Staged deployment isolated R&D first, ship only verified technologies
Quotes
"We'll be like first graders trying to supervise people with multiple doctorates."
"The intelligence explosion will be more like running a war than launching a product."
"We're counting way too much on luck here."
The Free World Must Prevail
"Superintelligence will give a decisive economic and military advantage."
China is still in the race. Its 7nm chips are good enough, it can out-build the U.S. on power, and it can steal the algorithms. The size of the free world's lead decides who wins, and whether there's time to be careful.
Lead length = safety margin
With a tight race, both sides cut corners. Arms control is unstable because "breakout" is too easy. The author proposes a U.S.-led alliance of democracies that uses its lead to enforce safety norms and a nuclear-style nonproliferation regime.
China's three levers
SMIC 7nm Huawei Ascend 910B is ~2–3× worse on performance per dollar than Nvidia, and chips are only part of datacenter cost.
China added about as much generating capacity in a decade as the entire U.S. has. U.S. capacity stayed flat.
The West is ahead by the equivalent of a 10–100× bigger cluster, but weak lab security makes that lead stealable, weights included.
Current state (mid-2024): Chinese LLMs at best match the second tier of U.S. labs, and some build on American open-source models. The authoritarian risk: robotic police, total surveillance and AI loyalty checks could lock a regime in permanently.
Quotes
"Superintelligence will be the most powerful technology—and most powerful weapon—mankind has ever developed."
"A 2 year vs. a 2 month lead could easily make all the difference."
"the military technological advances of a century compressed to less than a decade."
"The free world must prevail over the authoritarian powers in this race."
The Project
"No startup can handle superintelligence."
As capabilities become undeniable, the national-security state steps in. The author expects a government AGI effort, "The Project," by 2027/28, structured like defense contracting rather than literal nationalization. He frames this as a prediction, not a wish.
How The Project comes togetherBars span the author's stated windows
Why not leave it to the labs
- Securitystartups can't withstand full-scale Chinese espionage
- Chain of commandCEOs or nonprofit boards shouldn't control a nuclear-class capability
- Safetycommercial incentives push labs to race, and regulation would be too slow
- Military balanceexisting arsenals could become obsolete
- Stabilitypreventing proliferation after superintelligence needs U.S. leadership
Shape & precedents
- Structure: a joint venture of cloud providers, labs and government, like Boeing or Lockheed with the military. Labs "voluntarily" merge in, Congress appropriates money for chips and power, and key officials are Senate-confirmed.
- COVID, Feb 2020: officials slow to act, then a response that was large and fast but "late, crude, blunt."
- Manhattan Project: Szilard ignored at first. Fermi got $6k, and the British report sat for months.
Endgame
- Alliance Quebec-Agreement-style pact with the UK, Japan, South Korea and NATO
- Benefit-sharing "Atoms for Peace": others give up their own projects in exchange for access
- Nonproliferation keep superintelligence-enabled weapons from rogue states and terrorists
Caveats & quotes
The author admits governments have poor incentives and limited competence, and says he's skeptical of heavy state involvement in tech. He rejects open-sourcing the frontier because it hands it to adversaries, but expects open models a couple of years behind to stay useful. He expects civilian uses to flourish privately once the initial danger passes. Whether the Project would be competently run is "the key open question."
"In a few years, The Project will be on."
"Imagine if we had developed atomic bombs by letting Uber just improvise."
"But of the options we have, The Project is the only sane one."
Parting Thoughts
"What if we're right?"
The author rejects both doomers (overconfident, unworkable) and e/accs (unserious) and proposes a third way he calls AGI Realism.
"The most powerful weapon mankind has ever built." More important than anything else people in the field will work on.
"The torch of liberty will not survive Xi getting AGI first." A pause isn't realistic. Scale U.S. power, lock down the labs, keep core infrastructure under American control.
Take misuse and loss of control seriously, with deliberate work rather than improvisation.
"there is no crack team coming to handle this"
According to the author, only a few hundred people grasp what's coming, nearly all within one degree of separation from him. In his telling, the midgame rests on them: "These are great and honorable people. But they are just people."
Author's own caveat: he has "likely gotten important details wrong," and his error bars are large.