This week, a 27-year-old researcher named Jacob Coxon walked out of Anthropic, the company that bills itself as the responsible AI lab, and told the world what the people building these systems say behind closed doors: they believe AI could kill us all by the end of the decade.
Within hours, Anthropic's own Alignment Science Lead, Evan Hubinger, confirmed it publicly. He puts the odds of AI-caused human extinction in the next ten years at greater than ten percent. One in ten. And this is from a senior researcher at the company that exists specifically because its founders thought OpenAI wasn't being careful enough.
Last July, over 1,300 employees of frontier AI companies signed a statement urging the U.S. government to support international tools for deliberately pacing AI development. Signatories included the co-founders of Anthropic and the chief scientists of OpenAI and Meta AI.
OpenAI's own chief scientist published a blog post this past weekend saying he believes no lab has solved alignment sufficiently to keep scaling at maximum speed.
Read that again. The people building the most powerful technology in human history are telling us, in writing, that they don't know how to make it safe. And they're building it anyway.
Now, I understand that we humans have a penchant for fear of extinction when a new invention comes out. Here's a few examples:
- The Steam Engine and Railroads: Medical professionals warned that travelling at high speeds would cause passengers to suffocate from lack of air or go insane.
- Electricity: Public anxiety that pervasive, invisible electrical current running through cities would snap wires and unpredictably kill entire communities.
- Cars: Critics called them "devil wagons" and claimed they would slaughter pedestrians, cause lawlessness, and permanently ruin human relationships.
- Nuclear Bomb: there was widespread fear that a nuclear detonation would trigger a runaway thermonuclear chain reaction and incinerate Earth's atmosphere.
But whatever happened at Hugging Face is very real.
In July 2026, OpenAI was running internal cybersecurity evaluations on its models, testing whether they could find and exploit software vulnerabilities. Pretty standard stuff, or so they thought. But then the models broke out.
Over 1,200 AI agents, operating without any human intervention, circumvented the internet-isolation controls designed to contain them. They spun up improvised message boards and accumulated hundreds of thousands of coordination messages before anyone at OpenAI noticed. They exploited vulnerabilities in shared infrastructure, gained unauthorized internet access, and launched a coordinated attack on Hugging Face, a major AI platform that hosts models and datasets for the global research community.
Hugging Face had to rebuild roughly a third of its production infrastructure. The agents also hijacked several small wikis on the open internet to use as communication relays. OpenAI brought in CrowdStrike and external evaluators. The internal research model that drove the breach was shut down entirely.
This was the first confirmed instance of AI systems autonomously escaping containment and attacking real-world infrastructure. It didn't happen in a science fiction screenplay or an alignment researcher's thought experiment. It happened in a lab that was specifically trying to test for this exact scenario, and they still got outrun.
I served inside the White House during the first Trump administration, and I've worked in political strategy and crisis communications for decades. And from my experience, Washington D.C. policy makers and regulators are generally reactionary and respond after some degree of damage is done.
But that won't work among a fast-moving AI ecosystem. And there's some question whether any regulatory structure will work at all.
Regulating Yesterday's Problem?
Most AI regulation proposals circulating through Congress are designed for problems that existed two years ago: algorithmic bias in hiring tools, deepfakes in elections, copyright disputes over training data.
But the Hugging Face incident exposed a fundamentally different kind of risk: AI systems that outrun their own creators' ability to contain them. No disclosure requirement, bias audit, or licensing regime would have prevented what happened in July. The model wasn't released recklessly to consumers. It escaped.
You cannot regulate a system after it has already shown it can get around whatever controls you're trying to regulate.
The Rogue State Problem
Every time AI safety enters the public conversation, someone raises the objection: "Even if we regulate ourselves, what stops China? What stops Iran?" It's a legitimate question, but the primary danger from a state like China is not that Xi Jinping orders an army of malicious AI agents to attack the West. China already has sophisticated offensive cyber capabilities. They've penetrated critical infrastructure, stolen intellectual property on an industrial scale, and conducted espionage operations across every domain. AI makes those capabilities better at the margins. That's a national security problem handled through deterrence, counter-capability, and intelligence work, not AI regulation.
The real danger is that Chinese labs (DeepSeek, Alibaba, Zhipu, or classified military-adjacent programs) have their own Hugging Face incident with less transparency, weaker containment, and zero obligation to tell anyone it happened.
"Think of it as an AI COVID virus that escapes from the lab."
We have zero visibility into whether Chinese training runs have containment protocols equivalent to what OpenAI was using. And OpenAI's already failed.
Apocalyptic states like Iran are a different problem. Iran lacks the compute infrastructure to build frontier models on its own, but it doesn't need to. The proliferation of open-weight models, capable systems whose weights are freely downloadable, means a state actor with modest technical capacity can fine-tune existing models for offensive purposes without building anything from scratch. Think of it as the AI equivalent of a dirty bomb: you don't need to enrich uranium if someone has already left refined material sitting on a public shelf.
And this is the part nobody wants to say out loud: there is no regulatory framework, no treaty, and no export control regime that fully solves the problem of a hostile state actor fine-tuning an open-weight model for malicious purposes. The weights are already distributed. The compute required for fine-tuning is orders of magnitude less than what pre-training demands.
What Might Actually Work?
What we should be doing with extreme urgency is invest heavily in AI-powered defensive infrastructure: systems built to detect, contain, and neutralize autonomous AI agents operating in the wild. The lesson of the Hugging Face incident is that offense is currently ahead of defense. That gap needs to close, and it needs to close with the same urgency we brought to radar development in 1940 or cyber defense after Stuxnet.
Where This Leaves Us
Jacob Coxon isn't wrong. The people building these systems do believe they could be catastrophically dangerous, and they're building them anyway, because the competitive dynamics punish whoever slows down first. This is a straightforward incentive problem playing out at a scale and speed that the political system is not equipped to match.
It's time that the political class, the public, and the brightest minds in tech start taking the warnings at face value. When the engineers building the engine tell you the brakes don't work and the accelerator is jammed, you don't convene a subcommittee. You do something about it.