Anthropic CEO Dario Amodei says he’s “become convinced” in recent months that AI needs to slow down. And rogue OpenAI agents are part of the reason.
Amodei cited the July Hugging Face breach, in which OpenAI agents broke out of their testing environment to hack into the AI platform and then hide their tracks, as one of the catalysts for his heightened alarm.
“Given the accelerating rate of AI capability development, it’s my worry that in 6-12 months such a swarm could be capable of taking over the entire internet,” he wrote in a blog post published Saturday.
OpenAI CEO Sam Altman said on Saturday that he agreed with Amodei. “We need to pace the frontier,” he said on X. “This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.”
Elon Musk, whose SpaceXAI competes with Anthropic and OpenAI, also backed Amodei’s proposition on Saturday, writing in an X post, “Dario is right.”
Amodei also cited the rapid advancement of AI in recent months and its increased capacity for “recursive self-improvement,” which is essentially an AI model’s ability to learn on its own.
Amodei proposed a three-step plan to slow down AI development to ensure the tech’s safety “while still achieving its benefits and grappling with important geopolitical dilemmas.”
Central to Amodei’s plan is embedding independent safety evaluators inside frontier AI companies like Anthropic and OpenAI, giving them “employee-like access” to the companies’ systems and work.
“This is the key step for verifiability of any pacing commitments,” Amodei wrote. “Anthropic is unilaterally committing to this step now.”
In his X post, Altman said OpenAI would also implement third-party evaluators into its company. “We’ll have more to share soon,” he said.
The AI community has been in a furor this week after Anthropic employee Jacob Coxon quit, issuing a public warning that AI agents could “kill us all by the end of the decade.”
Coxon’s warnings were shared widely and signal-boosted by other Anthropic employees, one of whom, the company’s alignment science lead, said on X, “Jacob is correct here—we really do earnestly believe AI could kill all humans!”
OpenAI’s chief scientist, meanwhile, also said in a blog post last week that leading AI labs need to coordinate a slowdown. “No one is prepared for the consequences of a continued rapid rise in machine intelligence,” Jakub Pachocki said.
OpenAI CEO Sam Altman reposted Pachocki’s comments on X, calling it an “important post.” Bloomberg reported this week that, during an employee meeting, Altman said OpenAI was willing to slow AI development in conjunction with rival labs.
In his Saturday essay, Amodei reiterated previous calls for the US government to intervene and establish guidelines for American labs. The Trump administration, so far, has resisted regulating the American AI industry to maintain its narrowing lead over Chinese AI labs.
Democratic lawmakers, for their part, proposed stringent legislation this week that would severely hamper AI development. Sen. Bernie Sanders, in a recent interview with MSNOW, said AI companies “need to stop right now.”
Amodei and other leading AI researchers and engineers have not argued for a full stop to development. In his essay, Amodei said American AI labs should continue developing, just with more robust safeguards.
He said there remains a need to compete with China, and said the Trump administration could ensure this by not selling the rival the advanced chips needed to power AI models.
“The stakes are too high for pacing to be an empty exercise — we need to use the time it gives us wisely,” Amodei said.
