Dario Amodei, the chief executive of AI developer Anthropic, has called for the pace of large model development to be slowed and for tighter external oversight, arguing that the risks posed by advanced artificial intelligence are "serious" and demand time for companies and governments to respond.
Three-part plan for oversight
In an essay published publicly over the weekend, Amodei set out a three-point proposal aimed at creating stronger checks on how new AI capabilities are built and released. He emphasised that he was not opposing research itself but sought measures to manage the potential harms.
| Proposal element | Purpose |
|---|---|
| Independent monitoring | Third-party evaluation of models during development |
| Industry-wide regulation | Common standards and obligations for firms |
| Global regulation | International frameworks to manage cross-border risks |
The proposals gained immediate traction among senior figures in the technology sector. OpenAI’s chief executive, Sam Altman, publicly said he agreed that pace should be controlled and praised the idea of third-party evaluators as "a great idea". Billionaire entrepreneur Elon Musk also signalled support.
Inside the industry: concern and alarm
The debate has been intensified by statements from researchers who have recently left Anthropic. One former researcher told the BBC that if the current speed of progress continued unchecked, there was a perceived risk to humanity within a short timeframe, reflecting deep unease among some technical staff.
"If we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future."
That stark assessment, attributed to a researcher speaking to the BBC, underscores how serious some practitioners view the potential downside of accelerating capabilities. The same report quoted that many people closely involved in building these systems place non-zero probabilities — in some cases greater than 10% — on extinction-level outcomes, and that these concerns are held privately by those working on advanced models.
What proponents and critics are saying
Amodei’s essay frames the conversation as one of risk management rather than prohibition. He proposes concrete steps that would allow monitoring to occur while preserving the ability to continue research. Support from rival executives suggests a rare moment of consensus at the top of the industry on the need for additional safeguards.
- Industry alignment: Senior leaders at multiple firms have signalled agreement with calls for pacing and independent review.
- Regulatory gap: Executives acknowledge that current standards and oversight are not yet adequate for controlling the next wave of capability growth.
- Worker concern: Some technical staff are reportedly alarmed enough to publicly warn of severe risks if development continues unchecked.
While the conversation centres on managing existential risks, it also raises practical questions for policymakers. Setting up credible independent evaluators and creating enforceable rules across an international landscape of competing companies will be complex. The balance policymakers must strike is between preventing catastrophic outcomes and avoiding unduly constraining beneficial research and commercial innovation.
The emerging alignment of prominent industry figures behind Amodei’s proposals is likely to intensify calls for concrete regulatory proposals in the UK, EU, US and at global forums. If momentum continues, governments may face increased pressure to define testing regimes, reporting requirements and thresholds for external intervention in model development.
For now, the debate remains largely about principles and proposed structures. Translating those into binding frameworks will require detailed technical standards, new institutions or accreditation systems for independent monitors, and international cooperation — all while keeping pace with rapid technological change.
The significance of the conversation is that it acknowledges a shared, if uneasy, recognition within the industry: the technology’s potential benefits are matched by risks that some insiders say need urgent, systemic action.
