Anthropic and OpenAI announced competing advances on Sept. 1 that highlight both rapid progress in large language models and the thorny safety choices facing developers and regulators.
Benchmarks show big gains — and new risks
Anthropic released Claude Fable 5.1 alongside Claude Mythos 5.1 and reported a striking improvement on a specialised scientific benchmark. On Terminal-Bench-Science 0.1, an agentic evaluation covering 70 scientist-contributed workflows across life, physical, Earth, mathematical and engineering sciences, Anthropic said Fable 5.1 scored 52.6%. That compares with 24.7% for Fable 5, 29.0% for Claude Opus 5 and 22.4% for GPT-5.6 Sol. Anthropic noted a standard error of roughly 3.5 to 4.5 points per model.
On the same day, OpenAI disclosed internal results for a forthcoming model called Astra, saying one configuration achieved 100% on the public ExploitBench benchmark, which measures the ability of models to devise exploits for known vulnerabilities. To probe potential test contamination, OpenAI built an internal variant of the benchmark containing 20 high-severity V8 vulnerabilities disclosed between June and August 2026. Against that internal set, Astra reached an arbitrary code-execution rate of about 39% using roughly 76,000 output tokens. For comparison, OpenAI reported GPT-5.6 Sol remained near 1% at a similar token budget, rising to approximately 11% after about 138,000 tokens.
“We plan to make Astra available soon, but access to its most advanced cybersecurity capabilities will be more limited.”
What the numbers mean
Benchmarks like Terminal-Bench-Science and ExploitBench are designed to measure particular capabilities in constrained settings. Terminal-Bench-Science runs agents in self-contained terminal environments and grades submitted artefacts against hidden tests. ExploitBench tests whether a model can generate working exploits for known vulnerabilities.
The companies’ disclosures illustrate two competing dynamics: rapid capability growth in research and operational AI, and an effort by firms to control how the strongest capabilities are distributed. OpenAI said Astra even identified two previously unknown vulnerabilities and used them in an exploit chain — a claim drawn from an internal evaluation — and indicated that the most capable configuration was not intended for broad public release.
Consequences for policy, research and security
The announcements are likely to provoke renewed attention from policy-makers, cyber-defence professionals and research funders. Key implications include:
- Security trade-offs: Higher-capability models can both accelerate legitimate scientific workflows and lower the barrier to creating cyber exploits.
- Access and governance: Firms are signalling they will restrict advanced configurations; that raises questions about who decides access and how safeguards are audited.
- Benchmark limitations: Scores across synthetic or internal tests can vary with prompt design, token budgets and test composition, so cross-model comparisons require careful interpretation.
Anthropic emphasised substantial performance improvements in scientific tasks, while OpenAI highlighted Astra’s potent cybersecurity performance but said it plans to limit access to the model’s most advanced capabilities. Both companies noted caveats: Anthropic published a standard-error range for its scores, and OpenAI said it used a non-default configuration called Daybreak Blue in its internal evaluations.
| Model / Metric | Reported result |
|---|---|
| Fable 5.1 — Terminal-Bench-Science 0.1 | 52.6% |
| Fable 5 — Terminal-Bench-Science 0.1 | 24.7% |
| Claude Opus 5 — Terminal-Bench-Science 0.1 | 29.0% |
| GPT-5.6 Sol — Terminal-Bench-Science 0.1 | 22.4% |
| Astra — internal ExploitBench (20 V8 vulns) | 39% arbitrary code-execution (≈76,000 tokens) |
| GPT-5.6 Sol — internal ExploitBench | ~1% (similar budget); ~11% after ≈138,000 tokens |
For Canadian researchers and institutions, the developments underscore the accelerating pace of capabilities in both scientific assistance and cyber offence techniques. Regulators and defenders will need up-to-date threat models and clearer frameworks for evaluating when and how models with dual-use potential should be tested, shared or constrained.
As firms race to improve performance on specialised tasks, transparency about evaluation methods, dataset provenance and access controls will matter more than ever for public trust and safety.
