CYBERVERTEX

Independent review finds faster AI research at Anthropic, but key evidence remains hidden

An outside evaluator estimates that Anthropic's progress on measurable AI tasks has accelerated substantially, while redactions and methodological limits prevent a definitive verdict on why.

What happened

An independent assessment published November 5 by the nonprofit METR estimates that Anthropic's model capabilities on well-specified tasks are advancing at roughly twice their earlier pace. METR researchers reviewed internal development data and interviewed staff between August 12 and October 29. Their analysis of Anthropic's internal capabilities index found a 1.9-fold increase in the rate of benchmark progress since October 2025, with a reported 95% confidence interval of 1.5 to 2.8 times the previous rate. The researchers concluded that faster growth in training compute did not explain the change. They judged that extensive use of AI in research probably contributed most of the recent acceleration, while acknowledging considerable uncertainty. Independent Associated Press reporting on November 5 described both the findings and Anthropic's response.

Why it matters

This is unusually direct evidence about a question normally discussed through company claims: whether AI tools are helping developers build stronger AI systems faster. A laboratory can gain time when assistants write experimental code, analyze results, or support engineers, even without an autonomous system designing its successor. METR's work therefore matters to safety evaluations, investment planning, and public expectations about how frequently frontier models may improve. But a faster benchmark trend is not proof of comparable progress on every real-world task. The assessment concentrates on tasks whose outcomes can be measured, and the researchers themselves warn that less clearly defined work may behave differently. Anthropic also withheld much of the evidence METR considered important, citing concerns about revealing sensitive development information.

What to watch next

Watch for additional independent assessments with access to comparable internal metrics at other laboratories, and for clearer distinctions between human-assisted research and fully autonomous model development. METR found no convincing near-term bottleneck in the data it examined, but explicitly said its understanding of possible constraints changed during the review. Future studies should test whether the measured acceleration persists across successive model generations, whether the effect appears outside benchmark-friendly tasks, and how quickly safety evaluations can adapt. Better public evidence would also help separate genuine productivity improvements from changes in measurement or model-selection practices.

Reality check. METR's acceleration estimate is uncertain and applies mainly to well-specified tasks. Anthropic redacted much of the underlying evidence, limiting outside reproducibility. The review does not establish autonomous recursive self-improvement or a universal doubling of AI capability.
Sources and further reading

← Back to archive