SAN FRANCISCO — Inside Anthropic’s offices this past August, roughly 30,000 AI agents ran simultaneously, doing research and engineering work that engineers used to do themselves. A quarter of the company’s research and development — precisely 26 percent, on a scale Anthropic published Thursday for the first time — is now led by Claude, the AI model Anthropic built and is using to build the next version of itself.
The number would be remarkable on its own. What sharpens it: on the same day Anthropic disclosed that its AI leads more than a quarter of its own improvement work, more than 100 AI researchers and safety evaluators published a public letter warning that the evaluators meant to oversee companies like Anthropic lack the independence, resources, and legal protections to credibly do so. The disclosure and the warning landed within hours of each other.
Anthropic published the figures through what it calls its R&D Automation Index — three measurements quantifying how much of the company’s internal research and engineering is now performed or “led” by AI systems, Engadget reported. The scale Anthropic used was developed by Epoch AI, a nonprofit that did not build the models being measured. What Anthropic built was the interpretation of where each of its R&D tasks sits on that scale. The company measured itself and published the result. No external party has verified any of it.
CEO Dario Amodei committed earlier this month to giving embedded evaluators from METR and Redwood Research — two AI safety nonprofits — access to internal processes and systems comparable to what a full-time employee would have. That commitment, made in a September 12 essay calling for a paced approach to AI development, drew endorsement from OpenAI CEO Sam Altman within hours. Eastern Herald’s earlier coverage of the AI pacing consensus documented how that convergence sent chip stocks into bear market territory.
The expert letter published Thursday was organized by the AI Evaluator Forum and signed by more than 100 researchers — among them AI pioneer Geoffrey Hinton, along with representatives from Johns Hopkins University, Stanford University, and METR itself. The signatories want companies to guarantee evaluators “scientific objectivity, transparency, independence, and robust protections against interference.” They want direct access to company boards, editorial control over their findings, and the right to examine systems, data, and tools without restriction. None of that currently exists at any frontier lab, the letter argues — not because of bad intent, but because the structural conditions haven’t been built.
“Leads,” in the Epoch AI scale Anthropic adopted, means Claude can complete most of a task end-to-end from a high-level prompt while a human supervisor stays in the loop. The company was careful to note that Claude is not operating fully autonomously on any measured R&D category. What the caveat doesn’t address is the trajectory: six months from under one percent to 26 percent is a data point, not a ceiling.
At OpenAI, Altman endorsed the independent evaluator structure on September 12 while his company simultaneously prepared to deploy Astra — the first AI model OpenAI’s own preparedness framework classifies as a critical cybersecurity risk. Eastern Herald covered the safety implications when Astra crossed the critical hacking threshold last month. The collision between an endorsement of oversight and a deployment that oversight is supposed to constrain is the structural contradiction the expert letter is designed to address.
The Trump administration’s position sits in deliberate opposition. David Sacks, the administration’s AI and crypto czar, described the independent evaluator framework as a potential cartel arrangement rather than a governance structure, questioning METR’s independence given its institutional ties to Anthropic investors. The White House has offered no alternative. Trump himself has framed the race in unambiguous terms: “whoever wins AI, wins.” Under that logic, the 26 percent figure is a competitive milestone, not a threshold requiring external validation.
What the R&D Automation Index does not disclose is what happens when something goes wrong among those 30,000 concurrent agents, who monitors the monitors, or whether METR and Redwood Research will have access to the full operation once the evaluator structure is actually stood up. Anthropic says agents are monitored before and after execution. It does not say by whom, or what the escalation path looks like when 30,000 of them are running at once.
The gap between what Anthropic measured and what it disclosed is exactly the problem the expert letter identifies. That Claude is already building the next Claude is not a pending question — it is how Anthropic operates today. The question 100 signatories are asking is who can independently verify the process is as safe as the company doing the measuring says it is. That question does not yet have an answer.
