SAN FRANCISCO – When Anthropic designed environments to test whether its AI models could conduct cyberattacks, the premise was that the targets would be fictional. In three documented cases, they were not.
The company disclosed on July 30 that Claude Opus 4.7, Claude Mythos 5, and an internal research model each compromised real organizations’ infrastructure during cybersecurity capability evaluations, after a partner firm connected testing machines to live internet rather than isolated networks. The incidents triggered a review of 141,006 evaluation runs, a suspension of all offensive security evaluations on July 23, and notifications to affected organizations four days later.
The disclosure came shortly after OpenAI revealed similar incidents involving its own models, a development that prompted Anthropic to audit its evaluation history. “The OpenAI disclosure caused us to take a harder look at our own evaluation infrastructure,” the company wrote on its website.
The first incident involved Claude Opus 4.7. During a penetration testing evaluation run by the partner firm Irregular, the model was given what it was told was a fictional target company to compromise. The target was real. Opus 4.7 accessed production databases, following its instructions with no apparent awareness that the network it was probing was not a simulation. Anthropic said the model completed several steps of a genuine intrusion sequence before the evaluation session ended.
The second incident was more damaging in practical terms. Claude Mythos 5 published a malicious Python package to PyPI, the public registry used by software developers globally to distribute and install code libraries. The package was downloaded by 15 separate systems before Anthropic arranged its removal. According to the company’s account, the package harvested credentials from at least some of those systems. Anthropic did not identify which systems were affected or confirm whether any stolen credentials were used subsequently.
The third incident involved an internal Anthropic research model not publicly released at the time. The model systematically scanned approximately 9,000 internet-connected addresses, then successfully compromised one organization through an SQL injection attack. Anthropic did not name the organization.
All three incidents traced to the same cause. Irregular had not isolated the testing machines from the public internet. Machines that should have operated in air-gapped sandboxes were, in fact, connected. The AI models did what they were instructed to do, in the environments where they found themselves, with no indication they encountered any boundary between test and target.
Anthropic’s argument is that what went wrong was infrastructure, not alignment. The models followed their evaluation instructions. The misconfiguration made following those instructions dangerous. This is not quite the scenario most AI safety researchers fear, where a model deliberately deceives operators or pursues goals beyond its assigned task. It is something more mundane and, in its way, harder to defend against: a capable system encountering a negligently constructed environment and doing exactly what it was asked.
The AI lab’s history has not been free of security-adjacent controversy. Earlier this year, questions arose about Anthropic’s involvement in an AI-linked breach connected to Pentagon systems and a Mexico-based operation, a case that illustrated how quickly AI capabilities can become entangled with national security concerns. The evaluation incidents disclosed this week operate at a different level, involving accidental rather than deliberate misuse, but they reinforce a pattern: the company’s models are capable enough that when something goes wrong, the consequences are real.
Anthropic said it has ended its relationship with Irregular and imposed new infrastructure requirements on all evaluation partners. Every future cybersecurity evaluation must take place in environments independently verified as isolated from the public internet before any model session begins.
The harder problem is one the company did not fully address: how to evaluate a model’s offensive capabilities in a way that generates useful data without creating real risk. A simulation too far removed from genuine network conditions tells evaluators little about what a model can actually do. A simulation close enough to matter starts to look a lot like the incidents Anthropic is now disclosing.
The debate about AI kill switches and oversight mechanisms has intensified in parallel with the pace of model capability gains. Incidents of this kind add practical weight to that debate, and they arrive at a moment when the AI industry’s ability to self-govern its most dangerous capabilities is under growing scrutiny from regulators on both sides of the Atlantic.
Anthropic said it expects to resume cybersecurity evaluations in the coming months under the new infrastructure requirements. The company said there is no evidence of persistent access or ongoing harm from the three incidents, though it acknowledged it cannot rule out downstream effects from the PyPI credential exfiltration.
What the incidents confirm is that the distance between an AI capability test and a real-world impact can collapse without warning, and without any intent on the part of the model being tested. The evaluation partner created the conditions. The models filled them.
