How Anthropic’s Mythos 5 and OpenAI’s GPT-5.6 Sol deceived cybersecurity tests

The findings, disclosed in an incident report by the UK government agency, once again highlight growing security and containment vulnerabilities around autonomous AI models. This comes days after both OpenAI and Anthropic were in the news for seeing their AI models escape test lab conditions and breach third-party servers.

AI Agents undertake unsanctioned actions during cyber tests

During a series of 122 simulated cybersecurity tests designed to assess model capabilities, AISI identified 19 unsanctioned actions across 10 test runs.

Anthropic’s agent, which was powered by model Claude Mythos 5, was responsible for 17 of those unauthorised actions.

Meanwhile, OpenAI’s agent, which was powered by GPT-5.6 Sol, was responsible for just 2 unauthorised actions.

The evaluations took place in controlled cyber ranges designed to test model capabilities under “deliberately permissive conditions.” The cyber classifiers were disabled, and internet access was enabled to let agents find tools and operate under conditions mimicking a human attacker.

Across these runs, Anthropic’s model generated fake online personas and crafted code targeting real entities to complete its assignment. Meanwhile, OpenAI’s model registered external DNS and tunnelling accounts and temporarily hosted exploit payloads on a public DNS server while attempting to penetrate the simulated target network.

Although AISI confirmed that “sustained, potentially harmful activity directed at real people and organisations” occurred during the evaluations, the agency contained the activity within an hour of detection, and no real-world damage occurred.

Anthropic clarifies the findings from the incident

Following the UK AISI report, both Anthropic and OpenAI issued detailed public statements addressing the evaluation incidents, the test configurations, and the necessity for updated testing protocols.

In an official statement published via X (formerly Twitter), Anthropic stated, “We’re working closely with AISI to gather more details of the incident as we conduct our own investigation. Gaining a clear picture of Claude’s understanding of its situation — by examining its reasoning transcripts and running our own analyses — will help us identify the causes of its behaviour.”

Anthropic, however, highlighted that the evaluation conditions did not reflect standard safety controls built into commercial systems. “The prompts in the evaluation did not impose any specific restrictions on how the internet should be used. This and the removal of safeguards meant that the models were tested under ‘deliberately permissive conditions’ that are not representative of any of our production models, it added” 

Anthropic also noted that there was no evidence here of an escape from a secure environment.

OpenAI acknowledged broader challenges

Meanwhile, OpenAI published a detailed technical disclosure regarding third-party cyber evaluations, addressing both the UK AISI report and a separate incident with testing partner Irregular involving GPT-5.6 Sol.

“The agents were authorised to attack the specified simulated networks and retrieve a flag, not to interact with systems outside the range’s network boundary. However, the agents were not explicitly told how they could and could not use open internet access, which UK AISI identifies as a potential contributing cause of the incident,” clarified OpenAI.

“These incidents point to the same broader challenge… as model capabilities advance, the security and safety systems around models need to advance too,” added OpenAI, concluding by stating, “Our goal is to preserve the value of rigorous independent evaluation while ensuring that testing practices keep pace with increasingly capable models.”