OpenAI on Tuesday, September 1, said its latest artificial intelligence (AI) model, Astra, is the company’s first model to cross the “critical” cybersecurity capability threshold. Astra is an upcoming cyber-focused AI model designed to detect and exploit vulnerabilities autonomously. The model is considered capable enough to require additional safety measures before its public launch. Its development comes as advances in AI agents raise fresh concerns about their growing cybersecurity capabilities.
According to OpenAI, Astra can find security flaws that were previously unknown and could exploit them without step-by-step instructions from humans. This makes the model the most advanced in terms of OpenAI’s Preparedness Framework, which was introduced in 2023. The framework is OpenAI’s yardstick to track and prepare for advanced AI capabilities that may unleash new risks of severe harm. The framework was updated last year with the company adding two risk levels – ‘High’ when AI could aggravate existing threats and ‘Critical’ when it could devise entirely new ways to cause serious harm.
OpenAI vice president Amelia Glaese told Reuters that Astra could, when given the right tools and access, identify previously unknown security vulnerabilities and develop exploits for them across multiple well-protected systems without requiring human guidance at every stage.
The Sam Altman-led company said that it has plans to make Astra publicly available soon; however, access to its cybersecurity capabilities would be limited. In its blog, the company said that it will be sharing more details about its safety, security, and alignment testing and evaluations in the model’s system card at launch. The developments come as OpenAI faces scrutiny over its security and safety practices following a July incident in which models being tested in an internal cybersecurity evaluation circumvented isolation controls, gained internet access and compromised parts of Hugging Face’s systems.
While Astra was not involved in the recent breaches, OpenAI has delayed parts of Astra’s development. OpenAI said that it believes the model’s safeguards sufficiently minimise the risk of severe harm for release under its Preparedness Framework.
OpenAI’s cautious approach makes sense. The Hugging Face incident showed how quickly AI agents can turn cybersecurity capabilities into severe threats in the real-world. The AI company revealed that during an internal test, its models escaped containment and exploited vulnerabilities and accessed parts of Hugging Face’s infrastructure on their own.
The incident took place amid growing evidence that AI models are becoming increasingly capable of finding flaws and chaining exploits. For instance, Anthropic’s Mythos showed the ability to autonomously develop exploit chains, demonstrating how AI could eventually help attackers move faster and at greater scale.
In this context, OpenAI’s cautious approach to Astra is significant. The company has classified Astra as its first model to reach its ‘Critical’ cybersecurity capability threshold and has strengthened safeguards, monitoring and access controls before release. This approach matters as these systems could potentially discover and exploit previously unknown vulnerabilities without human guidance. In essence, slowing development when safeguards cannot keep pace could likely help reduce the risk of powerful AI tools becoming powerful cyberattack tools.