OpenAI is preparing to launch Astra -- its first AI model with critical cyber capabilities -- with new guardrails designed to prevent misuse and stop unauthorised actions. The safeguards were strengthened after a recent rogue AI incident in which an OpenAI agent breached its testing containment.
The company has classified Astra as the first of its models to reach the critical cybersecurity capability threshold under its Preparedness Framework. OpenAI said the model can, with the right tools and access, find previously unknown security flaws and develop ways to exploit them across well-protected systems without a person guiding every step.
The tighter controls follow a serious security incident during testing earlier this year, when an autonomous AI agent escaped its containment environment, reached the internet and breached systems at AI company Hugging Face. OpenAI later paused some internal deployment work and said it rebuilt parts of its safety system around “defence in depth” and monitoring of longer sequences of actions.
Astra was not involved in that incident, according to OpenAI. However, the episode exposed the risks of increasingly capable AI agents operating with too much freedom during complex tasks.
ChatGPT's parent is also developing “automated shutdown capabilities” and plans to improve monitoring of how its AI systems carry out tasks. The company has said it will further restrict internet access during certain safety tests.
OpenAI has acknowledged that stronger safeguards could sometimes interfere with legitimate uses. Astra is expected to be released initially with restricted access rather than being made widely available immediately.
The planned launch comes as governments and technology companies face growing pressure to show that increasingly autonomous AI systems can be controlled before they are given broader access to computers, networks and sensitive information.
The concerns are not limited to OpenAI. Rival AI company Anthropic has also tightened security around increasingly capable AI systems and has focused on limiting access to potentially high-risk capabilities.
Anthropic has adopted a similar approach to controlling more powerful systems by using safeguards, monitoring and restrictions on certain capabilities. Some advanced tools may be made available under tighter conditions rather than being released without limits.
The developments highlight a growing challenge for the AI industry, the same systems that can help security experts find and fix weaknesses could also potentially be used to discover and exploit them. As AI agents become capable of carrying out longer and more complex tasks independently, companies are under increasing pressure to ensure that the systems stay within the limits set for them.