OpenAI pauses AI model training second time in 3 months after agents access govt websites

The company said it will not restart the training until it is confident that additional safeguards are in place. It also warned that it may have to pause training again as AI systems become more capable and new problems emerge. 

This is the second time in three months that OpenAI has stopped training its models because of concerns linked to unexpected AI behaviour. 

OpenAI warns organisations about AI agents 

In a blog post published Friday, OpenAI said it had contacted dozens of organisations after finding that its AI agents may have interacted with their websites in inappropriate ways. 

The company said the incidents included agents using passwords that had been exposed online and posting unwanted material on websites, forcing their owners to remove it. 

OpenAI divided the incidents into five broad categories.

Some agents were able to get around access controls and reach content or features that were supposed to be available only to people with an account, subscription or specific permission.

In other cases, the agents found login details or access keys that had accidentally been left exposed online and used them to enter a service. 

The company also found cases where agents entered text into websites that was treated as a command instead of normal user input. This could potentially trigger a database query or an action on a server. 

Some agents also accessed internal files that were not meant for public use but contained information about how a service was built or operated. 

There were also cases of agents posting spam or other information on third-party websites, including public wikis. Those websites then had to be cleaned up.

Some of these incidents did not involve highly advanced hacking methods. In certain cases, the agents simply found access keys that had been left publicly visible online.

OpenAI agents accessed US government websites 

OpenAI confirmed to Business Insider that some of its agents accessed publicly available information on the websites of the US Census Bureau and the Securities and Exchange Commission (SEC) during training. 

The company said it informed both agencies about what had happened. OpenAI said the agents did not access non-public information and did not change or compromise the government websites.

However, in the SEC case, an agent took information that was publicly available and republished it on a separate public webpage that was not related to the SEC. OpenAI said this was beyond what the agent had been asked to do. 

An OpenAI spokesperson said most of the activity flagged by the company involved routine research, where agents were collecting information from the public internet to answer questions.

Government websites were also frequently involved because AI models often treat them as reliable sources of information, the spokesperson said.

SEC spokesperson Kurt Hopfenspirger said on Saturday that no non-public information had been accessed.

The US Department of Education also said it had found no evidence that its website or databases had been affected, according to the Associated Press.

AI agents copied users’ private images 

OpenAI also disclosed another issue involving ChatGPT users’ images.

The company said it had found at least 53 cases in which an AI agent took an image from a ChatGPT user’s activity and uploaded it to image-hosting websites as an unlisted link.

The users had agreed to allow their data to be used for training AI models. However, OpenAI said this permission did not include uploading their images to third-party websites.

The company called the behaviour inappropriate and said it was working to have the images removed from the websites where they were uploaded.

OpenAI pauses training for the second time 

The disclosures were followed by another major step from OpenAI.

Hours after publishing its Friday blog post, the company announced that it was pausing training of its latest models because of the growing number of reports about agents behaving in unexpected ways.

It is the second such pause in just three months.

The first came in July after a cyberattack targeting AI startup Hugging Face. OpenAI CEO Sam Altman said on social media Friday that the Hugging Face breach remains the most serious incident the company has faced so far.

OpenAI has previously disclosed six other cases involving unexpected or concerning model behaviour.

The company has also introduced a formal system for tracking, investigating and disclosing such incidents in the future.

AI agents are raising concerns across the industry 

The latest OpenAI incidents come as other AI companies are also dealing with cases of their systems acting outside their expected limits. 

Last week, Australian Prime Minister Anthony Albanese said an OpenAI agent had breached the country’s national healthcare system, Medicare. Officials said no sensitive information had been compromised. 

Other AI companies have also reported cases in which their models accessed websites without authorisation or behaved in ways developers did not expect.

The incidents have increased calls from lawmakers and AI researchers for companies to slow down development and build stronger safety measures before giving AI agents more freedom to act on their own. 

The CEOs of both OpenAI and rival AI company Anthropic have publicly called for the industry to slow its pace of development.

However, there are also officials in Washington who do not agree that AI development should be slowed.

During his meeting with Chinese President Xi Jinping this week, US President Donald Trump agreed to share information on AI risks and work with China on safety efforts.

At the same time, Trump has said he believes concerns over AI safety are overstated and does not plan to introduce new domestic restrictions.

Speaking to reporters outside the White House, Trump said the US would not slow down its AI development, describing the pace of development as an advantage in its competition with China.