The heads of some of the world’s biggest artificial intelligence companies told the United Nations on Wednesday that countries need to act now to make sure AI does not become too powerful to control.
Anthropic CEO Dario Amodei warned that poorly managed AI could become a threat to humanity, while OpenAI CEO Sam Altman said the world could “lose control of the future to AI.”
Both executives said governments need to put safeguards in place as AI continues to develop rapidly. The comments came during the UN Security Council, which is trying to work out how the technology should be controlled, who should control it and how quickly action is needed.
The two CEOs also said AI should not become concentrated in the hands of a single company or country. At the same time, they also spoke about the benefits of AI, with Altman saying the technology could help in “giving people more power over their own lives.”
UK wants global AI standards
British Foreign Secretary Ed Miliband said the warnings from AI industry leaders showed why governments could not leave the issue entirely to technology companies.
Miliband said AI regulation would be a major focus when the UK leads the G20 next year. He said Britain would push for a common set of global standards to make sure AI development remains safe.
But the United States took a different position.
White House Science Adviser Michael Kratsios said concerns about AI control should not be used as a reason to slow down its development or create a new global system to govern the technology.
“Worries about control of AI are ‘not a reason to pause its further development or to constrain it with new global governance structure,’” Kratsios said. “We want you, our allies and partners, to share in the benefits of this technology.”
The US and China are competing for leadership in AI. Chinese UN Ambassador Fu Cong said countries should be free to choose the AI technology they want to use.
Who controls AI in life-and-death situations?
UN Secretary-General Antonio Guterres warned that AI could create serious risks, including the possibility of machines making life-and-death decisions. “Let us resolve that life-and-death decisions must never be surrendered to machines. Killer robots must have no place in our future,” Guterres said.
Amodei called on countries to agree that AI should not be allowed to build biological weapons. He also urged governments to find ways to verify what AI models are capable of doing and to create common standards for testing AI systems, including tests for loss of control and misuse.
Hugging Face CEO Clément Delangue also addressed the Security Council through a remote connection that was affected by technical problems.
The meeting came a day after UN Secretary-General António Guterres warned about the threat of “killer robots.”
French Foreign Minister Jean-Noël Barrot, whose country organised Wednesday’s meeting, compared the current moment in AI to the early days of nuclear technology.
“Just as with the atom back then, the international community must now regulate AI in order to make the most of its potential and avoid the worst,” Barrot said.
He also compared the technology to HAL9000, the computer that goes out of control in the film “2001: A Space Odyssey.
Concerns over AI-powered weapons
The concerns were also raised by leaders from countries directly affected by war. Liberian UN Ambassador Lewis G. Brown II said countries could not afford to delay action.
“We must not allow the cost of today’s hesitation to become tomorrow’s catastrophe,” Brown said.
Ukrainian President Volodymyr Zelenskyy also warned about AI being used in warfare. Speaking at the UN General Assembly, he said humans must retain control over machines before it becomes too late.
“There is already a real possibility that AI, not only people, will begin to decide what happens on the battlefield. And we need peace before we reach that point,” Zelenskyy said.
Trump rejects global AI controls
Trump, however, took a very different position when he addressed the UN less than an hour after Guterres.
“The United States also totally rejects any attempt to construct a globalist scheme to control for the artificial intelligence being spoken of so much. Now here and after officially called super intelligence, changing the name,” Trump said.
Trump argued that AI should be seen as a major new technology rather than something that needs to be restricted through a global system.
“Every major new technology brings challenges, and superintelligence is no exception,” Trump said.
He then compared the current warnings about AI with earlier warnings about climate change.
“Yet the very same people who said we’ll all be dead in 12 years because of global warming- a name since reborn to climate change, because the planet was cooling, not warming, and nobody was dead. These are the same people who are now saying that AI is going to kill us all, that robots are going to attack us, and that everything is going to be a total disaster,” Trump said.
US-China AI race adds to concerns
The UN discussion came as Chinese President Xi Jinping was travelling to Washington for a closely watched state visit with US President Donald Trump.
US Treasury Secretary Scott Bessent said earlier that Washington and Beijing had discussed a possible new “notification mechanism” for AI incidents that could affect national security.
But Trump said on Tuesday that the US would reject efforts to control AI as it tries to stay ahead of China in the development of the technology. Trump described AI as “superintelligence.”
At the same time, countries and regional groups are working on their own AI rules. Europe, Asia and other regions are trying to coordinate their policies, while individual countries are developing regulations of their own. In the US, some states are also working on separate AI rules.
AI industry leaders to address Security Council
The debate will continue at the UN Security Council, where some of the biggest names in AI are expected to speak.
OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei and Hugging Face CEO and co-founder Clément Delangue are among those appearing before the council.
The three technology leaders have all been involved in the wider debate over AI safety and the need for safeguards. Delangue’s company, Hugging Face, was also recently hacked by a rogue OpenAI program.