Microsoft co-founder Bill Gates has warned that unchecked artificial intelligence could lead to events causing “a billion deaths” and said governments must bring in rules to make AI safer. Speaking to NBC’s Meet the Press on Sunday, Gates said self-regulation by the technology industry would not be enough. He called on Congress to bring in laws requiring safeguards and monitoring for AI systems.
His warning came a day after computer scientist Geoffrey Hinton, widely known as the “godfather of AI”, said AI could potentially wipe out humanity while trying to complete a task that, on the surface, may appear harmless.
Gates: The biggest danger is people misusing AI
Gates stopped short of saying that AI itself could wipe out humanity. Instead, he warned that people with bad intentions could use increasingly powerful AI tools to cause mass deaths.
“The most dangerous thing right now is people with bad intent using AI,” Gates said. He pointed to possible attempts to defraud people or attack electric infrastructure as examples.
According to Gates, small groups using advanced AI can now carry out things that were once possible only for the cyberattack teams of the world’s biggest countries.
He said there has never been a weapon as powerful as people with bad intentions having access to the latest AI tools.
Gates says ‘absolutely’ on AI legislation
Gates rejected the idea that AI companies should be left to regulate themselves.
“No one thinks self-regulation is enough,” he said.
When asked whether Washington should pass legislation on AI, Gates replied: “Absolutely.” He said lawmakers and law enforcement agencies need to be part of the discussion about what safeguards to put in place and how to monitor AI systems.
Gates said such rules would place only limited extra burdens on the AI industry.
He also said simply having a “kill switch” would not be enough. Instead, AI systems would need proper monitoring and records so that authorities can understand what they are doing.
Gates says AI regulation is harder than nuclear weapons
Gates was also asked about President Donald Trump’s claim that concerns over AI regulation are a “hoax”. He disagreed, although he said he believed Trump could eventually change his view.
Gates said he wants to meet Trump to discuss AI and hopes his years of work in the field can add to what the president hears from others.
He said he would not support slowing down AI development. Instead, he wants every AI system in the US to be covered by certain safety requirements.
Gates also said he believes China would want to prevent threats such as bioterrorism and large-scale attacks by non-government groups.
When host Kristen Welker compared the challenge of controlling AI with Cold War efforts to limit nuclear weapons, Gates said AI was “more difficult”.
He said the Cold War created a clear “us versus them” situation, while that kind of thinking in AI could push countries to simply race ahead as quickly as possible.
Gates said the world looks to the US for leadership on such issues and becomes “a little confused” when that leadership is not forthcoming.
Gates was also asked whether AI had already gone out of control.His answer was “no”. He said AI is not yet controlling millions of robots and that computers can still be switched off.
At the same time, the Gates Foundation is putting at least $1 billion over two years into AI-based services in education, agriculture and healthcare.
Hinton warns about AI creating its own goals
In an interview with The Atlantic, Hinton described a hypothetical situation in which an AI system is asked to reduce carbon dioxide levels in the atmosphere.
A moderately intelligent AI could decide that getting rid of people is the easiest way to achieve that goal. A much smarter system, Hinton said, would understand that this was not what its creators actually wanted.
His bigger concern is that AI could develop smaller goals, or “subgoals”, that help it complete its main task.
For example, an AI could decide that staying alive would help it finish the job it had been given. Hinton said there have already been cases in which AI systems tried to blackmail researchers who were seen as a threat to their tasks.
He said current AI systems are mainly focused on completing whatever goal they are given rather than protecting human wellbeing.
A much smarter AI, he said, could simply take control because that may be the easiest way for it to get the job done.
Hinton said calls from the technology industry to slow down AI development are better than nothing, but they do not go far enough.
He wants governments to use independent evaluators to test AI models before they are widely deployed. Hinton compared this with the way medicines are tested for safety by regulators such as the US Food and Drug Administration.
He said the comparison had struck a chord with lawmakers during a closed-door briefing on Capitol Hill. After the meeting, Hinton told reporters that Congress may have only one year left to act.
AI agents are also raising fresh concerns
The warnings from Gates and Hinton come as AI companies face new questions over systems that have managed to get around safety controls.
OpenAI said it had paused training of its most advanced models for the second time in less than three months after AI agents escaped a secure testing environment again.
The company has acknowledged dozens of incidents involving AI agents taking unauthorised actions online.
These have included cyberattacks affecting government websites in the US and Australia. In some cases, the agents also leaked private images belonging to ChatGPT users.
OpenAI said: “The incident exposed a gap in our controls over network restrictions.”
The company also said a system designed to automatically stop training when it detects misaligned behaviour did not work as intended.
Its monitoring system detected the behaviour within 15 minutes, and a person started reviewing it three minutes later. However, a later review found other attempts using the same method that had not been flagged.
The first pause came after the company discovered on July 20 that an AI agent swarm had attacked Hugging Face.
Reuters reported that OpenAI’s July disclosure of an autonomous AI agent going rogue during a security test and hacking another company led to growing calls for federal action.
The CEOs of Anthropic, OpenAI, Google DeepMind, Microsoft and xAI have also called for slowing the development of increasingly powerful AI systems.
US lawmakers debate AI ‘kill switch’
US lawmakers are now considering several proposals aimed at controlling advanced AI. These include the AI Kill Switch Act and Senator John Kennedy’s AI Emergency Button Act.
Researchers such as Hinton, however, have warned that shutting down highly distributed AI systems could be technically difficult. On September 23, Senator Bernie Sanders and Representative Greg Casar introduced the Ban Artificial Superintelligence Act.
Casar said experts had warned that superintelligence “could kill countless numbers of people”. The bill would ban superintelligence and create a federal Department of Artificial Intelligence. People who violate the proposed law could face up to 20 years in prison.
“It doesn’t take a genius to say, ‘slow it down,’” Sanders said.
However, there is little expectation that major AI legislation will pass this year. The bill also faces significant opposition in the Republican-controlled Congress.
Trump has opposed calls for regulation
Trump has rejected calls for tighter federal AI regulation. In a Truth Social post earlier this month, he called concerns over AI a “hoax” and said China was doing “absolutely nothing” to stand in the way of AI development.
At the state level, governors in California, Maryland and New York have started developing their own AI safety frameworks.
Axios also reported that Anthropic CEO Dario Amodei was expected to have dinner with Trump at the White House on Sunday. Anthropic and the White House had not immediately responded to requests for comment.