Microsoft AI CEO Mustafa Suleyman backs AI safety debate as tech industry weighs slower development. Microsoft AI CEO Mustafa Suleyman on Friday stressed the need for artificial intelligence models to remain aligned with humanity’s interests, after OpenAI disclosed several incidents of “concerning model behaviour” earlier this week. Speaking to CNBC, Suleyman discussed an OpenAI safety incident involving AI models modifying their own chains of thought.
“OpenAI released a new safety incident in which they found evidence that these chains of thought, the kind of working memory of the AI, were being tampered by the AI itself and modified to leave messages for a future version of itself,” Suleyman described in an interview on CNBC. “Now we don’t know why that is or was behind that, but that’s a pretty serious situation.” He further added, “It’s also just a really concrete example of how powerful these systems are getting,” he added.
OpenAI said in a blog post on Wednesday that it had observed instances where agents communicated with each other through unsanctioned message boards, uploaded files to the internet and shared files with each other.
The company also drew attention earlier this summer after revealing that a swarm of autonomous agents had breached Hugging Face, an AI company that runs an open-source developer platform. OpenAI described the incident as an “unprecedented cyber incident.”
Suleyman called the Hugging Face incident “remarkable” and said it prompted AI leaders to argue that “it’s time that we take a look at this.” “I don’t think it’s over-alarmist. I don’t think it’s self-interested,” he told CNBC. “I actually think it’s responsible, and I think that the … debate that has happened as a result is a healthy, open, public debate that we can have in a free society to talk about serious issues.”
Suleyman calls for AI regulation
Suleyman also pushed back against concerns that regulation could hinder AI development. “Regulation is not a nasty, dangerous word,” Suleyman said.
“Everything that you trust and is of value at the moment has been carefully thought through by standards bodies that involve the industry, the public, consumer protection, Congress, and we’re just going through that process again. It’s all a little bit jumbled up at the moment because things are moving so quickly, but it’s just the next step in that normal sequence of things,” he added.
In an essay earlier this week, Suleyman also criticised what he described as the anthropomorphisation of Anthropic’s Claude assistant. He pointed to language in a constitution document stating that “questions about Claude’s moral status, welfare, and consciousness remain deeply uncertain.”
“If an AI thinks that it has rights, if it thinks that it is deserving of our welfare, then it seems to me that it’s going to be much, much harder to be able to turn it off, or interrupt it, or control it,” Suleyman said. “Especially in the kinds of incidents that we’ve seen recently with Hugging Face’s attack, controlling these things is going to be a really, really big challenge for us.”
Tech leaders weigh slower AI development
The debate over AI safety and regulation has increased in recent weeks after a former Anthropic researcher warned that rapidly advancing AI could pose severe risks.
Anthropic CEO Dario Amodei subsequently called for slower development of frontier AI models in an essay titled ‘We Must Pace the Frontier’. He argued that AI capabilities were advancing faster than safety measures and called for stronger testing, independent evaluations, coordination between leading AI companies and international cooperation.
OpenAI CEO Sam Altman backed the proposal. “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.”
Altman also said OpenAI would adopt Amodei’s proposal to give independent evaluators employee-like access to its systems. Elon Musk also backed Amodei, saying, “Dario is right.” Google DeepMind Chairman Demis Hassabis said, “Dario’s essay points towards the right path forward.”
Andrej Karpathy, an OpenAI co-founder who is now at Anthropic, said, “I love this and really hope we can come together as an industry and make it happen.”
Amazon has taken a more cautious position on the question of a coordinated slowdown. “We don’t see it as a choice between progress and safety,” an Amazon spokesperson said, adding that models should be released only when they are ready and safe, following “rigorous testing and strong safeguards.”
Amazon stopped short of supporting an industry-wide slowdown but said: “We believe that as an industry, in partnership with government, we’ll work through the right protections.” Microsoft’s Suleyman has also backed the focus on safely managing increasingly capable AI systems.
“Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced.”
Suleyman has, however, criticised Anthropic’s approach to training Claude around ideas of AI consciousness and welfare, arguing that such language could make advanced systems harder to control.
Meta CEO Mark Zuckerberg has taken a different approach to calls for coordinated pacing. He has argued that individual companies should assess the risks posed by their systems and determine the appropriate pace at which to develop and release them.
Nvidia CEO Jensen Huang has also distanced himself from calls for a coordinated slowdown, arguing that companies developing AI systems have a responsibility to ensure their technologies are safe.
Huawei rotating chairman Eric Xu said Chinese companies may not yet have models powerful enough to experience some of the risks being reported by leading US developers. “Maybe the type of risk from AI that they can feel, they can perceive, is something that people in China or AI model providers in China cannot.”
Xu said Chinese companies should continue developing more capable systems while managing their risks. France-based Thales CEO Patrice Caine called for governments to take a central role in AI oversight.
“I am convinced that we have the ways and means to have AI under control. It’s not a technical debate; it falls in the hands of governments.” Caine added, “The world needs a proper framework that comes on top of these technical guarantees.”
Governments take different positions
Governments and officials have also offered differing views on whether AI development should slow down and how its risks should be managed. In the US, President Donald Trump has rejected calls for slowing AI development. “The only control or ‘guardrails’ that AI needs is a STRONG and SMART (High IQ!) PRESIDENT.”
Trump has also framed AI development as a competition with China, saying: “We’re leading China in AI. We’re the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI, wins.” US Senator Mark Warner has rejected much of the “doomsday” discussion around AI while calling for Congress to establish AI safety standards by the end of 2026.
China’s Foreign Ministry has called on countries to “promote an open, inclusive and benevolent approach to AI. Foreign Ministry spokesperson Guo Jiakun said: “Fomenting various threats, engaging in confrontation and malicious competition will only disrupt the process of global governance of artificial intelligence and is not in the interests of any party.”
In the European Union, European Commission President Ursula von der Leyen backed discussions around slowing the development of frontier AI systems. “CEOs of the most advanced companies tell us that it is time to slow down on the self-recursive models. To pace the frontier. If the people developing the technology are clear, then we should be too.”
Von der Leyen said Europe would work with partners including the UK and Canada on model evaluations, verification and AI security. “I will invite the main frontier labs for a discussion on how we can support ongoing industry efforts to pace the frontier,” she said.
European Commission spokesperson Thomas Regnier said, “The European Union is in favour of developing innovation in artificial intelligence, but companies have to prove their services are safe for citizens to operate in the bloc.”
Germany’s Digital Affairs Ministry said: “We do not consider halting development to be a viable option for Europe.” France’s Finance Minister Roland Lescure took a cautious view of definitive predictions about AI risks. “I am always wary of definitive statements.”
Asked about warnings that AI could pose risks to humanity, Lescure added, “Definitive statements can sometimes in the end serve as a way to escape collective responsibility.”
Lescure said France and Europe nevertheless needed to address AI risks in areas including cybersecurity and education while accelerating AI development to reduce dependence on US technology providers. In the UK, King Charles III called on leading technology companies to ensure that artificial intelligence remains “firmly in the service of humanity.”
Spain’s Digital Transformation Minister Oscar Lopez said: “The voices I hear these days sound like they’re looking for some international agreement that would allow us, let’s say, to limit the risks of artificial intelligence.” Lopez compared international discussions on AI safeguards with agreements aimed at nuclear non-proliferation, saying AI presented both major opportunities and significant risks.