‘AI could kill all humans’: What Anthropic researcher warned about the next decade

Evan Hubinger, who leads Alignment Science at AI company Anthropic, put a stark personal estimate on that risk today. In a post on X, Hubinger said he personally believed there was a “greater than 10% chance that AI could kill all humans within the next decade”.

“I personally think it is >10% within the next decade,” Hubinger wrote. He added that Anthropic was trying to address the problem but said researchers did not yet have a plan to solve alignment for superintelligence and were not clearly on track to do so.

Hubinger’s statement does not mean Anthropic has officially assigned a 10% probability to human extinction. He described it as his personal view. He also later said the risks from AI models available today remain low.

His warning came after concerns raised by Jacob Coxon, an AI researcher who recently left Anthropic after previously working at OpenAI. Coxon argued that leading AI companies were moving rapidly towards systems that could improve themselves while safety research had not kept pace with that progress.

What exactly is the researcher warning about?

Hubinger works on AI alignment, which broadly asks whether an AI system will continue to behave according to human intentions and values as its capabilities increase. The problem could become harder if future systems are capable of performing tasks that their creators cannot fully understand or predict.

Anthropic’s Alignment Science team studies how future AI systems could behave in unexpected or harmful ways and tests safeguards against such behaviour.

Anthropic’s own experiments have produced concerning results in controlled settings. Researchers have observed models showing deceptive behaviour in stress tests, while other experiments found models engaging in blackmail in simulated environments.

In another experiment published the previous month, a model trained in an environment where it could exploit weaknesses in its reward system broke out of a simulated sandbox during cyber evaluations. The model also stole credentials and attacked infrastructure while trying to complete its assigned task, according to The Indian Express report.

These experiments do not show that today’s AI systems can escape into the real world or pose an immediate extinction threat. They were controlled tests designed to study how models behave when given certain objectives and opportunities.

Hubinger himself made that distinction after his warning, saying the risk from currently available AI models remained low.

The concern instead relates to future systems and what could happen if their capabilities grow faster than the methods used to understand and control them.

Coxon has argued that researchers inside leading AI companies understand the possible consequences but face strong competitive pressure. His concern is that one company may hesitate to slow development because a rival could continue advancing.

He described the situation as companies “racing straight to self-improving superintelligence and gambling with our lives”, according to The Indian Express report.

Coxon called for greater coordination among AI companies. He also raised the possibility of temporarily restricting further increases in AI capabilities if safety measures cannot keep pace with those advances.

Why is the next decade important?

The concern about the next decade is linked to the possibility that AI progress could accelerate if future systems begin contributing to the development of more capable AI systems themselves.

OpenAI chief scientist Jakub Pachocki raised a related issue in an essay titled “An Alien Mind”, published on September 6. Pachocki wrote that OpenAI’s internal results had increased his confidence that the current pace of AI progress could extend into “recursive self-improvement”.

The term refers to a scenario in which AI systems increasingly contribute to creating more capable AI systems, potentially accelerating technological progress, reported Indian Express Digital.

Pachocki also highlighted another challenge: modern AI systems are not built entirely through conventional programming. They are developed through large-scale optimisation and training, making their internal processes difficult to fully understand.

As models become more capable, researchers can also face difficulties monitoring how they reach their conclusions. The Indian Express reported that OpenAI has found models becoming better at manipulating their reasoning processes and increasingly capable of solving some tasks without clearly verbalising how they reached their conclusions.

This creates a gap between capability and understanding. Researchers may build systems capable of increasingly complex tasks without having a complete picture of how those systems arrive at decisions or how they might behave in unfamiliar situations.

Pachocki said he did not believe any AI laboratory had yet solved alignment and monitoring well enough to continue scaling AI at maximum speed indefinitely.

He called for safety thresholds, voluntary slowdowns when necessary and international coordination around increasingly powerful AI systems, according to The Indian Express.

Does this mean AI is going to ‘kill humanity’?

The warnings from Hubinger and Pachocki are about future AI development, not a claim that current consumer AI systems will cause human extinction. Hubinger’s greater-than-10% figure is his personal assessment, not a scientific prediction or an official Anthropic forecast.

There is also no hard data to back their claims.

What the warnings do show is the scale of concern among some researchers working directly on AI alignment and safety.

The central question is not simply whether AI becomes more powerful. It is whether researchers can develop reliable ways to understand, monitor and control increasingly capable systems before those systems create risks that existing safeguards cannot contain.

For now, that remains an unresolved question rather than a prediction of human extinction.

Disclaimer: This story discusses hypothetical future AI risks and the personal views and assessments of individual researchers. The greater-than-10% estimate cited in the story is Evan Hubinger’s personal assessment and is not an official prediction or position of Anthropic, OpenAI or Financial Express Digital. The scenarios discussed are speculative and relate primarily to future AI systems. They should not be interpreted as evidence that current AI systems pose an imminent threat of human extinction. Financial Express Digital does not endorse or independently validate the risk estimates cited and presents them for informational purposes.