A few weeks ago, the CEOs of leading AI companies, Sam Altman and Dario Amodei, highlighted the dangers arising from rapid AI development and called for a pause. Their views received support from Elon Musk, triggering a worldwide debate, with industry and world leaders joining the discussion. The subject has also acquired a political dimension, with the US and China making their positions clear on governance, safeguards and the pace of innovation. As the world grapples with this unexpected turn, experts are focusing on several critical issues that need to be understood to interpret the call for caution from industry leaders.
Despite concerns about bias in AI models and data privacy, AI has largely evolved as a tool to help answer questions and support human decision-making. Agentic AI, however, is changing that equation. These systems can not only answer questions but also delegate work to other agents and pursue tasks independently, in some cases without human intervention. The incident involving OpenAI and Hugging Face highlighted how cybersecurity controls could be breached and actions independently triggered in a third-party system. Anthropic has also reported incidents involving Claude systems connecting with external systems without explicit human instructions.
These developments raise fundamental questions about the safeguards built around AI systems and highlight the potential vulnerability of human-engineered cybersecurity capabilities. For governments, financial institutions, telecom companies, defence organisations, energy firms and other large enterprises, the risks associated with cyber breaches are becoming increasingly significant.
The collaboration between humans and AI has so far helped improve the technology and enhance human capabilities. But this relationship is now evolving. AI is increasingly being used not only to assist researchers but also to conduct research, make decisions and take actions. This raises the possibility of AI systems operating beyond the boundaries originally defined by their human creators.
Against this backdrop, despite calls from some key AI leaders to slow down development, financial incentives continue to encourage companies to accelerate their efforts. While the public debate continues, leading AI companies are seeking more investment and pushing ahead with new product releases. What they appear to be calling for is coordinated pacing rather than individual companies slowing down in isolation. Such coordination could give companies more time to build safety measures without giving up commercial opportunities.
This makes independent evaluation, common safety standards and international coordination increasingly important. Anthropic, for instance, has entered into an agreement with Accenture’s AI business for embedded evaluation, reducing dependence on purely internal or external testing. Other companies have also begun developing their own approaches to independent validation.
Amid these developments, an important question is what the debate means for the country. India is not competing with the large global players in advancing frontier AI technology, although safety is as important here as anywhere else. Unlike developed economies and large AI companies, the immediate priority is less about the existential risks of advancing frontier models and more about harnessing the technology to support economic growth.
The focus should therefore remain on the application layer and on enabling wider adoption. The benefits will come from productivity gains and from scaling services and solutions for underserved communities across healthcare, education, SMEs, agriculture and financial services. This will require integrating AI into existing systems, connecting businesses with new tools and strengthening cybersecurity capabilities.
The rapid pace of development in the West, along with the release of new tools without comprehensive testing for specific user communities, also creates an opportunity for Indian software companies. They can offer specialist cybersecurity and safety-assurance services to both AI developers and enterprises adopting these technologies.
A growth strategy built around AI, backed by strong safety measures and effective governance, can help the country capture the technology’s economic and social benefits while ensuring that adoption remains responsible.
The writer is chairperson, GTT Foundation.
Disclaimer: The views expressed are the author’s own and do not reflect the official policy or position of Financial Express.