‘Narrative that frontier AI gives threat actors advantage is flawed’: Daryl Pereira, Google Cloud

How do companies safeguard themselves against AI-led cyberattacks without access to the most advanced models?

AI is expanding the attack surface area by a great deal. Autonomous agents are interacting with data, the entire system and environment within the company. Organisations can’t just focus on access anymore. They have to check the data provenance and employ strong governance structures. Trust has become more important as we continuously validate it in an AI-first world… But the narrative that attackers typically possess an inherent advantage because of raw, advanced frontier models is flawed. Defenders also hold the advantage of knowing the operational context of a company which a generic AI model doesn’t have. AI models have to work blindly across multiple targets that are diverse across sectors like banking, telco or IT. We believe building and testing a well-engineered harness that is fine-tuned to the environment is actually more powerful. We can also use smaller, specialised models that don’t consume too many AI tokens and that are more precise as a defence against threats.

What are the newer kinds of cyberattacks that will become common?

The biggest concern around frontier AI models is that they can scan and identify previously unexploitable or unknown vulnerabilities and then chain these exploits…binding them together to form a ‘super exploit.’ In a bid to fight this, we are relying on an AI security agent built by Google DeepMind called CodeMender, that scans our codebase and then writes a proof of concept to attempt and exploit security flaws within a secure sandbox to check its susceptibility. If there is a gap, the tool generates a high fidelity security patch around the code.

How does India fare in cyber resilience globally?

I don’t think India lags behind other developing economies in the area. There are just a vast number of firms that operate within complex environments in such a large country. There’s a strong institutional footing and world-class digital foundations built here. India has robust tracking and monitoring with good responsiveness frameworks like CERT-In. The challenge is scale. Billions of transactions are done every month over platforms like UPI making it one of the largest digital attack surfaces. The gap needs to be bridged in the skill and localisation context. While the public infrastructure is resilient, many Tier-2 companies and private enterprises suffer from a steep shortage of AI security operations talent. However, this is also present in first world economies like Japan. I would advise these firms to pivot from trying to build an AI-driven defence on their own to a cloud-based platform.

How do we ensure proper governance over AI agents to prevent any rogue incidents?

Developing AI must have a proper governance framework and ideally an AI committee internally. You also need to ensure the right guardrails are in place…. Generally, there are three tenets to implementing guardrails. Firstly, keep your AI agents in isolation. Secondly, have semantic boundaries so the agent doesn’t interpret any hidden or unstructured data as executable command. Thirdly and very importantly, we need to maintain a human-in-the-loop checkpoint. Any kind of high-risk tasks like big financial transfers, deployment of code into production, modifying infrastructure firewall rules or even bulk deletion of enterprise data must strictly be gatekept by a human authority that has been cryptographically authorised.

Has the zero trust architecture become obsolete in the AI era?

I don’t think that’s true. Rather, in my opinion, its more critical than ever and is probably our best line of defence against AI-led attacks. Because of the speed at which AI detects vulnerabilities, traditional security frameworks are essentially primitive now. Firewalls don’t work anymore.

Zero trust principles offer the least leeway, do rigorous microsegmentation of the network and crucially conduct continuous posture assessment. A mix of these ingredients prevent any kind of automated malware from moving across the network. Additionally, zero trust identity verification has to be extended beyond human users…to silicon identities like agents. The zero trust model also has to evolve as agents evolve. We need to use short-lived cryptographic tokens and authorise every single microtransaction executed by agents to be secure.