The best AI model is usually the smallest one that is capable of getting the job done, according to Andi Gutmans, VP for Data Cloud at Google. Gutmans tells Poulomi Chatterjee, that Google is using a combination of agentic as well as a humans-in-the-loop approach which differentiates it from the competition. Excerpts:
Q. As AI moves to agentic phase, how can enterprises ensure data quality?
A. One of Google’s biggest advantages is that we have the full-stack, which includes the model, data platform, AI infrastructure. So, we have a common security model throughout the stack starting from the agent till where the data is stored. We’re trying to ensure the full stack is secure and that there’s fine-grained access control in terms of governance.
Q. Is human-in-the-loop a foolproof system?
A. We use a combination of agentic as well as a humans-in-the-loop approach which is different from what the competition is doing. Human-in-the-loop is difficult to do constantly, 100% of the time, because you can’t hire enough data stewards or Forward Deployed Engineers (FDEs) to curate all your data. But we definitely work with customers in regulated sectors like core banking who want the human-in-the-loop element. We may infer the knowledge but clients can still decide if they want a human in the system. In some sectors, they are happy to be almost fully autonomous.
Q. How do enterprises control costs as AI agent usage grows and modernisation cycles become shorter?
A. Customers are very conscious and all our conversations with them usually revolve around ROI, how to increase revenue, improve the bottom line, or reduce risks. Tokens and model choice is an important part of this. The best model is the smallest one that gets the job done. Models like Gemini Flash are very impactful because it’s advanced while being much cheaper than Gemini Pro or other bigger models. Additionally, context is vital because the better and more focused it is, the fewer reasoning loops and lesser tokens consumed.
Our differentiation and Knowledge Catalog is very refined at exposing the context that really matters while a lot of other vendors put in a lot of context into the model and make it generic. Thirdly, spending doesn’t just consist of tokens but also compute. This is where our differentiated, disaggregated compute and storage architecture allows us to drive down costs for customers even further. We also collaborate with DeepMind to lower costs dramatically for some functions. The cost can be 100 times cheaper per customer than an actual large language model.
Q. How is adoption for Google’s Agentic Data Cloud which uses AI agents to manage a database?
A. We’re seeing a lot of success helping customers manage legacy systems with AI agents because in many instances they don’t have as much talent available anymore. There are also AI agents that can modernise legacy systems successfully now. Different paradigms exist where agents can help business users, data scientists or data engineers reimagine how they work with AI. This is not in the AI assistant, copilot arrangement which the last couple of years were about. More proactive agents are really working on the practitioner’s behalf. This is also resulting in much shorter timelines for modernisation.