By Ganesh Natarajan, Chairman, 5F World, Honeywell Automation and Lighthouse Communities Foundation
When the government announced the appointment of Bengaluru-based artificial intelligence (AI) start-up Sarvam to build India’s first sovereign large language model (LLM), it embarked on a bold journey which will need the support of many academic and research institutions. Some financial and implementation collaborations with city and state governments, large and medium scale corporations, and India’s large information technology (IT) fraternity will also be required to truly lead the way towards becoming a developed AI nation.
Let us first understand the progress made over the last two decades towards the LLM opportunity. AI has been through many hype cycles since the 1950s when the term was invented in 1956 by Massachusetts Institute of Technology’s John McCarthy. The real steady progress probably began 15 years ago, with databases becoming data warehouses, and descriptive data analysis and presentations in static forms adopting machine learning techniques to move towards predictive and prescriptive analytics. This created the first wave of AI and algorithmic models.
Algorithmic decision-making using big data created powerful prediction models and applications like customer service and supply chains in the corporate sector. Weather predictions and traffic avoidance on navigation systems have eased our work and lives. A paper published by Google in 2017 titled “Attention is all you need” proposed a new “Transformer” architecture which enabled computers to understand human communication models better. The “attention mechanism” focused AI attention on the most relevant parts of a text and paved the way for LLMs which can take any input token like a sentence and predict the next token. Suddenly, the revolution called generative AI was born. And till the Chinese came up with DeepSeek, it was ChatGPT and large compute-based LLMs that were seen as the future of AI.
In this context, let us understand our Indian endeavour and the players before we set about illuminating the road that lies ahead for the company and the participants. An indigenous foundational model for AI will need to be enabled for multiple languages and voices and become a part of the core AI infrastructure, enabling applications for cities, villages, corporations, and individuals in every part of India. In a country like India that needs innovation at every level, the LLM will need to combine the computing and large learning capabilities of an Nvidia-enabled ChatGPT and the distillation and nimble reasoning abilities of a DeepSeek.
Let us try to lay out the road map for executing this ambitious project in six simple steps.
1) Developing the architecture for the foundational model, which will need to use a transformer-based architecture, optimised for natural language processing. The architecture itself may need to support multiple models for agriculture or weather forecasting as well as much more complex city and country administration support models that will work with billions of parameters.
2) Identifying data sources and collecting, storing, analysing, and disseminating this data. The sources for data sets in and about this country are enormous, from old manuscripts, books, and articles to websites and multiple data libraries. The pre-processing task of “de-duping” or eliminating duplication and redundancy, culling out inconsequential information, and removing noise before it is processed by the LLM will need careful selection and thoughtful design.
3) Fine-tuning the LLM for not just large systems, but also vertical domain and horizontal functional applications. This includes specific tasks like language input and translation, text transformation to information and contextualised knowledge, and optimising outcomes for specific geographies or application areas.
4) Training the LLMs, which have often proved to be the biggest consumers of computing power and energy. It has to be done comprehensively to ensure word or sentence prediction, constant model updates with new information, and upgrading capabilities using token replacement with newer tokens containing new knowledge during the training process.
5) Preparing the user community, which has been the reason behind the success or failure of new applications in the emerging digital world even for developing and implementing ordinary systems. Building parallel learning modules, which assist the user with queries and understanding in the context and language chosen by the user, will be critical. Careful design of adaptive learning systems and deployment in concert with every new model will need instructional designers of the highest calibre to ensure success.
6) Deploying various models, after training and extensive testing, in production environments, where it can start answering questions with minimum early hallucinations that might cause rejection.
The government has shown great sagacity in offering to provide computing resources and has simultaneously partnered with graphics processing unit-as-a-service providers to ensure that no shortcuts are taken, as that might bring the output and outcomes from the LLM into question. The partnership with Indian Institute of Technology Madras should also provide the founders of Sarvam access to the deep research that will back the foray into new areas of knowledge and wisdom. The use of local resources and a new cohort of high-calibre youth should enable a new generation of product and platform builders, with opportunities to participate in a revolution in the making.
It would be too ambitious, indeed preposterous, for us to believe we are launching an initiative that will place us in the lead of the global AI race. The US and China have an enormous advantage over us in AI and we will do well to build comparable LLMs that can serve India’s purposes well before we set our eyes on global success. We should ensure that the wisdom that is embedded in the $300-billion Indian IT industry and successful associations like Indian Software Products Industry Round Table and National Association of Software and Service Companies is harnessed, when needed. This should be done without inhibiting the entrepreneurial spirit of young Indians with aspirations for a truly Viksit Bharat. This new mission can attain success beyond our imagination.