By Kunal Walia & Rositsa Zaimova
Santosh, a small farmer in Uttar Pradesh, is eagerly awaiting harvest season. He recently acquired an artificial intelligence (AI)-powered farm advisory device that was distributed in his village as part of a pilot by a global agri-tech firm. The device runs on a software that assesses weather, soil, and water conditions to predict crop patterns. Having proved useful for farmers in the American Midwest, the technology has generated great excitement in this UP hamlet. A week before harvest, however, Santosh finds that his fields have been destroyed by pests endemic to the region. The software, he learns, is built on data based on North American topography. Santosh now has to rely on informal jobs in the nearest city to make ends meet till the next sowing season.
This scenario illustrates the dangers of positioning AI technologies as transformative without ensuring the accuracy and reliability of the data on which it is built. In March, the European Union approved the Artificial Intelligence Act, which could help address such concerns — including in sectors that have a direct impact on people’s lives and livelihoods such as agriculture, healthcare, or education. The pioneering legislation defines four risk levels for AI and poses strict obligations on “high-risk” systems, which include ensuring quality data sets to minimise discriminatory outcomes and biases.
The approval of the law is timely, given that we are increasingly seeing off-the-shelf AI solutions that are trained on western and non-diverse data sets being deployed in developing countries. The new regulation — and the impetus it offers other regions to draft their own laws — is a useful backdrop against which AI developers and impact institutions could work together to reverse this trend.
Assessing AI bias
Thinking back to Santosh’s predicament, if the inaccurate data used by the tech firm had been detected early on or the AI advisory had been supplemented with locally relevant information, it could have helped him avert disaster. In order for this to happen, people and organisations building AI systems need to introduce an additional layer of testing for bias assessment. Fortunately, there are several frameworks they can choose from.
For instance, retrieval-augmented generation assessment or RAGA, coined by researchers from Facebook AI Research, University College London, and New York University. Retrieval-augmented generation refers to a group of large language model (LLMs) applications that use external data. They have two components: retrieval, the extraction of information from external contexts; and generation of a response based on that information.
Another popular framework, recall-oriented understudy for gisting evaluation, or ROUGE, uses human output as a metric. This is primarily useful for LLM applications that generate text-based output such as summaries or translations.
Equally important is establishing regular feedback loops through human oversight — what is known as reinforcement learning from human feedback or RLHF. This is indispensable as it allows AI systems to get better at mimicking human output over time so it can perform more layered tasks. Such AI bias assessment frameworks need to be rigorously applied by all stakeholders that have a hand in creating AI tools. Further, productising these frameworks could introduce an even greater degree of compliance within the private sector.
But if well-developed solutions already exist, why is uptake not up to the mark?
Funding support
Tools that aid rigorous stress-testing of AI models before deployment require large-scale investment. While comprehensive data on AI funding is limited, we know the private sector tends to hold back from investing in areas that do not have demonstrable proof of return on investment. Impact institutions such as philanthropies and international development organisations can play a key role in bridging the funding gap.
Social impact institutions are increasingly engaging with AI innovations. For instance, USAID supports GeoKRISHI, a web application that integrates government, geographical information system, and crowdsourced data to provide evaluations of crop suitability for key stages. It has also invested in Dimagi, which has an open-source platform combining mobile health data and machine learning-based satellite image analysis to identify and forecast Zika virus hotspots. Similarly, the World Bank Group is backing Project Artemis, which is building an AI model to forecast famine risk at a local level.
While this is promising, the scope of support needs to expand to include risk and bias assessment. Dalberg analysis of 10 leading global philanthropic organisations showed that only 23% of the grants towards AI are directed towards addressing concerns around risks and ethics. With a greater push by social impact institutions and the right incentives for the private sector, we can ensure that AI systems become more inclusive of the people they seek to serve. And that the conversation driven by a handful of large institutions in the Global North do not impact the course of action in the Global South.
Kunal Walia & Rositsa Zaimova are partner, Dalberg Advisors, and partner, Dalberg Data Insights, respectively.
Disclaimer: Views expressed are personal and do not reflect the official position or policy of FinancialExpress.com. Reproducing this content without permission is prohibited.