Part 3 - What can AI say?
Problem
How do you make evidence-based decisions when the evidence base far exceeds what you can process? Understanding how to make sustainable change to people’s lives, mitigate the damage of climate change and build a fairer, more equitable world is a significant challenge. There are no easy metrics to indicate success. Evidence is crucial. By building on the lessons learnt by organisations seeking change, whether that be grassroots community-led initiatives or international programmes spanning years, evidence gives us a foundation to inform and guide our efforts.
In large donor agencies like USAID, the challenge of leveraging evidence to inform their work is immense. With a huge portfolio of programmes and an ever-increasing body of internal evidence, plus an international community of researchers generating valuable external evidence, USAID (and many other development organisations) have found that there is often more evidence out there than they can be expected to process.
What can we do to approach this problem, and where might AI be able to help?
The idea
While there certainly isn’t an AI silver bullet to this challenge, there are ways in which we can leverage the growing capability of LLMs to start addressing the problem. Working in close collaboration with USAID, DevelopMetrics - a software company working at the intersection of international development and advanced AI - built their Development Evidence Large Learning Model (DEELM).
DEELM is a multi-purpose LLM which collates a number of USAID databases of reports, programme assessments and resources into a searchable dashboard, which users can sift through and interrogate.
The tool has several use cases, which the team co-designs with different departments in the agency. To understand the idea behind the tool, we’ll focus on their work with the Innovation Technology and Research (ITR) department at USAID. Collaborating with ITR, they set about answering the question: ‘What is every digital intervention USAID has done throughout its portfolio?’
Drawing on USAID databases of information, they pulled together a dashboard which was categorised by outcome and technology. Users could explore which technologies had been deployed to address specific outcomes, say blockchain to improve governance outcomes, and read through a comprehensive list of USAID interventions in that space. The dashboard includes specific excerpts from the reports and a quantitative, peer-reviewed assessment of the intervention’s success, from which they created a ranking system.
The dashboard is being used by ITR to auto-generate reports for Congress, summarising USAID’s work on certain technologies, and what they’d learnt.