Using LLMs as a tool for International Development professionals

THE QUESTION

Can AI systems that retrieve and synthesise data support international development practitioners with common evidence-based tasks?

---

LOCATION | United Kingdom
SECTOR | Evidence/Evaluation
TECH | AI/Big Data
TIMELINE | August 2023 - Present
PIONEER | Seb Mhatre, Robert Phillips
PARTNER | Baobab Tech

The Challenge

International development organisations both publish and fund the publishing of tens of thousands of documents every year. 

Most people working in international development only have the time to read a tiny fraction of the documents that are potentially relevant to their work. This means that it is normally the case that there is a considerable amount of information that would have been useful for a decision that an international development professional is making or a task they are performing that wasn’t used. 

We see an opportunity to leverage emergent capabilities of Large Language Models (LLMs) to tackle three key problems facing potential users of publicly available information on the International Aid Transparency Initiative (IATI) database:  

1. Volume of information: The volume of evidence that advisors are tasked with summarising is large and growing, even within sub-fields.   

2. Information management: There are challenges around accessing and finding evidence sources that are relevant to a particular task 

3. Groupthink. Groups who are using and assessing evidence to make decisions, have a similar professional background and approach issues in a similar way.  

The impact of these issues is that people cannot be comprehensive in their work which relies upon evidence, whether that be an evidence review of a partner multilateral’s performance in a certain area, or in the design of a business case for a programme.

The Idea

This pilot aims to explore how a Large Language Model can help international development professionals to do their work by making it easier to use knowledge stored in the huge number of publicly available published documents. The pilot will support international development professionals including, but not limited to FCDO to find relevant knowledge by providing prompts and suggestions, synthesising available sources, and presenting information in forms useful for decision making. This will result in development professionals spending less time on searching for the high-quality evidence they need and thus, better evidence-based decision making and increased development impact. 

Following initial focus groups to identify promising use cases and tasks, we have developed an advanced Retrieval-Augmented Generation (RAG) prototype that can retrieve information from the FCDO documents within the IATI dataset, such as annual reviews, programme completion reviews, evaluations and business cases. We aim to learn what best practice looks like when using interactive LLM systems and to create a collaborative approach and community with other international development organisations.

---

Our learnings and stories so far