Part 2 - What can AI see?
Cost – understanding compute
As we’ve explained previously, a key aspect for ensuring that AI solutions provide value in international development contexts is cost. Operating in resource constrained contexts, there must be a clear case that the value-add being provided by AI outweighs the cost of developing a tool.
The model developed by Rara Labs has substantially reduced the cost of validating errors in the NWASH data. In an experiment conducted on a standard workstation, the model made predictions on 200 samples in less than 30 seconds, whereas an expert required over 30 minutes for the same task. This efficiency enables experts to allocate their time to strategic resource planning. However, it's crucial to evaluate whether the upfront costs of developing AI solutions are justified by the reduction in costs facilitated by the model, taking into account both development and maintenance expenses. It is also critical to consider any wider benefits from the AI solution, when evaluating the case for investing in development costs. In the case of this pilot, the AI provided clear additional benefits, in terms of improving the accuracy of the data in NWASH, which could justify development expenses.
The cost of developing AI solutions is a function of many things: the hardware needed to gather data and run the model, the skills of software developers and subject matter experts, and the computational resources needed to train and maintain the model.
For the early warning forest fire system, a big challenge was creating a cost-effective way to collect the data which the model would then assess. In this use-case, the data was readily available, and the main cost consideration was compute.
“there must be a clear case that the value-add being provided by AI outweighs the cost of developing a tool…”
What is compute, and why is it so important to developing AI use cases?
When any application is run on a computer it requires computational resource (storage, processing power, memory) to function. Compute is a measurable quantity of the computational resource used to perform a specific task. The storage of training data, fine-tuning of systems, and day to day use of AI systems, all requires compute to function (AWS, n.d.).
There are broadly two ways in which this compute can be accessed:
-
Organisations can purchase their own physical servers, which can be used to store data and process applications which make use of that data.
-
Alternatively, companies like Amazon, Google and Microsoft allow organisations to rent computational resource, on demand, via the internet (Michal, 2024).
As Rara Labs explained to us, the cost of compute is highest during the training phase. The process of finetuning an off the shelf algorithm to a specific use-cases requires a significant amount of compute to process. Additionally, if your fine-tuned model fails to achieve your intended outcome, you have to go back to the drawing board and re-start the training process. However, once you have built the model, the cost of compute needed to run it on a day-to-day basis reduces.
As we’ve explained previously, creating a use-case which is well suited to the context and capable of delivering high levels of accuracy over time is a continuous process. The training data doesn’t provide an immutable foundation for a tool which can be readily adapted to changes in the demands of the real-world use-case, but rather a snapshot of a specific time. As the context changes, new data needs to be collected which can represent those changes and feed them into the structure of the system. A Mckinsey study exploring 100 hundred cross-sectoral use-cases found that at a least a third of use-cases required a monthly update of data, and just under one in four use-cases, required a daily refresh (Chui et al., 2018).
So, while the cost of compute is most intense during training, dropping off after the model has been developed, there are still compute costs which come with the long-term optimisation of the model, through continual training and adaptation of the solution.