Measuring the real impact of development initiatives

  • December 10, 2013
Measuring the real impact of development initiatives

marlen1

Sometimes major international governance schemes begin simply: with a glass of beer, and questions of how to measure the making of it. Dolo, a slightly-alcoholic beverage, is made primarily in Burkina Faso by boiling sorghum malt and water. The boiling is the problem. The primary source of fuel across the country is firewood and in many areas more than half of the available supply goes into making dolo. To make more energy available for other uses, there are two obvious choices. Get more firewood or use less.

Unsurprisingly, the latter has become the mantra of a number of development actors. The largest is the United Nation’s Global Alliance for Clean Cookstoves, launched in 2010. By 2020, the initiative aims to bring energy-efficient stoves to 100 million of the 2.7bn households that rely on firewood or charcoal for cooking. The solution seems simple; give people efficient stoves and they will use less firewood, thereby reducing the depletion of natural resources. In addition, the stoves mitigate the risks associated with cooking on open fires, which cause about four million deaths annually, making it a more potent killer than malaria or tuberculosis.

Last summer, I was sent to Burkina Faso to find out whether energy-efficient stoves delivered on their promise. I assisted a research institute in their evaluation of the initiative Foyers Ameliorés de Burkina Faso (FAFASO).  FAFASO is part of the EnDev programme, financed by a multitude of international stakeholders and executed by the German Society for International Cooperation (GIZ) in over 20 countries. FAFASO works with dolotières, masons, potters, and whitesmiths regarding the construction and dissemination of energy-efficient stoves in the country.  As such, the project provides a platform for exchange to the people who have expertise and a stake in the production of dolo and encourages them to work together to develop and implement the new stoves.

My purpose in Burkina Faso was to answer one question: Are the dolotières who use the energy-efficient stoves using less firewood? What I found was an answer to that question – they use about 25 per cent less – and another, much larger question: What is the project’s impact?

Lost narratives

While in Burkina, I spoke to over 200 dolotières. Looking at my notes after I left, I reread the stories they had told me about their businesses, lives, plans, hopes and much more: Florence, who became involved in her local dolotières association when she purchased the stove, or Marie who could not find a mason to repair the stove for her or Sandrine who complained that customers doubted the quality of the dolo she brewed on the new stoves. I wondered why their narratives did not seem to have a place in the impact evaluation.

Since one size rarely fits all and individual stories can overpower the common experience, evaluators often prefer to let numbers tell the story. But when evaluating a project that is based on information and experience sharing such as FAFASO, traditional impact evaluations may only be able to take us so far. FAFASO’s impact may reach beyond the reduction of biomass usage, as the stories of Florence, Marie and Sandrine suggest. In offering channels for communication between groups, the project may have a profound effect on dolotières’ emancipation, local supply chains or customer behaviour, to name just a few areas.

I therefore think it may be time to reconsider the gap between the impact that development initiatives can have and the impact that evaluations measure. One way to start may be by letting people targeted by development initiatives speak just as loudly as numbers and statistics.

*Marlen de la Chaux (2013) is currently pursuing a PhD in Management Studies. The picture does not depict an energy-efficient stove.

Latest Blogs

How Cambridge Analytica influenced Nigeria’s elections

My research broadly tackles questions of technology, equity, and accountability within the Global South. I emphasise the West’s use of sub-Saharan Africa as a testing ground or laboratory for its nascent technologies before launching them in the West. This blog is an excerpt from my recent presentation at the 2022 Gates Day of Research, titled […]

Why algorithms are necessarily value-laden

Algorithmic decision-making systems applied in social contexts drape value-laden solutions in an illusory veil of objectivity. Machine learning plays an increasingly prominent role in mediating institutional decisions in everything from corporate hiring practices to criminal sentencing. This ongoing AI spring has invigorated discussions of the ethical dimensions of these techno-social arrangements. In particular, there is […]

Preparing for all scenarios in an unstable world

On March 9th, Mohamed A. El-Erian joined the Gates Cambridge community for a virtual fireside chat, where he discussed decision-making in conditions of uncertainty, the economic impact of the pandemic and relief efforts and the importance of diversity of thought and scenario planning. El-Erian is President of Queens’ College, Cambridge and Chief Economic Advisor of […]

How can the international community help Belarus?

Last Sunday represented a tipping point in the recent history of Belarus which has had an immediate effect on the lives of its citizens, including mine. Independent exit polls and observers representing the diplomatic community, verified by the crowdsourcing platform Golos, show that, had it been a fair and transparent election, the uninterrupted, 26-year-long reign […]