Measuring the real impact of development initiatives

  • December 10, 2013
Measuring the real impact of development initiatives

marlen1

Sometimes major international governance schemes begin simply: with a glass of beer, and questions of how to measure the making of it. Dolo, a slightly-alcoholic beverage, is made primarily in Burkina Faso by boiling sorghum malt and water. The boiling is the problem. The primary source of fuel across the country is firewood and in many areas more than half of the available supply goes into making dolo. To make more energy available for other uses, there are two obvious choices. Get more firewood or use less.

Unsurprisingly, the latter has become the mantra of a number of development actors. The largest is the United Nation’s Global Alliance for Clean Cookstoves, launched in 2010. By 2020, the initiative aims to bring energy-efficient stoves to 100 million of the 2.7bn households that rely on firewood or charcoal for cooking. The solution seems simple; give people efficient stoves and they will use less firewood, thereby reducing the depletion of natural resources. In addition, the stoves mitigate the risks associated with cooking on open fires, which cause about four million deaths annually, making it a more potent killer than malaria or tuberculosis.

Last summer, I was sent to Burkina Faso to find out whether energy-efficient stoves delivered on their promise. I assisted a research institute in their evaluation of the initiative Foyers Ameliorés de Burkina Faso (FAFASO).  FAFASO is part of the EnDev programme, financed by a multitude of international stakeholders and executed by the German Society for International Cooperation (GIZ) in over 20 countries. FAFASO works with dolotières, masons, potters, and whitesmiths regarding the construction and dissemination of energy-efficient stoves in the country.  As such, the project provides a platform for exchange to the people who have expertise and a stake in the production of dolo and encourages them to work together to develop and implement the new stoves.

My purpose in Burkina Faso was to answer one question: Are the dolotières who use the energy-efficient stoves using less firewood? What I found was an answer to that question – they use about 25 per cent less – and another, much larger question: What is the project’s impact?

Lost narratives

While in Burkina, I spoke to over 200 dolotières. Looking at my notes after I left, I reread the stories they had told me about their businesses, lives, plans, hopes and much more: Florence, who became involved in her local dolotières association when she purchased the stove, or Marie who could not find a mason to repair the stove for her or Sandrine who complained that customers doubted the quality of the dolo she brewed on the new stoves. I wondered why their narratives did not seem to have a place in the impact evaluation.

Since one size rarely fits all and individual stories can overpower the common experience, evaluators often prefer to let numbers tell the story. But when evaluating a project that is based on information and experience sharing such as FAFASO, traditional impact evaluations may only be able to take us so far. FAFASO’s impact may reach beyond the reduction of biomass usage, as the stories of Florence, Marie and Sandrine suggest. In offering channels for communication between groups, the project may have a profound effect on dolotières’ emancipation, local supply chains or customer behaviour, to name just a few areas.

I therefore think it may be time to reconsider the gap between the impact that development initiatives can have and the impact that evaluations measure. One way to start may be by letting people targeted by development initiatives speak just as loudly as numbers and statistics.

*Marlen de la Chaux (2013) is currently pursuing a PhD in Management Studies. The picture does not depict an energy-efficient stove.

Latest Blogs

How COVID-19 holds a mirror up to our societies

Being a Chinese student at a British university in the middle of the coronavirus pandemic provides an interesting perspective on our different societies’ approach to COVID-19, says Jingwen (Alice) Fan.

Towards a better, more global approach to traumatic brain injury

In 2014 Hafsa Durrani tragically lost both of her brothers in a terrorist attack at Army Public School and College in Peshawar, where the terrorists executed 148 people in the school’s auditorium – 132 students, all the teachers, a lab assistant, a guard and the principal. Five years on, Hafsa, who is from a Pashtoon […]

Private money interests at the heart of Hong Kong protests

Another round of resistance was recorded over the weekend between protesters in Hong Kong and the Chinese government. Organisers of the previous Sunday’s pro-democracy rally estimated that 1.7 million Hong Kongers poured into city streets. Much of the news commentary depicts a David vs. Goliath standoff in which Hong Kong can only lose. Yet this view overlooks an […]

Female voices on climate change

During last week’s panel discussion on “Female Voices on Climate Change” at the 2019 Hay Festival, an audience member asked: How can we make people care about climate change? “That’s the million-dollar question,” quipped the panel chair, journalist Rosie Boycott. Indeed, it is the million-dollar question. But while there’s no easy answer, this Hay Festival […]