Ethiopia, the IMF, and the PSNP
On the importance of social safety nets, and of the evidence of their importance
As I set foot in Ethiopia — an extraordinary country steeped in history and now my new family home — the International Monetary Fund (IMF) Board approved a four-year USD 3.4 billion Extended Credit Facility (ECF). This facility is designed to support Ethiopia’s reform agenda, address macroeconomic imbalances and debt sustainability, and 'lay the foundations for higher, inclusive, and private sector-led growth.' As a development economist specialising in impact evaluation, I know I cannot claim any credit for this milestone… as correlation does not imply causation 😊.
My background in impact evaluation also tells me it will be crucial to gather evidence on the impacts of this financing package on Ethiopia’s socioeconomic development and poverty reduction efforts. The ECF aims to support the Ethiopian government’s own Homegrown Economic Reform (HGER) programme by boosting private sector activity and economic openness, with the ultimate goal of achieving more inclusive growth. The focus on inclusivity is also reflected in the intention to mitigate the negative consequences of the planned reforms on vulnerable households by strengthening social safety nets and increasing the budget allocated to the Productive Safety Net Programme (PSNP), a cornerstone of Ethiopia’s social protection framework since its launch in 2005. It is telling and indicative of the journey the IMF (and other Washington Consensus institutions) have been on since the days of the 1990s Structural Adjustment Reforms, that the Fund’s Deputy Managing Director, Ms Antoinette Sayeh, explicitly mentioned in her statement announcing the agreement that ‘continuing to widen the reach and impact of the social safety-net programmes will be important.’
The importance and positive impacts of social safety nets have been evidenced over the years by a growing body of literature and evaluations across different countries and continents, including Africa. Social safety nets have emerged as pivotal mechanisms in the quest to alleviate poverty, improve resilience, and enhance economic stability. These programmes are critical in cushioning vulnerable households and individuals against shocks such as natural disasters and health crises but also economic downturns, thereby preventing them from falling deeper into poverty and helping them invest in their human capital and productive assets.
An excellent example of a well-designed and rigorously evaluated programme (and not only because I was part of the evaluation team 😊) is Kenya's Hunger Safety Net Programme (HSNP), which provides regular cash transfers to households in the arid and semi-arid lands of northern Kenya. The HSNP has been instrumental in reducing poverty and vulnerability, enhancing the resilience of households to shocks, and contributing to overall economic development. The HSNP evaluations, including quantitative quasi-experimental estimations of impact, qualitative research on households and communities, as well as local economy analysis, have fed into the design of the subsequent phases of the programme, improving its targeting and implementation processes while emphasising the policy potential of well-designed social safety nets in alleviating poverty and fostering resilience.
Similarly, within a context characterised by inflationary pressure and persistent instability in some areas of the country, Ethiopia’s PSNP will have to play a central role in mitigating any potentially negative consequences of the significant economic reforms contained in the HGER programme supported by the IMF’s ECF arrangement (a form of economic shock, albeit one aimed at achieving inclusive growth and development). Evaluating the impact of the PSNP will be equally important.
Existing evidence shows the PSNP’s potential for reducing poverty and enhancing food security. Previous studies have found that the PSNP can improve household consumption and stabilise food intake, also during periods of drought and economic stress. More recent evidence covering the 2016-2021 phase 4 of the PSNP points toward geographical disparities in the type and magnitude of impacts achieved, such as a reduction in food gaps and an increase in livestock assets in the Highlands but no impact in the Lowlands. There is also little evidence of any meaningful levels of graduation taking place. However, that was a period characterised by a series of negative events in Ethiopia, including successive droughts, political unrest, the COVID-19 pandemic and even a desert locust invasion in the harvest season of 2020.
The task at hand is to ensure that credible and comprehensive evidence is now gathered on the PSNP’s role in smoothing Ethiopia’s renewed and supported path towards poverty reduction and inclusive growth. A rigorous evaluation will be key.
Based on my experience evaluating HSNP and many other cash transfer programmes (e.g. Evaluation of the Child Grant Programme – CGP – in Lesotho), the evaluation of the current phase of the PSNP should include, as a minimum, the following components:
A revised and up-to-date Theory of Change to capture the multiple channels through which the PSNP is expected to achieve its impacts, whilst also accounting for the current contextual factors and assumptions, such as the expected fluctuations of the currency given the new forex regime, and how this may affect inflation and the purchasing power of the vulnerable population. Crucially, this will also allow for a revision and refinement of the Evaluation Questions, which should always be at the heart of any evaluation design.
A counterfactual-based estimation of impact to provide confidence on both (1) the exact measure of impact detected for each outcome indicator; and (2) whether any impact detected is attributable to the PSNP. Considerations related to statistical power/the size of the quantitative samples should be paramount so as to avoid underpowered samples that prevented previous PSNP evaluations from disaggregating the impact analysis across sub-groups and categories of interest.
In-depth qualitative interviews and focus group discussions both at the household and community levels to unpack questions around the impact (or lack thereof) of the PSNP on wellbeing, thus integrating and enriching the narrative emerging from the quantitative findings, within a fully mixed-methods framework from the design stage through to the analysis and reporting stages.
A detailed process evaluation review to both track the fidelity of the PSNP implementation to its original plans, assess the viability of implementation mechanisms and determine the feasibility of further programme scale-ups. This seems particularly important given the problems flagged by previous evaluations on the regularly and predictability of the cash transfer payments as well as the stated aim to improve and increase the degree of digitisation of the programme.
An integrated Value for Money (VfM) analysis to provide a more comprehensive VfM assessment of the PSNP. Previous VfM analyses have tended to rely heavily on qualitative information, mainly due to the lack of relevant quantitative data at the household level. However, mixing quantitative (both administrative data and household survey data) and qualitative evaluation sources would enable the integration of different metrics of social, economic, environment, and cultural values. The Value for Investment approach ideated and developed by Julian King and others at Oxford Policy Management and Verian would be the right starting point for rethinking a thorough and insightful VfM assessment of the PSNP.
Rigorous and relevant findings produced through a robust mixed-methods impact evaluation built on the fundamentals sketched above would not only contribute to the debate on the role of safety-nets as critical tools for shock mitigation and sustainable development. They would also feed into the design of the next phase of the PSNP, thus reinforcing the foundations of Ethiopia’s social protection framework.
