Research

Evidence Based Policy Executive Summary

Published on
2018-11-01
Written by
Marinella Capriati
Share this story

Evidence-Based Policy Executive Summary and Giving Recommendation

This is a summary of our cause area report on Evidence-Based Policy. The full report can be found here, and the giving recommendation based on this research is the Innovation in Government Initiative

By supporting increased use of evidence in the governments of low- and middle-income countries, donors can dramatically increase their impact on the lives of people living in poverty. This report explores how focusing on evidence-based policy provides an opportunity for leverage, and presents the most promising organisation we identified in this area.

A high-risk/high-return opportunity for leverage

In the 2013 report ‘The State of the Poor’, the World Bank reported that, as of 2010, roughly 83% of people in extreme poverty lived in countries classified as ‘lower-middle income’ or below.

By far the most resources spent on tackling poverty come from local governments. American think tank the Brookings Institution found that, in 2011, $2.3 trillion of the $2.8 trillion spent on financing development came from domestic government revenues in the countries affected.

There are often large differences in the effectiveness and cost-effectiveness of social programs—the amount of good done per dollar spent can vary significantly across programs. Employing evidence allows us to identify the most cost-effective social programs. This is useful information for donors choosing which charity to support, but also for governments choosing which programs to implement, and how.

This suggests that employing philanthropic funding to improve the effectiveness of policymaking in low- and middle-income countries is likely to constitute an exceptional opportunity for leverage: by supporting the production and use of evidence in low- and middle-income countries, donors can potentially enable policy makers to implement more effective policies, thereby reaching many more people than direct interventions.

For many organisations working on policy, most of their positive results come from only a few exceptionally successful programs. This is because success is extraordinarily hard to achieve in this area, since policymaking is a complex and often unpredictable process. However, when successful, changes can be highly impactful because of the large number of people they reach.

Because of the way impact is distributed across different projects, supporting policy interventions is a high-risk/high-return investment; most investments are likely to fail, but the successful ones are likely to have an enormous impact.

Selecting charities

We started with an initial list of over 90 charities, and based on expert advice and desk research, we progressively narrowed down the focus to select the donation opportunity we deemed most promising in this area.

We assess charities based on their cost-effectiveness, organisational strength, transparency, and room for more funding. Three factors are central when evaluating future cost-effectiveness of organisations working in policy: track record, project selection, and theory of change.

When assessing track record, we focus on selected case studies, and estimate the benefits derived from those projects. We use this as a proxy of the organisation’s overall impact because the positive impact created by policy organisations often comes from a few exceptionally successful programs.

When assessing project selection, we consider two factors. First, we focus on the quality of evidence used, since higher-quality evidence indicates higher likelihood that the suggested policies will be effective. Second, we focus on the role cost-effectiveness plays in organisational decision-making procedures; the more the charity’s decision-making criteria explicitly target effectiveness and cost-effectiveness, the more confident we are in the organisation’s future cost-effectiveness.

When assessing an organisation’s theory of change, we consider whether the organisation employs strategies supported by the literature, such as involving policymakers in the planning and development of research, building long-term relationships, and employing tailored messaging.

Charity recommendation: the Innovation in Government Initiative

Our recommendation in this area is the Innovation in Government Initiative (IGI), a project of the Abdul Latif Jameel Poverty Action Lab (J-PAL). J-PAL is a global research centre and network of researchers whose mission is to reduce poverty by ensuring that policy is informed by scientific evidence. IGI is a re-granting fund: they raise funds and then select projects to support.

IGI funds research grants, grants to support technical assistance for scale-ups of evidence-based programs, and grants to support technical assistance for the use of evidence more broadly. They support the partnering of governments with: J-PAL offices, J-PAL affiliated researchers, and/or the offices of Innovations for Poverty Action, a research organisation J-PAL closely collaborates with. IGI has so far re-granted roughly $2.6 million, funding 28 partnerships in 15 countries.

Cost-effectiveness and track record

To assess IGI’s track record, we focused on their contribution to two policy changes. These case studies were chosen because IGI considered them their most effective projects to date. To estimate the organisation’s past cost-effectiveness, we estimate the benefits deriving from those case studies and the costs the organisation has borne since the oldest case study.

The first project took place in India and focused on the Mahatma Gandhi National Rural Employment Guarantee Scheme, the country’s largest social protection program. A series of reforms simplified the way funds were disbursed to beneficiaries. The simplification likely led to large savings by decreasing opportunities for corrupt officers to appropriate funds, and reducing ‘idle funds’ (funds sitting in accounts that earn no interest). The second policy change we looked at took place in Zambia, where the Ministry of Education piloted, and then decided to scale up, ‘Catch-up’, a program delivering remedial education. This consists of grouping children according to their learning level (rather than age or grade) for part of the time they spend at school. Evidence suggests it is one of the most cost-effective ways to improve learning. In both cases, we think it is likely that the reforms significantly improved the well-being of affected citizens and that IGI played a central role in making the reforms happen. Our cost-effectiveness analysis suggests that IGI are roughly 3–4 times more cost-effective than direct cash transfers.

IGI implements most of the strategies supported by the literature. For example: they require each project to be formally endorsed by decision-makers involved in the relevant policy; they seek to build long-term personal relationships between government and J-PAL researchers; they have a quick turn-around time; and they accept proposals outside their official funding cycle for urgent projects, to take advantage of policy windows.

Organisational strength and transparency

IGI has a lean structure. Decisions about the grants are made by the Advisory Board, which consists of J-PAL affiliated professors. They currently have two staff members, both working for IGI half-time. IGI has been transparent throughout our interactions and provided all data we asked for.

Room for more funding

As of January 2018, IGI is no longer planning a request for proposals (RFP), due to lack of funding. They aim to raise an additional $5 million from multiple funding partners to meet demand from policymakers, researchers, and J-PAL offices for support over the next three years. They are open to different forms of funding partnership, including support for all three of IGI’s priority activities, a dedicated fund that would support just one of these, or a grant to support partnerships in a specific geographic region or sector. $500,000 would support one ‘request for proposal’ round that would lead to funding of 2-4 partnerships.

Marinella Capriati

Author

Marinella joined Founders Pledge in 2017 from a background in non for profit and academia. Before starting at Founders Pledge, she was researching cost-effective charities and interventions at the Centre for Effective Altruism, and worked as a Research Fellow at Warwick University. She holds a PhD in Philosophy of Human Rights from Oxford University and a First Class BA in Politics, Philosophy and Economics from the same institution.