Rethinking evidence-informed policy and scaling: lessons, challenges, and opportunities

31 January 2025

Evidence is essential for policymaking and decision-making. It informs decisions—but it also legitimises actions, frames issues, and defines priorities. However, its influence is often overestimated. Even if used, evidence is not always the “right” evidence, nor is it always used in the “right” way. Even when the evidence is robust, systemic resistance, political interests, and operational challenges often prevent its full integration into policy.

This article is based on notes I prepared for a session at the Friends of Education in-person meeting in Switzerland in January. It explores the complexities of evidence-informed policymaking and scaling, drawing lessons from global contexts and specific examples such as Peru’s MineduLab. By rethinking traditional models, addressing systemic resistance, and integrating political, social, and ethical considerations, we can better align evidence use with impactful policy outcomes.

This short article should (or can) be read alongside these two resources:

The role and limitations of evidence in policy

Evidence plays many roles in policymaking. It can be instrumental in informing decisions or serve to legitimise actions already taken. It is often used to frame issues, explain problems, raise awareness, or define priorities. Evidence and its use are not politically neutral. In fact, evidence-based policymaking emerged as a new political narrative in the late 1990s.

Hence, evidence-informed policymaking is not as straightforward as it may seem. Policymakers often use evidence selectively—choosing what aligns with their objectives or values—and sometimes dismiss evidence entirely (which is a form of use).

Narratives play a critical role in explaining why and how evidence is used.

And use is not always upward. Before the term “evidence use” was in fashion, there was “research uptake”. Then, and now, the main focus was on policymakers using evidence. This limits how we understand the challenge. Uptake can be side-take (e.g. other researchers and experts) and down-take (e.g. the public, NGOs, grassroots). (And no-take.)

Moreover, there is a disproportionate emphasis on “gold-standard” evidence like Randomised Controlled Trials (RCTs) and systematic reviews, which are often unnecessary for routine decisions and do not have the impact they promise (this paper from 2014 and this one from 2024 make the very clear case for the very low influence that these evaluations of impact have on decisions). In most cases, practical tools like administrative data and the “cross-tabs” that junior research assistants are familiar with suffice.

However, across the Global South, most policymakers (supported by or part of weak civil services) don’t even have access to this.

Importantly, using evidence does not automatically mean being well-informed. To be informed, policymakers must consider findings from multiple studies, weigh political implications, incorporate personal values, and navigate public interests to make decisions. This is starkly clear when studying policy debates—rather than policy decisions.

Finally, evidence informs decisions, but values and interests ultimately guide them. The scientific method, however robust, can only tell us what has happened or may happen. It cannot tell us what to do.

Decisions and actions are firmly in the space of values.

Scaling and evidence

Scaling is a critical process in policymaking, whether it involves expanding resources, delivering services to larger populations, increasing intervention coverage, deepening interventions, transforming good ideas into national policies, etc. Traditionally, scaling has been seen as a linear process of doing more of the same. However, transformational scaling focuses on achieving sustainable, long-term goals, not just growing budgets or projects.

Scaling efforts, therefore, face and raise unique challenges. 

King and Crewe, for example, identify cultural (i.e. those who design policies do not live like those who the policies are supposed to benefit) and operational (i.e. those who design policies are not the ones implementing the policies) disconnects that often emerge when scaling from theory or small-scale interventions to larger contexts. Deep socio-economic inequalities (e.g. who has the power to make decisions) and systemic flaws (e.g. how government works and civil service capacity) frequently underpin these challenges. A different sort of evidence can help address these disconnects: evidence about power and systems rather than about what works on any given sector.

However, addressing these root causes requires more than evidence; it requires systemic reform and long-term political commitment, among other things.

Alternative approaches, such as the International Development Research Centre’s (IDRC) “scaling impact” framework, shift the focus from interventions to individuals, communities, and polities. This model emphasises deeper, more sustainable impacts over simple expansion. Scaling must be technically, politically, and ethically justified, delivered in collaborations and bigger is not always better.

This calls for a different evidence infrastructure or architecture that accompanies the entire policymaking/scaling process and responds to all parties’ needs. It cannot, therefore, continue to rely on linear conceptions of evidence-informed policymaking.

Systemic challenges in evidence-informed policy

One of the biggest challenges in evidence-informed policymaking is its often-linear framing. This approach assumes a straightforward path from evidence generation to policy impact, overlooking the complexities of political, social, and economic systems. This is easy to spot in the names of many initiatives; just look for a “to”, or “from”, or “for” as in “evidence to policy” or “research for development”.

Transformative scaling and system-level evidence-informed change require addressing these complexities head-on.

Systems are inherently resistant to change. They evolve to maintain the status quo and often obstruct reform efforts. They do not have a single centre or mothership that can be brought down and end the game.

Understanding the political economy of change—incorporating political science, sociology, and other social sciences—is essential. For example, Eduardo Dargent’s El Páramo Reformista explores why reforms in Peru often fail, even when leaders have a mandate, resources, and a clear plan based on so-called “what works” evidence. His analysis highlights key obstacles, including:

  • Resistance to change: Deep societal and institutional resistance, often grounded in long-standing beliefs and vested interests.
  • Sustaining reforms: Difficulty in maintaining reforms due to high leadership turnover and shifting political priorities.
  • Complexity of transformation: The need for cross-sectoral and multi-level coordination in policymaking.
  • Public demand: Challenges in building and sustaining public support for reforms.
  • Realism and urgency: The need to acknowledge systemic challenges while maintaining momentum for reform.

The recent development of the field of evidence-informed policy has taken a technocratic or scientistic—a parody of science—(as Moira Fall described at the Friends of Education event) direction. It has turned its back on uncomfortable and complex issues that nevertheless trump all efforts to improve the use of evidence in policymaking. We call these Thorny Issues -and there are plenty in Education. They are characterised by:

  • Their complexity: these issues often involve deep-seated structural, political or cultural factors which cannot be easily understood or solved.
  • Their influence: these issues can skew and derail the best-laid plans to promote better-informed policy.
  • Their distance from the evidence-informed policymaking field: these are issues that have their own academic and practitioner communities and, thus, are not the focus of EIP scholars and practitioners.
  • The discomfort they cause among stakeholders—makes them difficult to discuss and resolve openly.

Think of state capture, wide-spread corruption, deep-seated extreme values, weak political party systems and civil services, decades-underfunded university systems, etc. These can explain why gains made in embedding evidence use in education policymaking can be easily erased or why significant investments in research on challenging issues like gender-based violence fail to deliver meaningful outcomes.

Rethinking models of evidence use

The way we think about the evidence-informed policymaking field can also explain our failure to move forward. Narratives, after all, affect how we interpret and use this evidence.

Narratives, for instance, are crucial in helping us understand the political economy of change. We invest in different strategies to bring about change, depending on how we understand change happens.

Similarly, traditional narratives for understanding how evidence is used in policymaking fall short by oversimplifying reality and conditioning interventions to fall back on linear solutions.

The “Bridge” narrative assumes two distinct communities—researchers and policymakers— that speak different languages, have different needs, work along different timelines, etc. To connect them, funders have invested in initiatives emphasising research communications and communities of practice.

The “Supply and Demand” narrative reduces the process to a marketplace of buyers and sellers. To connect them, funders have primarily opted for the model of the broker or intermediary.

Both narratives, however, fail to see that while there are differences in the communities, there is also significant overlap. When actually studying a policy research community, one finds multiple connections between researchers and policymakers: common educational backgrounds, professional memberships, social connections, familial relationships, revolving doors between research and policy roles, etc.

Even the “Ecosystem” model, which is much better at recognising these rich networks, often lacks practical application (which is resolved by using either bridging or brokering solutions).

Maybe a better approach might be to focus on relational dynamics and public dialogue. Mirko Lauer’s concept of “information density” offers a useful lens. Lauer argues that sectors, geographies or polities with high density (i.e. sources of evidence, channels of communication, etc.) tend to deliver policy decisions that are rarely not based on or informed by evidence. And even if the decisions are incorrect, it is unlikely that mistakes will go unnoticed for long. This would respond to the need for long-term accompaniment and responsiveness of scaling efforts. On the other hand, decisions in low-density contexts will find it hard to be informed – even if the intention is there.

 Drawing on his insights, I categorised decision-making contexts based on evidence density and political interest:

  1. High density, low political interest: Decisions are informed by technical or academic criteria.
  2. High density, high political interest: Decisions are informed by evidence but guided by political criteria.
  3. Low density, low political interest: Decisions are driven by personal values or limited research.
  4. Low density, high political interest: Decisions prioritise electoral demands and public opinion.

This framework underscores the importance of understanding the political and social dynamics that shape evidence use. Of course political dynamics on evidence use can be better explained by more nuanced frameworks such as Adolfo Garcé’s political knowledge regimes.

The role of policy research organisations

Policy research organisations, sometimes referred to as think tanks, occupy a unique space, not exactly universities, governments, private sector entities, and civil society organisations – but not entirely different. They often act as boundary workers of different communities and foster dialogue. They can do this because they are bound by the rules of each community. Examples of these boundary organisations include think tanks (e.g. between research and politics), some specialised publications (e.g. between research and journalism) and EdLabs, which exist simultaneously in the research and policymaking spaces. EdLabs (and other similarly embedded or closely associated policy research entities) present an excellent opportunity to faciliate the use of evidence. 

However, EdLabs face unique challenges, particularly when tasked with driving systemic reform to embed evidence-informed policymaking in government. These challenges mirror those faced by other reform efforts and are crucially political, economic and social.

Lessons from MineduLab

MineduLab, a policy lab in Peru, was initially celebrated as an innovation in evidence-informed policymaking. Supported by strong international sponsors and national political champions, it conducted experiments to improve education outcomes. However, it has struggled to sustain its impact over time. There are many key lessons that can be learned from MineduLab’s experience:

  • Data infrastructure: Early successes relied on existing administrative data, but a more comprehensive approach to data management is essential.
  • Strategic political integration: Policy labs must be embedded strategically within political and policymaking ecosystems.
  • Scaling strategies: Scaling requires dedicated planning and alignment with institutional priorities.
  • Ethical considerations: Addressing privacy and consent is crucial for legitimacy.
  • Local engagement: Over-reliance on international researchers undermined local capacity-building.

While MineduLab’s story is not a failure, it highlights the complexities of evidence-informed policymaking in challenging contexts.

Conclusion

Evidence-informed policymaking is far from straightforward. It requires navigating systemic resistance, aligning with political and social contexts, and integrating diverse inputs. Evidence-informed transformational scaling and sustainable reforms demand a shift from linear models to more dynamic, relational and political approaches.

The lessons from Peru’s MineduLab and broader global experiences offer valuable insights for policymakers, researchers, and practitioners.