I’ve been working on evidence use for the last seven years, or so. I started out in the VakaYiko Consortium, under the Building Capacity to Use Research Evidence initiative. This was funded by the UK’s Department for International Development – now the Foreign, Commonwealth and Development Office.
Since then, I’ve been lucky enough to gain insights into lots of different initiatives around the world, focused on improving evidence use in decision-making. Some practitioners in this space work within think tanks, non-governmental organisations and other organisations that broker partnerships with evidence users in government. There are also many evidence champions that work within governments, designing and leading initiatives to improve evidence use.
In this piece, I’ll outline some practical tools that I’ve used myself and seen others using. These tools seem to be helping us find practical ways to move forward in the increasingly complex and nuanced field of evidence use. They can be used for research and analysis but they’re primarily practical. Thus, those working with governments can use them to understand and address the issues affecting evidence use.
Tools for understanding factors affecting evidence use and identifying opportunities
The Context Matters Framework is a great tool. It was originally developed in 2016 by Vanesa Weyrauch, in collaboration with Leandro Echt and INASP. Weyrauch is the co-founder of Purpose & Ideas (formerly Politics & Ideas) and a long-time friend of On Think Tanks (OTT).
The framework lists and categorises factors that affect evidence use and builds on consultations with over 50 policy-makers, researchers and practitioners in the Global South. It’s also a useful way to categorise and prioritise the things that affect evidence use in a particular policy area or organisation. These contexts could range from the broader political economy to internal organisational dynamics. Because of this it can stand alone as a research or analysis tool.
But its core purpose is to guide a participatory process of reflection with evidence users and to identify opportunities for improvements in evidence use. It’s been used to do this in various contexts – e.g., the prime minister’s office in Uganda, Ghana’s Environmental Protection Agency and two regional offices of UNICEF.
Recently, others have developed similar tools for identifying factors that affect the institutionalisation of evidence use and opportunities for change. The importance of these structures and processes has been highlighted by the Global Commission on Evidence’s emphasis on domestic evidence support systems.
The WHO’s checklist for evidence use and the SEDI programme’s political economy + approach are tools that can be used to identify entry points for improvements in these systems. But whichever tool is used, it’s worth looking for two essential characteristics:
- Its approach should identify entry points in the context of the broader political economy and the organisation. Many tools do one or the other, but to design a practical initiative you really need both.
- It should consider both formal (technical/procedural/processes) and informal factors (relationships/behaviours/working cultures).
Tools to find space for organisational change in public institutions
The triple-A model (below) from the PDIA toolkit is a useful framing for understanding organisational entry points. It’s not specifically targeted at evidence use – it can be used to understand the appetite for any kind of change or reform in the public sector.
The basic premise is that those who want to support change in the public sector need to avoid ‘isomorphic mimicry’. This is the creation of tools or structures that make it look like reform is happening, when they’re actually not used in any meaningful sense. They should also avoid ‘premature load bearing’: where reforms are heaped onto organisational contexts that can’t absorb them.
To understand the space needed to effect real change that sticks, we need to look at the following:
- Authority – e.g., high-level buy-in.
- Acceptance – whether that high-level buy-in is matched by a general acceptance of the need for change within the organisation.
- Ability – whether there’s the time and capacity to actually implement the change.
We often hear about one of these three A’s. But what’s useful about this tool is that it helps us look at them alongside each other.
Tools for training public servants in evidence use
There are many resources for this, as it was one of the first approaches used in trying to strengthen evidence use. Understanding about the capacity for evidence use has since broadened to include more organisational and systemic capacities. But skills are still a core foundational aspect for many government agencies. So, many organisations that partner with them still provide training to improve evidence use – often co-created with civil servants.
Nesta’s new Engaging with Evidence Toolkit aims to support UK civil servants in synthesising, scrutinising and using evidence in decision-making. It does this through an engaging and innovative approach.
INASP’s Evidence-Informed Policy Making (EIPM) Toolkit was developed and piloted with Ghana’s Civil Service Training Centre and other partners in Africa. It focuses specifically on the needs of civil servants in developing countries to access, appraise and communicate evidence for decision-making.
INGSA’s case studies are a great collection of resources to use in workshops to simulate the complexity of using evidence in policy decisions.
Apolitical’s online course on evidence use features readings, videos and reflection activities targeted at civil servants, worldwide.
Finally, it’s worth remembering that civil servants aren’t the only ones who need skills to improve their evidence use. The European Commission’s in-house evidence body, the Joint Research Centre, has produced an excellent skills map. It outlines a professional development framework for researchers, with eight interlinked skills areas that they need to support their evidence use. This skills map complements their skills framework for policy-makers.
Also, their new Smart4Policy self-reflection tool helps both groups assess their own skills.
Tools for understanding evidence use in parliaments
Much of the evidence-informed policy discussion tends to focus on the executive branch of government. But the legislative branch plays a critical domestic accountability role, and evidence is a huge part of this.
There’s a growing body of research on this topic. A two-volume book on parliaments in African evidence systems and an article in Evidence in Policy, ‘Use of research evidence in legislatures: a systematic review’, were published recently. These publications detail the unique range of issues that affect evidence use in parliaments.
But I like to use ACEPA’s diagrams in Evidence in African Parliaments when I need to understand where a parliament stands on evidence use or to identify opportunities for improvement.
One of these diagrams (page 18) shows the internal units that typically make up a parliamentary evidence system. It’s a great starting point for understanding who you can engage with to strengthen evidence use in parliaments.
The other diagram (page 26), shows the link between legislative roles and information needs. It’s useful for starting a discussion in workshops, as it shows how evidence use enables a more transparent and accountable democratic space.
The Inter-Parliamentary Union published a useful resource for parliamentary staff and their partners. It’s a reliable resource, which was produced by a working group of parliamentary staff from around the world. It offers practical recommendations on how to establish a strong parliamentary research service.
Tools for developing theories of change on evidence use
For this, I usually recommend the framework by Langer and Weyrauch, in Chapter 3 of Goldman and Pabari’s Using Evidence in Policy and Practice: Lessons from Africa. This useful collection of case studies and lessons from Africa on evidence-informed policy details the different change mechanisms that enable evidence use. And it builds on Langer’s prior systematic review on this topic and on Weyrauch’s initial work in the aforementioned Context Matters Framework. It uses a capability/motivation/opportunity model, which bears some similarities to the triple-A model.
This framework discusses the different types of evidence use in decision-making – differentiating between symbolic, instrumental, process and conceptual use. But this might be more detail than you need – unless you’re designing or evaluating an evidence-use project. Yet, when you get into the detail of a thoery of change workshop, the framework provides some useful concepts.
Evidence use: an ever-evolving field
One of the most interesting things about working in the field of evidence use is that it’s continually evolving – there’s new research all the time. There’s also a growing community of practitioners around the world who are experimenting with practical ways to improve evidence use.
Many of the most exciting innovations are coming from the Global South. This great summary of the key lessons from the Africa Evidence Network shows this.
We’d love to hear more from you on this topic. Tell us, what are your favourite tools for practical work on strengthening evidence use? @onthinktanks