[Editor’s note: this post has been updated and has been republished to include new links.]
A bit of background: The Evidence-based Policy in Development Network (ebpdn) was set up to promote our understanding of the role that evidence plays in policy-making in developing countries and in international development policy. Several studies and events have helped to shed light on the factors that explain the uptake of evidence; factors that the Research and Policy in Development Programme synthesised, in 2003: the political context, the nature and presentation of the evidence, links or networks, and the external environment.
However, the ebpdn’s attention shifted in the late 2000s from an effort to understand the complex linkages that exist between research and policy communities to one focused on recording good influencing practices or demonstrating the impact that particular pieces of research had on policy. Research project driven case studies became the rule. This shift, in the view of some, limited the opportunities for learning that RAPID’s original work had offered.
Partly in response to this development, while I was Head of the RAPID programme, we, in partnership with the ebpdn’s Africa network, launched a series of studies that sought to turn this trend around and pay greater attention to the nuances of the relationship between research and policy.
In 2009 a group of Latin American researchers worked on a book published by ODI and International IDEAon the relationships between think tanks and political parties. The studies recognised that it was not possible to study policy research institutes, or think tanks, without understanding their political contexts. The case studies from Colombia (Partidos políticos y think tanks en Colombia), Ecuador (Partidos políticos y think tanks en el Ecuador), Peru (Think tanks y partidos políticos en el Perú: precariedad institucional y redes informales), Bolivia (Partidos políticos y think tanks en Bolivia), and Chile (Los think tanks y su rol en la arena política chilena) illustrated the complexity of the relationship between research and policy, as well as between researchers and policymakers. The idea of two separate research and policy communities was discarded, and the importance of their historical co-evolution highlighted.
A series of background studies for Sub-Saharan Africa, South Asia, and East and Southeast Asia followed the Latin American studies; and these were followed by a more recent volume of studies that pays particular attention to the relationship between media and research centres in Latin America, promoted by ebpdn members in the region.
Inspired by this, Emma Broadbent took on the challenge to describe the relationship between think tanks and their environment in Sub-Sarahan Africa. Instead of focusing on an organisation or a piece of research, she took the policy debate as the unit of analysis.
Here, a policy debate is understood as a contested policy issue involving any number of actors who contribute to the debate by offering an argument relating to any aspect of the policy, for instance the policy problem, policy options, means of implementation and monitoring and evaluation. A policy debate can take place in a single space as a one-off event (in which case the number of participants is limited), or can occupy a limitless participatory space over a period of time. This paper is concerned with the latter.
Policy debates are often conducted with reference to political interests and faulty evidence, with each participant in a debate coming to the table with a particular ‘ask’ and understanding of the policy problem. Debates are thus unequal playing fields: they are made up of participants who possess varying objectives, expectations, capacities, understandings, motivations and commitment. Importantly, only some of these may be made explicit, given the potential for some actors not to think and act in a unified manner. For instance, actions may not reflect stated values, or stated intent may not accurately reflect actual intent.
Using research-based evidence as a starting point for a case study also hides the unavoidable fact that evidence does not mean the same to everyone. The label is often attached to a great deal of things: facts, opinions, arguments, and observations. Little is said about the perception that different policy players have of these different types of evidence or their source; even if, as we know and as Emma Broadbent’s studies show, this perception plays a significant role in explaining why certain ideas are more rapidly accepted than others.
Most importantly, the focus on research-into-policy case studies assumes that what matters in policy decisions are the facts and findings emerging from studies rather than the arguments that, by their nature, must draw from a range of sources of knowledge and power: values, tradition, legislation, fears, imaging, etc. Arguments and big ideas are what change the world. Facts and findings simply provide them with ammunition.
The studies discussed here focus on policy debates in four of the countries in which the Mwananchi programme, which provided support for the study, operates:
- Synthesis of the case study findings
- Case Study 1: Decongestion in Accra,Ghana
- Case Study 2: Uganda’s HIV/AIDS Prevention and Control Bill
- Case Stury 3: The contemporary debate on genetically modified organisms in Zambia (a shorter version of this study was published in Zambia Analysis)
- Case Study 4: Chieftaincy reform in Sierra Leone
They offer an opportunity to address concerns about how evidence is used in policy-making. With the debates as a starting point, Broadbent tracked back the origin of the different arguments used by the various parties involved. She considered the different interpretations and sources of the evidence presented and employed by different actors; the roles that local and international policy actors play; and the specific and relative role played by research centres, researchers, and research-based evidence.
As a consequence, the case studies offer us a much richer description of the context, as well as ample opportunities to investigate further the complex relationship between research and policy.
It is worth reviewing the synthesis paper: Politics of research-based evidence in African policy debates. Its main findings, conclusions and implications include:
- Surprisingly (?), given the attention to make policy more evidence based, all four cases, specially in the Zambian one, the role of research based evidence in the policy debates was relatively high.
- However, we should not overestimate the role of research based evidence:
Even when it is used, research is often poorly referenced and seemingly selective; the full implications of research findings are poorly understood; and the logical leap required to move from research cited in relation to a specific policy problem (e.g. HIV/AIDS transmission trends in Uganda) to the policy prescription or solution proposed (e.g. the criminalisation of HIV/AIDS transmission) is often vast. Sometimes, research-based evidence plays almost no role, and arguments on one or more sides of the debate are driven by personal prediction, assumption, reflection on past precedent and commitment to the idea of progress. The case studies each emphasise the role of different types of evidence, particularly that arising from citizens, or the grassroots.
- To assess it we should consider three types of factors that explain the role research based evidence plays:
Debate-specific factors, relating to the locus of a debate and the perceived existence of a policy debate;
Discursive and cognitive factors, relating to how policy debates are framed, how research and evidence are understood and research capacity at institutional level; and
Proximate, agency-oriented factors, relating to the political, tactical and strategic factors that intersect with the nature of the debate and the discursive and cognitive aspects of policy debates identified.
- A key finding of the studies is that greater efforts need to be made to unpack what we understand by ‘evidence’ and recognise that we may not all be talking about the same thing. It may very well be that evidence is readily available but that, in fact, what is lacking is the capacity and incentives to use it. Not using certain evidence can be, in fact, a strategy -entirely logical if one recognises that policy processes are necessarily political:
However, when considering why the role of research-based evidence is smaller, this paper argues that this cannot be explained in terms of a ‘lack’ (of capacity, of research, of funding, of space for dialogue, of ownership) which can be filled (more capacity, more funding, more dialogue, better access to research); rather, it is not being used because there are significant incentives not to use it. Instrumentalisation of lack of capacity—which makes itself known in areas other than research–policy in Africa—thus describes a situation where there are significant advantages to a lack of capacity (assessed – in admittedly ill-defined – terms of the capacity to undertake, understand, and use research-based evidence), and/or significant disadvantages to improving this capacity (again, in this case, measured in terms of research-based evidence). The situation is thus sustained and in fact instrumentalised in order to fulfil a number of varied and interrelated objectives, including resistance to reform, the defence of national identity and autonomy and avoidance of scrutiny.
- Finally, the implications and recommendations of the research are relevant for think tanks and their funders (edited quotes):
- Researchers are supported to promote ‘my’ research, with little acknowledgement of the inevitable political interests, constraints, pressures and incentives research is a product of, nor of its discursive context
- Indeed, a more fruitful—and significantly more considerable—undertaking would be to turn our attention to improving the quality of policy debates to enhance the ability of people to discuss policy using critical thought.
- A central part of any effort needs to address levels of understanding relating to research methodologies and the philosophy of science, in order to help users of evidence understand and appreciate the limitations of particular evidence and locate an approach to gathering evidence among wider discussions about what constitutes valid evidence and rigorous research.
- Approaches to supporting ‘better’ policy debate would also include supporting the role of ‘mediators’ to analyse debates, thereby creating something of linearity in a debate in which evidence gaps can be identified and public demands for research-based evidence made and filled.
- In some cases, what appears to be a lack of capacity to undertake, use and understand research-based evidence cannot be addressed purely through ‘more’: ‘more’ capacity, ‘more’ research and ‘more’ links between researchers and policymakers.
Later on the cases inspired a book on communicating complex ideas. The nature of the African cases places ‘debate’ at their core. The cases on communicating complex ideas complied by Enrique Mendizabal offers another kind of debate: between researchers and communications as a way into exploring these issues.