And that’s the 60-million-dollar question: how do you get to that point where everything we all do as actors and the system responds to what needs to be done… as opposed to what we, who are very clever and think we know better than everyone else, think needs to be done.
This is the transcript of an interview with Peter Da Costa from 2018 at the side-lines of the African Think Tank Summit, in Accra, organised by the African Capacity Building Foundation.
It was not the last time we talked but I believe it was the last time we saw each other.+
I cannot understate the influence Peter had on my work – and my worldview. We met in 2003, when I joined Stephen Yeo and Peter on an evaluation of the IDRC-funded SISERA network in Africa. Then, again, when he helped to set up the Overseas Development Institute (ODI)’s first blog a few years later. And then, on and off over the years, we corresponded, debated and I learned from his insights into the world of evidence-informed policy and the development industry in Africa.
Please read: Rule of experts? Peter’s PhD thesis.
We have been wanting to celebrate Peter’s life at OTT, thinking hard about how best to do it. We think that is through celebrating his ideas.
In this interview we covered a range of issues related to the evidence-informed policymaking field – only a fraction of his expertise. His insights and the questions he presents us with are as relevant today as they were back then.
I often say that Peter is one of the few people whose opinion I really cared about before publishing or presenting an opinion. I would ask myself: ‘what would Peter say about this’, as an intellectual exercise when developing an argument. I still do.
The interview finishes with an encouragement from Peter to continue with our work. We will.
Enrique Mendizabal: I’m here with Peter Da Costa from the Hewlett Foundation to talk about the field of evidence-informed policymaking.
We are sitting in his hotel in Accra, after a couple of days discussing youth unemployment with think tanks from across the continent.
I think it’s been almost 15 years since I started working consciously in the evidence-informed policymaking field. But you’ve been working on it for longer.
So, what do you think is the state of the field right now, and in Africa in particular?
Peter da Costa: Thanks for inviting me to share my views. First, I should say I am a consultant to the Hewlett Foundation, which means that I’m independent and I don’t represent the views of the Foundation. I just spend a bit of time around the region looking at opportunities, understanding the landscape and helping inform their work in this field.
The first thing I would say is that this field has had many names over the years. If you recall, there was a little programme in a think tank called the Economic and Social Research Foundation (ESRF) in Tanzania. The programme was called EBPDN: evidence-based policy development network. It was a network based on a knowledge platform that was being curated by ESRF on behalf of a bunch of actors, some funded by DFID (the UK Department for International Development – now FCDO), to promote this whole field of evidence-based policymaking. +
At some point it was all about basing your policies on evidence. And then of course the literature, and the grey literature, and the practitioner literature and everything pointed out that you can’t base policy solely on evidence.
This is when we started understanding a bit about the political economy around it all. And so, we sort of changed it to evidence-informed policymaking.
But then there are also other great ways that this is all framed. As you say, this has been an endeavour that’s been ongoing for many years.
There are many different disciplines, sub-disciplines, silos, case studies and instances of people working with evidence, more or less scientific, to try and inform or influence policy outcomes and implementation. And what we learn influences the next cycle of programme and policy development.
So, it’s correct to say that this endeavour is not a new field. It’s not even a homogeneous field. It’s highly heterogeneous and extremely complex. And there may be new levels of engagement around certain aspects of it, but it’s definitely something that a lot of people have been working on for many years.
[EM] What is the end game and when can we say we’re satisfied that the capacity to use evidence in policy is there, or that enough evidence is being used in policymaking? Is there even an end game in sight?
[PDC] Extremely difficult. First of all because we don’t have a common understanding of the field at large. So, it’s difficult for us to establish a baseline to say where we are now and where we want to get to.
All we know is that we all believe in this mantra around evidence and the likelihood that good evidence, produced and generated and then pushed into policymaking in certain ways, is more likely to result in better policy.
But there are a lot of assumptions around that that we still haven’t yet come to terms with. You can have good evidence and policymakers that use the evidence and you can still have unexpected negative policy outcomes.
[EM] Yes.
[PDC] And we know that actually some of the best policy outcomes and best outcomes in terms of development are often unintended.
I think it’s taken quite a few people a very long time to open up to that possibility. The sense has been that purposeful, clever, driven people, supported by money, can make sure, in a linear way, that we push through the evidence value chain and result in better policy. And that governments and others who make decisions and implement them will make better decisions.
But of course, we’ve learned that this process is not linear, it’s totally dynamic. Like I said, we didn’t have any real baseline. All we know is that context is key. In every country in the world, and even within each country, there are different dynamics that shape and affect the extent to which evidence has any impact whatsoever.
And of course, there’s this kind of blanket statement that people make, for example ‘that evidence may account for 10% of the factors that determine or influence or inform policy.’ We have lots of assumptions.
And we’ve also learned about supply and demand side dynamics. At some point a lot of effort was put into generating the right kind of evidence and producing the right kind of packaging for that evidence. But then more recently, the emphasis has shifted to the uptake and use of evidence.
[EM] Yes. Because it was seen as not enough to fund research, knowledge.
[PDC] So, you find that every programme or project funded by DFID or other donors includes and requires a research uptake strategy.
But again, I think we’re still in the early years of that and it all depends on what sector we’re working in and what kind of intervention we’re looking at. And so, you’re paying for a lot of bad research uptake work and a little bit of good research uptake work. But it’s all part and parcel of this evolving engagement with evidence.
[EM] Are there any interesting initiatives around Africa that you would identify as very good practices, or, at the very least, interesting practices that are worth looking into further?
Yes. I think the first thing to say is that when we talk about Africa, we all slip into this tendency to talk about Africa as a whole. It’s 55 countries. It’s 150 or so borders. It’s the region with the most borders in the world. And this was a result of the colonial division of the continent. These had all kinds of effects, including transaction costs.
I mean, something you can say is that maybe more than 30 of the world’s least developed countries – according to UNCTAD (the United Nations Conference on Trade and Development) metrics – are in Africa. But I think that Africa means a lot of things to a lot of people and some of the things that it means also determine the way in which people try to intervene. So, there’s a certain objectification of the continent.
But if you look at individual countries, there are many things going on in many different contexts. We know about the work that’s happening on the demand side with parliaments, with ministries of health, with health committees: a number of organisations and networks have realised that you need to be able to engage the government at that granular level to build an appetite and interest in using evidence, and also to stimulate the desire among government officials to become better at commissioning evidence and using it.
So, there are a number of very interesting experiments, some of them were funded under the BCURE (Building Capacity to Use Research Evidence) kind of interventions that DFID funded through its research and evidence division. But there are many others that predated that.
I often talk about the behaviour change communication initiatives that USAID started supporting heavily in the 80s, if not before. Look at the footprint of the Johns Hopkins Centre for communication programmes. This is a world class school where people go and do a master’s in public policy with a heavy concentration on behaviour change communication. And then they join the consulting networks of these alumni and organisations and they get deployed around the world with USAID funding to deliver behaviour change communication programmes in health primarily, but also in agriculture and other areas.
So, USAID has put millions and millions of dollars into this kind of work over the last 30-40 years.
And some of the impacts have been dramatic. I mean, the whole practice of designing radio dramas and television soaps using evidence-informed methodologies and so on, all came out of that field.
The big impacts we’ve had in stemming HIV/AIDS transmission and social behaviour change, and the lessons we’ve learned about the fact that you can’t just focus on the individual but need to focus on the society and the context have all come from evidence, including policy.
[EM] Right. Not the other way around.
[PDC] That’s another area I would point to that is completely underrepresented in the current enthusiasm and discussion around evidence-informed policymaking, but which has been pivotal to the practice.
And then in agriculture there are a lot of innovations that are powered by scientific research and evidence. If you look around you find a lot of projects and programmes around the work of the FAO (United Nations Food and Agriculture Organization) and IFAD (United Nations International Fund for Agricultural Development), particularly the FAO and its development communication programme work.
Also, in the area of polio, which I do a little bit of work on. Polio policy is entirely evidence-informed. I mean right down to the last bit of data. And the campaigns where they’d been able to stop transmission of polio have all been down to very sophisticated gathering and use of evidence.
So, I think there are lots of things going on. But the current excitement seems …
[EM] … seems to now focus on a particular group of economists, forgetting that so much has been done for a very long time and in a range of disciplines, sectors and fields.
Yes, I think a lot of the excitement is on quotas and the focus on particular sub-disciplines or sub-fields for methodological strands is totally driven by bilateral donors like USAID, DFID and others. And it’s also driven by private US foundations, and by foundations who have an interest in technology, and so on.
For example, there’s a big interest in randomised controlled trials (RCTs) – which has kind of been hegemonic over the last 10-15 years, and has been fuelled by funders. In the process we learned a lot about what RCTs can and can’t do, and the extent to which they are or are not relevant.
But in the process, too, millions of dollars have been spent funding big world class centres of excellence that have proliferated themselves around the world. They have established in the global South (as well as in the global North) and still absorb huge amounts of funding.
And in the process what we learn is that actually these are not the most appropriate instruments in all instances. They do work for certain types of interventions, and help us understand that we need other forms of evaluation, other methods, methodologies, mixed methods and so on.
But I mean, this this is all quite cyclical because I think, probably, if you look back in the literature, you’ll find that these lessons were already learned years ago.
[EM] Yes, we have been here before.
[PDC] But a lot of this is driven by people who are very positive and want to make a difference to Africa, or whatever else, and are seized by the conviction that the best way to do it is the way they want to do it.
And so, you now have that industry, on the back of the SDGs (Sustainable Development Goals) movement and the need to make sure that you ‘leave no one behind’ and to make sure that everyone who counts is counted. There’s a big focus on data revolution, which basically came out of the frustration with the fact the MDGs (Millennium Development Goals) were very difficult to monitor and track.
Everyone recognises the power base and the importance of data. But data is quite a nebulous concept and is very difficult for citizens and ordinary people to grasp. But we are still at that stage of evangelism around data and we haven’t yet managed to figure out how data can really be relevant to people and to the realities on the ground.
However, it’s also attracting a lot of support funding wise. And there’s also an obsession with certain types of data generated by certain types of technology, for example big data. There is this sort of belief that big data can help solve all of the development problems we have because it provides you with the opportunity to get insights that you would never otherwise have gotten – the unofficial statistics or other types of information.
The only challenge with that is, of course, that on the demand side (on the government side) there is absolutely very limited interest in big data, very limited interest in a capacity to engage with it and the age old problem that evidence is not by any means the only driver of policy.
So, going back to your question about where we are and how do we know when we get there, and do we have any end goal in sight: we do not.
And I think that interventions are merely trying to learn, tinker with and test ideas. Some of which end up yielding very interesting results. I mentioned the parliamentary committee work and some of this other stuff. You may also find some interesting hybrid engagement between think tanks and data providers.
But then there are other huge gaps or huge areas in which there’s less emphasis. For example, there’s way less emphasis on tertiary education and research centres and universities, which historically have been pivotal to any development of any society. So, focusing on think tanks outside the university (maybe one or two within but mostly outside) without focusing on having a robust educational system with an emphasis on STEM (science, technology, engineering and mathematics) and Innovation (it’s not just science) and social sciences.
Because we also gleamed a lot of our insights around evidence-informed policy from anthropology, sociology and other fields.
But what you do get is a strong emphasis on things like behavioural economics, which is now becoming the new frontier for behaviour change.
You talk about nudging people’s behaviour. Again, it’s dominated by economists of a certain type and it’s very quantitative. And then you hear about human centred design and a number of these techniques which actually, I would say, are quite reductionist. When what we need is much broader and we need a stronger infrastructure in which evidence can actually be generated.
So, we need to strengthen the tertiary education sector. We need to strengthen education more broadly. We need to address the resourcing of this. And it needs primarily to be resourced by the countries themselves – and by actors in the country, whether public or private.
[EM] I see what you are saying, that you can’t have these functioning think tanks or governments in our countries without ourselves financing a human resource that is going to generate, communicate and use evidence of any kind.
And if we don’t invest in universities and in tertiary education, if we don’t invest in undergraduate and postgraduate degrees in our own countries, and across this very diverse region in Africa, we might have a few islands of excellence heavily funded by an external funder, but it won’t be lead to a systemic change.
I see that as the way forward: funding universities as a way to bring about these sustained changes. But these are long-term commitments. Do you see any current funders interested in these long-term commitments?
[PDC] Well, for example, I know that in Tanzania the Swedish Government’s research strategy has committed long-term support for universities, to make them stronger.
And of course, this is a historic commitment that seeks to reverse the disinvestment in tertiary education that came in the 80s with structural adjustments. When governments were told that evidence showed that primary education had the highest rate of return. Therefore, in a resource constrained environment, where countries were in debt, they should not invest in higher education or tertiary. They should focus on getting more kids into school at primary level.
And then they should not invest so much in health systems. And so, all of this is now coming back to haunt everybody. Because the reality is that disinvestment has cost African countries and other countries who had to implement structural adjustment programmes.
I think the Swedish model of investing in higher education is a good one. I think organisations like IDRC (the International Development Research Centre in Canada), who have been supporting research for more than 40 years, have advanced insights into how to build systems.
IDRC, for example, has an initiative called the Southern Granting Councils Initiative, which is all about building capacity of organisations in countries to be able to fund science and technology, STEM, in a system systematic way, and to leverage government on budget funding to do that.
And then we also have the DFID supported research hubs. I think there’s one in Asia in Nepal. And then another was set up three years ago in Kenya, the East Africa Research Hub. The whole idea there is to try and strengthen research systems in East African countries.
I think that’s the way forward.
But then, of course, to what extent is that joined up to these other current efforts to build the evidence and inform the policymaking ecosystem? And I think if you are in the business of looking to build parts of the ecosystem, you need to be connected to the other parts of it. And if one or two funders don’t have enough money or enough appetite to take on the whole system, they should at least be a way of having a joined-up approach.
It means, necessarily, including the governments and the actors in these countries. Whether they are from the private sector, civil society, or others who work on all of these issues of data, research, information. Or even those in media, including social and digital media. And lots of other actors who generate or use evidence. And citizens. They are all stakeholders. I think they should be in the driving seat and whatever people do in a given country or location should be informed and primarily driven by the needs on the ground.
And that’s the 60-million-dollar question: how do you get to that point where everything we all do as actors and the system responds to what needs to be done… as opposed to what we, who are very clever and think we know better than everyone else, think needs to be done.
[EM] Thank you, Peter, that was a perfect way of finishing our conversation and let’s see if we can answer that question the next time we talk.
[PDC] I’m sure you’ll answer it eventually.
[EM] Thank you, Peter