[This article was originally posted on William and Flora Hewlett Foundation’s blog.]
The best think tanks around the world provide timely, independent research that informs policy decisions and improves lives. Yet many of these organizations struggle to explain how they measure the impact of their work. My view is that moving beyond numbers — and talking instead about how they are positioned, what they decide to work on, and who they work with — can help think tanks overcome their angst about impact.
Here’s an example of what I mean by angst. At one of the South Asia regional meetings of the Think Tank Initiative, scholars made substantive presentations about their work, often framed around policy issues that motivated them — worker safety in Bangladesh, internal resettlement policies and post-war development strategies in Sri Lanka, renewable energy in India, making budgets more gender responsive in Pakistan.
In all of these presentations, scholars described a policy problem, how their research was meant to address it, and the beyond-the-research activities they pursued, such as convening multi-stakeholder discussions, outreach with government officials, partnering with civil society, data collection, and communications tools like social media, video and data visualization. They described looking across their fields to push the frontiers of research, test methods, frame future-oriented policy research questions, and build lasting relationships with government officials and other influencers.
Across the board, think tankers spoke of these activities with ease and confidence, and told well-rounded stories of getting ideas and evidence into the public domain and policy space. I loved that they tended to start from the end — describing first the problems they hoped to address, and second what they did about it.
The angst crept in during a session on “measuring impact.” They all reverted to starting at the beginning — numbers of papers published, events hosted, tweets tweeted, maybe a dissemination plan tacked on at the end. This start-with-the-paper impact story is much harder to tell, first because there is rarely a straight line from a paper to a policy change, and second because it assumes every single research product has to have an impact. Yet some think tanks bristle against anything else for fear of being accused of “advocacy,” not being neutral, claiming impact when it can’t be precisely attributed, or losing access to policymakers they aim to influence.
So how can think tanks break out of this narrow approach to measuring impact? It helps to think about the research-to-policy process in two ways, summarized by EPPI Center at University College London’s “The science of using science: Researching the use of research evidence in decision-making” as well as DFID’s “What is the evidence on the impact of research on international development?”
These papers help us understand the difference between an instrumental use of research (when research findings influence specific policy formulation or implementation) versus conceptual use (when research influences how policymakers and practitioners think about issues, problems, or potential solutions).
The instrumental approach is hardest to achieve and to measure, and is definitely not the only one that matters. Yet, it seems to be the focus of discussions (and angst) on think tank impact. So, how can think tanks demonstrate influence at a conceptual level? Thanks to what I’ve learned from think tanks we support through our evidence-informed policymakingstrategy, I have some ideas about how to answer this question.
My advice to think tanks — In addition to focusing on measuring instrumental impact, demonstrate how you think about conceptual impact. Here’s how:
1. Identify opportunities
As you look at the next couple of years, what opportunities are on the horizon to shape an agenda, inform a particular policy or program, showcase an issue, or feature a body of research? Whether it an international summit, a national election, changing leadership at a ministry or international organization, a high-profile commission or world event, when will there be increased attention to your issues? Are the opportunities instrumental or conceptual in nature? And what are the best ways to seize these opportunities — working paper, blog, event, working group, book, closed-door consultation with officials? Show that you are thinking about opportunities for influence, not just your to-do list of research projects.
2. Show how you are positioned
To have your work reverberate through policy circles — to influence how policymakers think, it is critical to have strong relationships across the actors that shape the issues you care about. Show a sophisticated understanding of this landscape and how you are positioned in it. Based on the opportunities you identify, who are the critical actors to engage early and often? Describe the spaces you are being invited into — be they testimonies in parliament, issue-specific working groups, high-level panels at international organizations, or consultations on policy and program formulation. Who is asking you questions? What spaces are you occupying and how did you get there?
A recent visit with the Initative Prospective Agricole et Rurale (IPAR) in Senegal drove this point home. The full team, from executive director, research director, communications lead and individual scholars talked fluidly about their engagement with actors that influence policymakers — through training journalist and NGOs in research findings and data analysis, engaging with farmers’ associations or working directly with ministries. It helps that IPAR has a multi-stakeholder constituency built into its general assembly, but all think tanks could learn from IPAR’s savvy approach to positioning themselves as a resource for the many actors who directly and indirectly influence policymakers.
3. Develop a creative research agenda, and explain how you do it
Most think tanks pursue a mix of enduring issues in their field (global health, trade policy, agricultural productivity and so on), and ahead-of-the-curve issues (cyber security, open contracting, tobacco control in West Africa, etc.). Don’t just say what these agenda-setting issues are, but how you know. Demonstrate that you have a pulse on policy debates and trends in your field, and how you apply this to your work.
4. Be smart about partnerships
Which partnerships will help you reach the audiences that matter most for your issues and what you want to achieve? Trying to help secure the best possible new president for the African Development Bank, from Washington? Host a candidates’ debate with think tanks in Ghana and Benin, as CGD Southern scholars trying to influence the SDG negotiations among UN missions in NY? Find partners at the heart of the action to bring scholars and country missions face to face, as Southern Voice did. Trying to influence politically-charged, thorny policy issues at the national level, partner with civil society groups that work closely with citizens, as CSTEP in India does on renewable energy, and CRES in Senegal does on tobacco controls.
5. Consider signature work
In some cases, branding an idea or body of work allows it to permeate policy spaces better. The Southern Voice “Post-2015 Data Test,” The ACET “Africa Transformation Report” or CGD’s “Cash on Delivery Aid” are all bodies of work that slide off the tongues and seep into the thinking of policymakers around the globe. To be powerful, there can’t be too many of these, so select carefully with the above considerations about opportunities, positioning, research agenda and partnerships in mind.
6. Reflect often and openly
We all know it is hard to navigate the many factors that influence policy change — politics, budget constraints, civil service capacity, populism, and so on. So let’s be honest about it! Talk about what you are seeing and why you think it is happening. What is getting lost in translation from research to policy? Are the big ideas resonating but the implementation details failing? Show that you have a pulse on where your ideas are going, where they are getting stuck, and how you’ll change your approach accordingly.
Let’s stop all the hand-wringing about whether think tanks can trace their precise impact, or the fine lines between research, policy engagement, and advocacy. Let’s get away from linear papers-to-policy log frames that don’t reflect the messiness of the policy process. For funders, this means not funding specific research projects and insisting on a communications plan as an afterthought. Rather, ask what change think tanks aim to achieve and how all their assets — research, outreach, communications, partnerships, and convening, contribute to this change. For think tanks, this means revealing more about how you are positioned, what tools you use to influence outcomes you care about, and what you are learning along the way.