The external evaluation of the Think Tank Initiative: “what we’re learning, and how we’re responding to those lessons”

24 September 2013

Nearly five years ago, TTI began to award grants to high-quality think tanks. Today, we support 48 institutions in Africa, Latin America and South Asia. The objectives for TTI’s support are innovative and ambitious: (1) provide think tanks with core funding, in the region of 20% to 30% of their operating expenses; (2) give them access to capacity building support to address organizational development needs, such as strengthening their communications capacity and research quality; and (3) share what we’re learning through our collective efforts.

Implementing an innovative and complex program such as TTI is generating many lessons. As we approach the end of Phase 1 of TTI, and look forward to Phase 2, we’re reflecting on some big questions such as “What has happened as a result of TTI’s support?”, “How have we done as a program?”, and “What can we improve on, now and in the future?”

To answer these questions, an independent evaluation team (from ECDPM and ODI) was commissioned by TTI’s donor partners to evaluate the program’s design, assess its implementation, determine TTI’s intermediate outcomes and impact, and suggest improvements for the program’s second phase. The approach was comprehensive – involving literature reviews, review of TTI’s own monitoring data, interviews, workshops, visits to 17 institutions and three regional offices in eight countries, the collection of 65 “Stories of Change”, and online follow-up surveys. You can view the detailed report here.

The evaluation reveals many lessons, about what has worked well, about what has worked less well, and about areas where we can strengthen our approach. The evaluators highlight a number of successes, including: 

  1. A strong formula:  TTI’s combination of core funding and technical assistance, including our use of Matching Funds designed to support collaboration, is proving effective.
  2. Enhanced organizational effectiveness: there are clear signs that think tanks are using their core funding to strengthen their organizational performance; for example, the Centre for Population and Environmental Development (CPED) in southern Nigeria used core funding to mobilize additional staff in administration, finance, research and communications, while also strengthening its governing board.
  3. Effective policy influence: Think tanks are having an increasing impact on policy: Instituto Desarrollo (ID) in Paraguay increased its role in policy processes by winning a contract to develop Paraguay’s regional development plan, while Grupo de Análisis para el Desarrollo’s (GRADE) analysis of a government program resulted in the Peruvian government retargeting programming to benefit rural communities.
  4. Investments in research quality: There is also evidence that think tanks are investing in, and strengthening, the quality of their research. For example, the Indian Institute for Dalit Studies (IIDS) doubled its number of staff holding PhDs, restructured its research unit, appointed senior research staff, and set up a network of external researchers for collaborative research.

Despite these successes, the evaluators have identified a number of areas where we can do more to improve our efforts. As we move towards Phase 2 of TTI, which will begin in October 2014, we are drawing on our own experience over the last 5 years, and learning from the recommendations of the external evaluation. We have already started to address the following needs for improvement:

  1. Sharing what we’re learning more effectively and more widely: The evaluation tells us that we need to focus our efforts more on capturing and communicating what we’ve learned to a wider public. We’re addressing this by developing and sharing interim lessons and insights on how Southern think tanks can contribute to national and global policy processesreporting on findings from commissioned studies, and communicating stories of think tank influence to a wide range of stakeholders, particularly think tanks.
  2. Understanding better how we are helping to strengthen research quality: We’ll test and adapt alternative methodologies to help us understand how TTI support influences research quality. We will also increase our efforts to help think tank researchers access, and use, a range of complementary research approaches that they can use to address challenging questions.
  3. Monitoring and evaluating on an ongoing basis: We are reviewing the way we implement our M&E strategy so that we can learn continuously from our experiences, and strengthen and adapt TTI to meet the needs of the think tanks we support. We’ve already begun to streamline our monitoring approach by improving the tools we use, and we’ll continue to minimize the burden that M&E can place on the think tanks themselves.
  4. Fostering collaboration: We’ll enhance TTI’s efforts to promote collaboration within the think tank community and between think tanks and other key policy actors, through initiatives such as Matching Funds and peer learning and exchange activities.

Increasingly, we are seeing evidence that the key decisions affecting development outcomes, whether in economic development, health, or education, are made by in-country decision-makers who are informed and influenced by those with context-specific knowledge. Think tanks play a crucial role in producing this type of knowledge, and helping inform the views and perspectives of a wide range of policy actors as they go about the complex, challenging task of shaping policies that ultimately improve peoples’ lives. TTI will maintain its support to think tanks and their work, and we recognize how important it is to keep learning as we do so, and to improve our support based on the lessons we learn.