Series on MEL for think tanks: conclusion

This series included 5 articles about the MEL of think tanks. The series began with a discussion by Dena Lomofsky of the challenges and possibilities of monitoring, evaluation and learning in think tanks. We all recognise that the MEL of policy influence is particularly tricky because attribution is so difficult to validate. The first article in this series argues for a different approach to policy MEL whereby each think tanks identifies very specific policy objectives, linked to their particular policy influence strategies, and then locate these within a broad understanding of the policy influence process. In this way, you are setting the agenda for your own MEL, and ensuring that you are defining outcomes that you will be able to measure or describe, and that these outcomes are directly related to your policy influence activities.

The premise of the series is that when think tanks practice good monitoring and evaluation and implement improvements based on their lessons from the data that they collect, then they will be more effective in achieving their objectives.

The common messages emphasised by all the authors in the series are:

• Be clear about what you want to achieve (have clear policy influence objectives)
• See monitoring and evaluation as an opportunity for learning, innovation and adaptive management, not only as a tedious accountability requirement for donors
• Develop systems for MEL based on what you have in place, be realistic about what you can achieve and build on your strengths

Since most think tanks use research and communications to achieve their policy engagement objectives, there is an article on the MEL of each. The series also includes an article on the MEL of governance because good governance is the basic foundation for good think tank management, without which its policy influence strategies are less likely to be successful. The series also includes a case study of an MEL design process for think tanks to provide some insights for organisations that are moving along this path.

MEL of Research Quality – RQ+

Zenda Offir wrote about RQ+ which is an approach to assessing research quality developed for IDRC that uses a systems approach. The RQ+ approach is of particular relevance for impact-orientated think-tanks – not only for use in external reviews and evaluations but also for its potential to enable critical reflection and adaptive management towards high quality, impactful research. In the article, Offir challenges think tanks to test and experiment with the framework to help improve it. If you have used RQ+ we would love to hear your comments on it.

MEL of communications

In this article, Carolina Kern suggests that MEL needs a rebrand from a very boring and time-consuming process, to exciting possibilities for learning and innovation. The MEL of research communications should enable you to improve how your research findings are packaged and disseminated so that they have a chance to feed into the policy debate and to influence some sort of positive change. This article provides useful links to resources on the MEL of communications, and tips for smaller think tanks who don’t have a dedicated MEL person.

One tip is to view MEL as integral to every part of your research and communication, not just something that gets tacked on at the end. Carolina includes very practical advice on indicators for the MEL of communications that will provide good insights into the successes and challenges of your communications activities such as database management, media engagement and event engagement and website traffic.  The article highlights the kinds of questions that think tanks should be asking about their communications efforts such as “Why did policy brief X do so much better in terms of downloads than policy brief Y?”.  Carolina emphasises the importance of then using the data you have, together with your own reflections to learn and improve for the future.

MEL of governance

The article on the MEL of Governance by Nana Davies also provides practical advice and suggests a framework for MEL of Governance. In this article, Nana provides an overview of issues related to the Board of Directors (governance) that think tanks should be paying attention to in their Monitoring, Evaluation and Learning (MEL) systems, and some methods for doing this. The article recognises that think tanks may be at different stages of maturity and that as a result, the role of the governance function will change over time. The MEL of governance should help to track the maturity of the think tank governance structures, roles and responsibilities.

The article suggests that there are 6 main areas that think tank boards must ensure are in place, and that should indicate good governance, and that is applicable to all kinds of think tanks. Indicators are also suggested for each of these. The domains are:

1. The establishment of internal structures and functions and provision of strategic guidance
2. Legal registration and compliance
3. Operational policies and procedures
4. Financial oversight
5. Resource mobilisation and sustainability
6. Planning, monitoring, evaluation and learning of the operation of the think tank

The article also suggests some processes for reviewing governance such as conducting regular SWOT analysis, using an Organisational Capacity Assessment process or developing an MEL plan for governance. Nana emphasises the importance of regular reviews of think tank governance, followed by improvement plans.

MEL for projects

If your organisation is only at the start of your MEL journey, you may want to start with just doing the MEL for one project, and then expand to other projects and then eventually to the whole organisation as your confidence in MEL grows.

The case study by Tracey Philips on the design of an MEL system for a project of a South African think tank (SAIIA) that was conducted by Southern Hemisphere, provides an overview of project level MEL in a think tank. The aim of developing the MEL system was to enhance the capacity of the project to measure its level of policy influence through greater standardisation and systematisation of data collection and analysis. The ability to measure and assess the level of policy influence would, in turn, enable the project team to reflect on, learn from and adapt its strategies and activities to strengthen their policy influence outcomes. The case study suggests a process for designing project level MEL which includes identifying and consolidating what you have in place through conducting a situation analysis of your MEL, developing a theory of change, MEL guideline document and plan, developing indicators and tools for data collection and reporting formats that include lessons learned and suggestions for improvement. Organisations would need to ensure that they have systems for data management, reporting and learning to support the project team.

Conclusion

Since each post in this series offered some form of framework for measuring organisational capacity in the areas covered, it has encouraged us to develop an organisational capacity assessment tool (OCAT) specifically for think tanks. We hope you enjoy reading the series and we are looking forward to sharing the OTT OCAT as it develops and incorporating your suggestions and improvements.

We wish everyone a successful 2018.