Telling your story of policy influence

When it comes to telling the story of what your organisation does, its achievements and failures, it is critical that you have all the available evidence that supports your statements. You can’t make up chunks of information and you also don’t want to have big gaps in your story line.  It is important that the data you collect gives you enough of the right information for a full story.

How do you do this?  The answer is a well-designed and systematic Monitoring, Evaluation and Learning (MEL) system.

A good story comes first

Before setting up an MEL system, though, you need a good storyline.  This is akin to developing the script before you start filming a video.  In the world of policy influence, having clear policy influence objectives is a critical start – what is it that you actually want to change?  Once you know where you are heading, you should be able to tell the story of how your organisation intends to reach these objectives. You then need to identify your actors – who the characters in your story are – and how you want them to think or act differently. Who are the main policy actors? Are they politicians, policy makers, or policy influencers such as other activist organisations and research groups?  Who else is there? What is your role in the story? How do you intend to influence these changes in the actors?

We need, preferably in this order:

1. Policy influencing objectives (or other objectives, depending on your organisation’s mandate and mission)
2. Strategies to influence these objectives (made up of the activities and outputs that lead to the objectives)
3. The changes that you want to bring about amongst the policy actors (outcomes – these can be separated into different levels of outcomes such as immediate outcomes and longer term outcomes)
4. The ultimate impact or benefit to society

These elements are shown in the results chain in the diagram below.

Once you have these main building blocks in place, the next task is to figure out how you will capture information for each block: what indicators and data sources will you use?

Indicators come next

Using planning and design tools such as theory of change or Outcomes Mapping is a good way to get to this story. Both build indicators into the process, so they act as the starting point for MEL as well.

The kind of data you collect will be determined by your indicators. To ensure that you have all the information you need to tell a full story, so it is important to be systematic in the way you design your indicators. You must have indicators for each level of your results hierarchy (outputs, outcomes and impact).

It helps to design your indicators around your policy influencing strategy. For instance, if your strategy is to shift the agenda on an issue by producing high quality research, disseminating and communicating this research and holding high level events around it, then your indicators should reflect the quality and quantity of your performance and what is changing as a result.

An example

Let’s look at research papers as an example.

Research output level indicators: (what did we do and how did we do it?)

• Number and topics of research papers
• Number and topics of publications and communications pieces
• Reach of publications and communications
• Relevance, quality and accessibility of research and publications

Research outcomes: (who is using the research, how is it changing things?)

• Extent of citations (and by which publications, and by whom? Who else is writing on the topic and using your work? This also tells you something about your policy community)
• Content of citations (how are people using your work to advance an argument?)
• Reactions to the research by key target groups (reviews, correspondence, social media comments)
• Trends in discourse on the topic (discourse analysis)
• Extent of policy initiatives reflecting the evidence put forward in the research

These are examples, and not intended to be a comprehensive list. However, if you had the data on at these indicators, you should be able to tell a complete story about what you do and what has changed as a result of your research – in your spheres of control and influence.

What about impact?

Measuring impact is another story, and we suggest that you develop impact indicators as a means of tracking trends in your sector, to see if things are generally getting better or worse, rather than to demonstrate your contribution to these trends. This is too much to ask of think tanks and is beyond the accountability ceiling of most.

Your basic health check for your policy influence MEL should be:

1. Do we have clear policy influence questions?
2. Do we have a clear policy influence strategy with specific objectives (activities, outputs and outcomes) that can be measured or observed?
3. Do we have all the levels right in our story of policy influence?
4. Do we have indicators that capture both what we do and what we want to change ? Is it clear who the actors are?
5. Do we have enough (but not too many) indicators?
6. Are we able to measure (gather and analyse) these indicators within our resource constraints?
7. Will we be able to answer our policy influence questions and learn how we can improve from our MEL efforts?

The main point is that you almost have to know what your story of change is before you start.  This can seem at odds with the research process but in fact you could argue that if your research leads you in a different direction, you will still be able to tell a new story. To tell the story, intended or emergent, it is important to be systematic and make sure you have all the aspects covered, capture what is necessary to be able to answer your policy influence questions and learn from your successes and failures.