RAPID Outcome Mapping Approach: more useful when used to plan the whole policy research initiative

24 February 2012

(Please note that in this post I am referring to policy research initiatives or programmes: initiatives that have explicit policy influencing objectives.)

The RAPID Outcome Mapping Approach is a methodology that I helped develop while working for the RAPID Programme at the Overseas Development Institute. I think we made a mistake (well, more than one, but let me focus on this one today). RAPID has always been in high demand when it comes to helping policy research organisations and programmes to plan, monitor and evaluate policy research influencing strategies. It is (and I still am) called to this help after the overall policy research programme has been designed: the objectives (and logframes) have been decided and the contracts has been signed with the funder.

Our mistake was to pitch it this way. We accepted (or did not care to challenge) the idea that there were separate components: research, capacity building, ….., and policy influencing (which focused mainly on communications), and that it was the latter that ROMA could help with. We let the researchers deal with the research component and took it as a given. There is a reason for this. Historically, RAPID has been seen within ODI as non-research-based (even though its work is quite solidly based on a great deal of research) and so we chose to focus most of our attention away from discussions related to the panning of the research component. We assumed (and it could still work) that this was a safe way in. Unfortunately, researchers, under pressure from donors to focus more and more on communications, still protect the research component and shield it from approaches such as ROMA. It is my impression that they are willing to talk about policy influence, research uptake, communications, etc. that as long as the research component is not affected.

But this is their mistake. ROMA is not that useful when it is brought in after the research component and the programme’s objectives have been decided. ROMA (and other similar approaches) is much more useful when it is used to plan the entire programme: including the research component of a policy research programme.

ROMA is about critical thinking. That is all it is. It can be used in any situation (big or small) and circumstance because it facilitates a process of reflection about our context, organisations, skills, objectives, partners, audiences, tactics, tools, how to use them, why, etc. It helps us to explain why we are doing what we do -and check and re-check if it is the right thing to do as more information becomes available. Users go through a narrative that helps them to identify and define objectives, think about the policy (broad and narrow) context that affect them, identify the main players in this context and those that the programme may want to target, determine more specific objectives for each, consider various ways of achieving them, developing and choosing the most appropriate approaches, tactics and tools, etc.

Among these approaches, tactics and tools are the usual: media campaigns, training and education, digital communications, networking, …, and research. Yes, research. In a policy research programme, research (analysis, literature reviews, case studies, systematic reviews, impact evaluations, randomised control trials, clinical trials, etc.) is a component of the overall programme; just like communications, capacity building, networking, etc, are components of the programme, too. Hence new research, like some of the activities of the other components, is not indispensable. It very well be that it could be possible to affect policy by focusing on using existing research and just promoting a public debate on a policy issue; or by improving the capacity of governments to make more informed decisions; or creating formal links between policymakers and experts; etc.

Similarly, it very well be that new research is absolutely necessary. In these cases, however, the research design cannot happen in isolation of policy influencing considerations. What kind of research is the most appropriate? ROMA can help decide what kind of research might be more relevant or useful to achieve the programme’s objectives. What questions should it answer? ROMA can help decide what questions need to be answered to develop the arguments that may influence the programme’s audiences. Should it be done collaboratively? ROMA can help decide. Who should we collaborate with? ROMA can help. What should be the outputs (products) of these research projects? ROMA can help. It can even help us decide who should be the researchers. I recall a case when a minister told me that the government had no problem with the research methods and conclusions but could not really use findings from the researcher who had carried it out. In another case, ROMA helped to avoid this situation. I say ROMA but of course I mean ‘a planning methodology like ROMA.’

The problem is that all these questions are currently decided before a discussion about the context, audiences, policy objectives, and the other components of the programme is had. Even the proposal writing process (and I have participated in many of these) is compartmentalised and often separates research from communications from capacity building from M&E. Each section tends to be drafted separately and then put together a few days before the deadline, the logframes are prepared at the last-minute, and all is then submitted to the donor. And all this is done before any real analysis of the policy context has been undertaken. I know this because whenever we come in to help with policy influencing the first thing we do is ask about this; and the answer is often the same: no. But by then it is too late.

Here is what I propose:

  • Before developing a strategy the donor or the organisations bidding for the policy research programme should carry out a ‘baseline’ study of the policy they intend to affect. This could be a political economy analysis of the policy process, or a study of the discourses that shape it. It should identify the various players involved, their interests, objectives, their use of evidence (or not), networks, etc. AusAid has recently conducted a series of diagnostics of the knowledge sector in Indonesia that could serve as an example. The kind of studies that Emma Broadbent has carried out on policy debates is also relevant.
  • This should help to clarify the policy objectives for the entire programme; they will be based on a realistic assessment of the context. Everyone this days seems to be talking about Theories of Change -but few base them on sound theories of how change actually happens.
  • In turn, these should help to consider which players the programme is proposing to focus its attention on and how is it that it could influence them or contribute towards changing their policy behaviours. Contribution here is the key word.
  • This focus should also help to decide what may be the most appropriate approaches, tactics, and tools for the programme to employ. And this will include, possibly, a research component. Depending on the audiences and objectives this may be very theoretical, a bit more practical, quantitative, qualitative, participatory, etc.
  • The research component, when designed at this stage and not before, will benefit from having a baseline that explains what, how, and why research is used, a clear audience among the key policy players, clear objectives, and a good sense of what are the other approaches (tactics and tools) which will be able to support and use research. This research will be inevitably better linked to the whole programme and not an isolated component developed before anyone bothered to think about the context.
  • Once the strategy is developed, and only then, the right team can be assembled. Today, bids are put together after the programme ‘partners’ and staff have been identified. The right order, however, is to find the right organisations and people for the job. It should not matter if they are in someone else’s team. Imagine if you hired someone and then checked to see what they could and could not do. This is the same thing that happens now.
  • Finally, and key to all of this, the programme strategy should accept that this process needs to be repeated over and over again. As the programme is implemented new information will become available, new challenges will appear, new opportunities will unravel, etc. Therefore, new approaches may be more appropriate, new partners and staff may be needed, and old ones may have to be let go.

This is not advertising for ROMA. I do not really mind what planning approach is used. What I am arguing is that for policy research initiatives, planning research and planning policy influence should not separated.