Skip to content

Posts tagged ‘research communication’

Aug 29

4 Comments

“A policy brief is a piece of paper. It doesn’t DO anything on its own”

 

[This post has been updated]

Quite some time ago, Jeff Knezovich reported on a study that was due to be published: Should think tanks write policy briefs? In this post he wrote something that we should all keep in mind:

A policy brief is a piece of paper. It doesn’t DO anything, and is therefore unlikely to have impact on its own.

This is something I try to remind people of all the time when discussing policy influence and research uptake. In fact, most communication outputs by themselves probably aren’t very impactful. It is a shame, then, that these outputs tend to be listed as deliverables in contracts with funders and thus tend to become viewed as an ‘end’ rather than a ‘means to an end’.

Well, the paper is out: Can a policy brief be an effective tool for policy influence?

3ie and the Institute of Development Studies (IDS), in collaboration with Norwegian Agency for Development Cooperation (Norad), explored the effectiveness of a policy brief for influencing readers’ beliefs and prompting them to act.

A multi-armed randomised control design was used to find answers to three research questions: Do policy briefs influence readers? Does the presence of an op-ed type commentary within the brief lead to more or less influence? and Does it matter if the commentary is assigned to a well-known name in the field?

In response I posted a rather long email on the ebpdn discussion board. I repost it below with some edits:

I do not want to always be the one to say the obvious (and this probably won’t make me any new friends, I am afraid) but was all this really necessary just to conclude that:

  • Policy briefs have to have clear messages;
  • That people who are well-known and respected are more likely to be listened to than people that nobody knows about;
  • That design matters; and
  • That they should be targeted at the people who matter?

As Jeff’s blog shows IDS has known all this for quite some time. Even without an RCT, I am sure that its communications and outreach staff knew what they were doing. All of these ‘lessons’ are also key components of ODI’s research communication workshops (which go all the way back to 2009!) and the communications team there have had how to guides for much longer that say all of this and much more. CIPPEC and other think tanks working on the subject have been saying and doing this too.

RPC advocates might say that before this study we did not really KNOW any of this; we just thought we knew. But, did we really have to KNOW this?  (In fact, as I will argue below I do not think the study is robust enough for us to really KNOW if policy briefs work, why and when.) Surely there are other things that are more important and that we do not know about for which RCTs can be really useful… but all of this for a simple piece of paper?

I had a few comments to make on the paper:

First, the theory of change for a policy brief presented in the study is a perfect example of a very important confusion. There is a mistaken assumption made that evidence based policy and policy influence mean the same thing. This, as Emma Broadbent showed in her series of papers on the political economy of research uptake in four policy issues in Sierra Leone, Uganda, Ghana and Zambia, and Kirsty Newman latter blogged about, is just not the case. Evidence based or informed policy and policy influence are two very different things.

Second, there seems to be an assumption that, as in medicine, the reader-patient of the policy brief has no other influences-treatment. It assumes that there are no other forms of communication affecting his or her ideas. If the intention was to test the effectiveness of a policy brief surely the treatment should have included other types of communication channels and tools to see which one had the most significant effect (on its own or in combination which others -but which others? There is no standard mix of communication tools that is used all the time by all organisations in all contexts). Now, that would have been truly interesting. But then how does one create a comparable situation among all the cases? A control? Some people would have had to be isolated from all other sources of information for the duration of the study.

In a way they did include another tool, an opinion piece, as part of the possible mix. But since they did not just present the opinion on its own, how do they know then that the opinion of the respected researchers would not have been enough to influence the patient? Again, that would have been an interesting finding.

Third, the choice of topic of the policy brief presents another problem. The thing is, well, nobody really cares. Of course nutrition is an important issue but out of all the people who responded, not all work on it And even for those that did, how ideological is this issue for them? For researchers in developing countries the real challenge is not in the more technocratic issues like these. In these case the absence of change even if the evidence is clear is more likely down to lack of motivation or capacity to change and implement recommendations than poor communication of the findings. The challenge is in the ideological issues. Those value-heavy policy choices that the developed world deals with on a daily basis but that international development community tends to dismiss in the developing world. It would have been more accurate to test the effects of policy briefs that dealt with free trade agreements, subsidies, the benefits (or costs) of large mining projects or the privatisation of water? Now, that would have been interesting.

The paper is full of limitations; and most are described at length in it. A key issue that comes up in the analysis but does not seem to merit a mention in the section on limitations is that the policy brief focuses on a specialist subject about which not many people know that much about. So really, all that should matter is if the people who do know about it change their mind or not. It makes little sense to see if someone who knew little or nothing about something changed their mind after reading about it. Of course they would. Policy briefs are meant to target informed people -interested, too. They are never expected to, on their own, convince someone of something they disagree with or to get someone who does not know anything about an issue to get exited about it.

So of course one would expect an increase in the number of people who say they believe what the brief says and that the strength of the evidence has increased after reading it if they did not know anything about it before. And of course people who already knew about it would not be likely to dramatically change their beliefs or opinions about what they already feel they know. But again, a policy brief is not there to change people’s minds but to inform them of a course of action. And in fact this is what the study found.

It seems to me that the study ought to have been clearer about what a policy brief is and is not for. And maybe it should have been clearer that  a policy brief is never published alone. If a policy brief is put out as a stand alone output then this should be seen as a failure in communication and so it does not seem appropriate to encourage research centres to do so by suggesting that all may be ok as long as they follow the recommendations of this paper.

The format of the policy brief is also important. The test used in the RCT is based on an IDS format -which I have no issues with (except that 4 pages may have been better than 3). But, as I found in a review of a RAPID project to help IDRC programmes in Latin America, Africa and Asia to develop policy briefs back in 2008 (if memory serves me right) different organisations have different views of what a policy brief ought to be. This is because different academic communities have different writing styles and expect different things from their researchers. Policymakers, who have been part of those communities at university, too, expect different things from researchers. So the 3 page (which by the way, is odd; ideally make them of 2, 4, 6 pages, even numbers so they can be easily printed and folded) policy brief may work alright in the UK but might not work in Vietnam or in Egypt. And the writing style used by the paper may excite some but baffle others.

An important question to ask about the effect of the tool is what it audiences did as a consequence of reading it. On this, nothing seems to be particularly new. The literature, and every communicator in the world knows that things that take more effort are less likely to be done by the recipient of a message. We all know that. In fact, that is the basis of Outcome Mapping’s Progress Markers: first reactions, then more active participation and engagement, and finally taking on the initiative and more transformative changes. Maybe they did an RCT we are not aware of.

Effort, of course, is linked to the power that people have; another ‘big’ finding of the paper. And power is likely to be linked to their education level and position in their organisations. So it is rather obvious that more powerful individuals will be more likely to act on the recommendations of the policy brief in ways that require resources and other people to do things for them.

Finally, the authors have something to say about the power of influential people/names. They argue that the reputation of the messenger can convince the reader. Well, in page 73 the authors mention a study by RAPID that contains a survey related to researchers’ view of the role of evidence in policy and the value of policy briefs; and they use its findings as undisputed fact. Unfortunately, they did not check the survey. If they had they would have found that the survey was far from representative of all researchers (I am sorry but 200 or so researchers from a an ODI/SciDev mail list is not representative of all researchers in the developing world) and so the correct way of expressing Jones’ and Walsh’s opinion (because that is all it can be said to be) is that half of the respondents to their survey (which was terribly biased to people we knew) were of the view that research communications are of poor quality and that policy briefs could help. So, certainly in this case, the authority effect worked here. Priceless.

In the end the paper does not tell us if a briefing paper works (without caveats), why it works and when it works. This is what one needs to know. Does it work best at the end of a study? To set the agenda? To advice on implementation? Does it work best when combined with blogs, opinions, videos, a working paper, a good presentation, personal networks? Is it better emailed or delivered personally? Is it better if it is 1 page? 2 pages? 4 pages? 6 pages? What about the writing style? How many tables should it have? What colours are the most appropriate? Surely the effort to test once and for all if a policy brief works should lead to answer some of the questions. Otherwise, I think I’ll stick with common sense.

I am not trying to discourage the use of RCTs but I do feel that this is a bit too much. None of this is new and it did not need to be studied in this way. And I think that the medicine metaphor has been taken too far, as well. Of course, do not take my word for any of this. Do, please, read the paper and make up your own minds.

My reaction is in part fuelled by the fact that the cost of the study would have paid for a very good communications expert to work for a year at a think tank in a developing country. In fact he or she could have helped a few think tanks in that country using the various channels and tools available to them and, most of all, using experience and common sense. The USD20k plus (I must say that this full disclosure is worth mentioning and applauding) spent could have, for example, paid for a couple of years of running the ebpdn in Latin America -which now does not have an active facilitator. It could have funded several study tours, a regional conference like the one organised in Latin America last year where about 40 think tanks came together to learn from each other and present original papers and ideas. I am sure you can think of other better ways of using this money.

As I have in the past recommend researchers and think tanks to just get on with it. Do not wait for the RCTs on blogs, opinion pieces, videos, twitter, working papers, etc. There are lots of ways to communicate research, and all you need to do is pick the mix that works for you and your organisation (for no other reason than it makes sense and you have actually thought about it and that it is within your reach), and then make sure you do things right. If you need help there are people (like Nick Scott, Jeff Knezovich, Vanesa Weyrauch, Laura Zommer or Lawrence MacDonald and others) out there thinking about this and who I am sure willing to lend a hand and provide some thoughtful advice. But remember that a poorly written briefing paper, a press release a day too late, a busy and static website, a boring event, or a poorly scripted video will not work regardless of the brilliance of our ideas; and that will be more a testament of our incapacity than of the ineffectiveness of the tool.

…

As an afterthought. It is interesting that the study did not consider if a policy brief with proper and robust research would have been more influential than one with questionable research -even if it had a clear message, was written by someone who is well-known, was nicely designed and accurately targeted. Now THAT would have been interesting.

The response to my post has been mixed on the ebpdn discussion list. 3ie, IDS, and others have defended the paper and the innovative effort of the researchers:

Kirsty Newman:

On Enrique’s question about whether it told us anything new, I agree that the findings support my pre-existing suspicion BUT remember that not everyone thinks like we do! There are plenty of examples of research communication strategies where the final objective is the production of a policy brief (or creation of a portal or holding of a seminar or whatever). This research is important evidence which helps build a case that if you seek to influence policy, you need to do more.

Maren Duvendack from ODI:

I have taken quite an interest in the 3ie-IDS-Norad study as I actually think that it is a pretty good idea! Looking at the influence of research on policy is pretty tricky (that’s essentially what the study is trying to do). Mainly qualitative tools are used in this context and many people (mainly economists!) find this very frustrating and crave some sort of quantification of the policy influence of research and the study does exactly this! Quantifying policy influence of research is not new, some people (mainly economists again!) tried calculating internal rates of return for example but that’s pretty flawed too.

Anyway, this is just another RCT and given their limitations one should always take their results with a pinch of salt!

But there have also been some voice supporting my critique:

Nick von Behr from behroutcomes.co.uk

As a bit of an outsider to Development I agree with everything that Enrique says in terms of policy influence. Better to spend the money on actually doing it through trial and error rather than taking a hugely scientific approach, laudable though the efforts have been. And as he says what matters is the balance of research evidence that actually supports/contradicts a new policy direction in whichever field (mine is education).

And from an actual communications practitioner like Maryam Mohsin also at ODI:

As a long time comms professional I find it a bit bizarre that so much time, resource and effort is being invested into applying RCTs to test how we can influence policy. Are we really trying to find a one size fits all way to communicate? Anyone with a bit of common sense will tell you how far that will get you in the complicated world of policy influence. And any communications professional who needed this RCT to tell them the key messages it contains should be fired immediately (I say in half jest – but I do wonder who this study is aimed at).

I simply do not believe policy influence is an exact science, it’s based on capacity, dialogue, strategy, pro-activeness and reflection. I think we need to stop trying to find ways of putting square pegs into round holes and shift away from placing emphasis on the tools, and move towards placing more emphasis on the strategic use of these tools.

We play a small part in the whole process of policy making, through to influence. We need to humbly recognise this and spend more time looking at what we do have the capacity to contribute towards and focus on being strategic in our attempt to give ourselves a better chance of success

This point made by Maryam echoes a follow up email I sent to the community in which I suggested that:

I see no future in this line of questioning. Sure, it is fun and I can see why a researcher would like to do it -but it is, in my view, useless and dangerous. I would encourage us instead to support organisations to reflect on what they do and how they do it. Invest in people. See Laura zommer’s fantastic arguments for inspiration. But don’t do what she says, follow her thought process and arrive at your own conclusions. Involve your communicators in strategic discussions. Involve your researchers in thinking about how they communicate and what influences change in their own policy communities. Come together in exercises like these supported by GDNet or the richer cases that this network supported in Latin America and Africa. THINK about it, don’t just wait for proof. There will be no proof. The IDS study’ caveats are way too large and significant to take any of its conclusions seriously. The best way forward is more and richer critical reflection (not just descriptive cases of successes).

Jul 5

2 Comments

INASP’s reflections on lessons from recent research communication capacity building experiences

[Editor's note: Dr Alexander Ademokun is the Acting Head of Programme for Evidence-informed Policy Making (EIPM) at INASP. This post is in response to Research communications support: why do donors, think tanks and consultants keep making the same mistakes?, Capacity development: it is time to think about new ways of generating and sharing knowledge, and Developing research communication capacity: lessons from recent experiences]

This is the third in a series of posts in response to a paper that Enrique and Martine produced after evaluating a capacity building for research communication project implemented by INASP and ODI.  There have been some very thoughtful discussions about this paper on the evidence-based policy in development (ebpdn) discussion forum and a couple of interesting blog posts from Caroline Cassidy from the RAPID team at ODI and Vanesa Weyruch from CIPPEC.

The report makes some key arguments and some recommendations from the assessment of this initiative.  During the discussions on ebpdn one of the points that came out was that these lessons, while focusing on a capacity building for research communication project, are relevant to capacity building initiatives more broadly.  Some issues from the discussion that I think are worth exploring further in the context of a wider capacity development conversation are:

  • The need to understand the internal systems of the organisation you are working with:  In the case of this project it is about internal communication systems but in other contexts it may be about the organisational culture or finding out how your project fits into a wider organisation strategy.  This takes time and is labour intensive but it is worth doing from the start.  It also ensures that even if you are building capacity at the individual level it fits into a wider institutional plan and the added capacity is more likely to be made use of.  Understanding where your project fits within a wider plan also reduces the tendency of responding to every call irrespective of ability or capacity to deliver.

For some service providers and intermediary organisations this may mean doing less but better and also being able to say no to offers of new projects.  There is an on-going conversation within the EIPM team at INASP about how we can balance the need to more deeply understand the organisations and context we work in whilst still finding the time to do all the activities we would like to.

  • Work locally and build on what already exists:  This message came through very strongly from Enrique’s and Martine’s report.  This raises some issues that we need to engage with. For instance working locally presumes the capacity and infrastructure exists to deliver the goals of the project.  If they do, that’s great.  If they don’t, have you got the time and expertise to truly build the necessary capacity at the target institution?  You may even find after engaging with the organisation that you are not best placed to deliver what’s needed – will you say so?

The tendency to plug a capacity gap with a highly visible workshop is strong but we know that to build long term solutions you will have to engage more.  This involves asking whose goals you are working towards and how flexible these goals are – is the capacity that is being developed a goal in itself or a means to an end?  The report talks about developing communication strategies with no money for implementation.  This led to lack of interest and lack of ownership.  The model of building on what already exists is illustrated by the example that Enrique gave on the ebpdn where he has decided to take a step back and work with organisations to build organisational communications strategies before building a strategy for a particular external project.  We can all learn from this approach but it requires commitment from the grantees, intermediaries and donors to recognise that we are not just thinking about our specific project but, again, about the sustainability of the capacity you are trying to develop.

Interests, incentives and commitment:  We need to take the time to understand why participants in an initiative are there.  The report mentions some participants who took part because they felt it was important to donors.  For some of us trainers or service providers an opportunity to try something new or work with a particular organisation may be our incentive.  We need to be clear about what we are each trying to achieve by being in the room before we even get started.

This is also linked to commitment.  It is easier to commit if we know what we are committing to: what does the end of the project look like and what happens at the end?  Does the end mean funds stop coming in, mentoring support stops or does the end simply mean a date three years’ down the line?  Working with all involved – funders, grantees and intermediaries to clearly define the end of a project (and what it means beyond financial support) is important at the start of the project.

Use the right people and understand the context:  The report highlighted the value of using local or regional facilitators who may have a better understanding of the context.  Over the last few years, INASP has used the training of trainers’ approach to build a cohort with both the capacity and the remit to deliver capacity building initiatives locally.  A report from a recent workshop in Asia for trainers of policy makers can be found here.  The years doing this work has taught us that just because you are a subject expert does not mean you are a good trainer.  Spending the time to find or build the capacity of trainers to train is just as important as developing or delivering content.  Likewise getting the trainers to understand the content and context before jumping in to deliver an ‘interactive’ workshop is important.  There is only so much small group work/drawing/flipcharts can do if your participants think you don’t understand their realities.  This need to understand how to train is an often undervalued aspect of capacity building.

Linked to this is the understanding that workshops are not the magic bullet they are often thought to be.  At INASP we use workshops as part of a package of activities to engage.  Sometimes this may mean taking the same group of participants through a series of workshops instead of trying to deliver everything in five days.  Other options include adding on a mentoring process before and after a workshop or mixing workshops with other learning models be they online, country visits or peer-exchanges.

We know most of this but don’t always do it yet we respond when the same issues are raised.  This tells me we want to do better.  Using the opportunities and networks we have to share our learning and constructively challenge our approaches is a good thing and I hope we carry on doing more of it.

Jul 4

6 Comments

Capacity development: it is time to think about new ways of generating and sharing knowledge

[Editor's note: Vanesa Weyrauch is the Principal Researcher of the Influence, Monitoring and Evaluation Programme at CIPPEC, which she created and has led since 2006. She founded a network of leading think tanks in Latin America, with the support of  GDNet, Directores Ejecutivos de America Latina (DEAL). This post is a response to Developing research communication capacity: lessons from recent experiences and can be read alongside Caroline Cassidy's own post]

I am not an expert on capacity development per se but I’ve been a practitioner from CIPPEC of a combination of activities during the last years in Latin America which have helped me reflect and learn on what seems more promising in terms of helping others improve the way they try to influence policy through research. Much of this work has been performed under EBPDN LA with ODI and most of it under the program “Spaces for engagement” with GDNet, a strategic partner for us in this field.

This is why Enrique’s and Martine’s review findings from a recent communications project that RAPID and INASP worked on for IDRC have highly raised my attention. Just in time, I thought. After a couple of years of trying out several ways of improving what we know about policy influence (combining formal training workshops with online courses, technical assistance to think tanks, design of an innovative M&E methodology focused on policy influence, etc.) we have decided at CIPPEC to develop a capacity building strategy for 2013-2018 that allows us to be more strategic in the way we use our limited resources to assist policy research organisations and its leaders to enhance policy impact.

Firstly, I believe that some responses to questions posed by Enrique in his previous post and some of his recommendations may vary according to each type of individual/organisation taking part of the initiative (they tend to be very heterogeneous, which on the one hand enriches the exercise but on the other hand makes it extremely hard to please all participants). At CIPPEC we had experiences in training networks on these issues and even though they might share beliefs, values, research projects, etc, each network member had very different capacities and interests in research communications revealed in a pre-diagnosis. So how do we deal with this when resources are scarce? Ideally we would have all the time and resources to work both in groups and individually to support real change processes with viable plans and enable cross-learning but this is not the case in most of the cases. Therefore we face the challenge to make the most of what is available; smart decisions that use the evidence shared by Enrique and our own experience are crucial then.

Another key and related decision is whether those offering the support aim at training individuals and/or organisations. Strategies to do so will differ significantly and it is extremely difficult to make these decisions at the very beginning of a capacity building project and when there is a diverse group that will take part of it.

Finally, another tricky but very profound question is: How do we monitor and evaluate these efforts? How do we know if and how we have contributed to developing this sort of capacities? I agree that stand-alone workshops are not then most desirable strategy but I’ve heard/seen persons and organisations making a big change after attending one where excellent trainers have been able to raise their awareness on these issues and spearheaded the right questions at the individual/organisational levels. Thus, what are we aiming at and how we will know if we’ve done good?

An excellent paper that has significantly influenced how I think about all these issues and how we plan to further develop our capacity to build capacity at CIPPEC is “Learning purposefully in capacity development. Why, what and when to measure?” by Peter Taylor and Alfredo Ortiz. We need to develop new thinking about these issues and this paper triggers this type of thinking for all of us: donors, “trainers”, “trainees”. As we titled one of our handbooks, I believe we are all a little bit of learners, teachers and practitioners. That’s why ways to generate and share knowledge are increasingly horizontal! For us, online courses have enlarged the opportunity to make this happen as the knowledge is shared and discusses between peers and colleagues. What participants of our courses ask, the reflections they make, the real live examples they share, have all largely enhanced the knowledge we share in the next edition of the same course.

Finally, I am more and more convinced of the value of constant cross-fertilisation between theory and practice (and that’s what we’ve tried to do in our work all these years) which will significantly affect what we consider valuable knowledge and how we share it. Sir Ken Robinson  has very effectively conveyed the importance of rethinking education: there is a real and increasing call for creativity in the way we treat knowledge. For this, group learning is key, collaboration is the strategy for defining crucial questions and co-building the answers. Spaces -like this blog- where we can share what we know and don’t know about the topics that we are passionate about are a promising sign of how capacity of all-teachers, learners and practitioners (which are changing roles for all individuals) can be further developed.

[Editor's note: If you'd like to join the conversation with a post of your own, please send us an email or tweet. See the original post for inspiration: Developing research communication capacity: lessons from recent experiences]

Jul 2

11 Comments

Research communications support: why do donors, think tanks and consultants keep making the same mistakes?

[Editor's note: Caroline Cassidy is the Research and Policy in Development (RAPID) Programme's Communication Officer. This post is a response to: Developing research communication capacity: lessons from recent experiences and can be read alongside Vanesa Weyrauch's own response -coming up this Wednesday]

Building capacity to develop research communications skills and competencies for policy influence is not a new thing. There are a multitude of players involved in the process who have been working in this area for years. And evaluating that capacity development is not really a new thing either. So why then should I be writing this blog if what I am about to say is nothing new? Because, despite clear recommendations for better support, time and time again, donors, think tanks and consultants keep coming up against the same challenges, leaving research communication to the end of the project, then getting caught up in a cycle of workshops and interventions that are unlikely to have the desired impact, and when researchers or teams are already looking to their next area of work.

I arrive at this type of capacity development from ODI’s Research and Policy in Development (RAPID) programme where I have been working with the team to build on ODI’s years of work helping to develop the capacity of researchers and organisations in a variety of contexts, to have impact in the policy realm. Enrique and Martine’s evaluation findings from a recent communications project that RAPID and INASP worked on for IDRC last year identify  some very interesting (though sadly not all new) issues that frequently surface when we do this type of work: contextual concerns – in a short space of time, can a consultant really get to the crux of the project without having a strong working knowledge of the context themself; support often comes at the end of a project so that therefore it feels like it is ‘tagged on’ as an extra dimension, rather than an integral one; and ensuring you have the right people in a team involved in the first place, who can benefit the most from the support.

One recommendation from Enrique and Martine that I don’t think we at ODI have seen before is assessing demand and talking directly to the grantees who need support before a contract is even signed, then deciding whether this capacity support should be provided and to whom. This is also related to another report lesson on researcher incentives and pressures beyond communications and the fact that many do not believe it is their role to engage at all – that it is someone else’s job. Therefore, assessing the demand and finding the right people within the organisation to work with as early as possible is absolutely critical, (and then re-evaluating this throughout the duration of the support, as circumstances alter). And if it looks as if it’s not going to have the necessary impact – consultants and think tanks should have the ability to just say no from the outset.

Yet, despite these and other well-established, clear and very sensible principles, there seem to be a few key confounding factors that often impede their implementation:

The first is funding; although there is a growing consensus of the importance of communicating research, funding for communication has undoubtedly suffered at the hands of the economic downturn and the growing ‘value for money’ agenda. It is not always seen as a major priority in the research cycle and often too closely, and even wrongly, associated with branding and marketing, rather than policy influence. Moreover, even in the communication arena donors often favour interventions that lead directly to visible outputs like, the workshop.

Secondly, as Enrique and Martine emphasise, there is often poor planning: donors and organisations realise quite late into a project and budget cycle that the teams need extra support in this area, but with not much time and little funding, a ‘quick’ workshop is often seen as an immediate ‘magic wand’.  As a blog by my colleague, Ajoy Datta highlights – workshops do give a good introduction to the topic and some initial support, but are unlikely to make a real impact once the participants have left the building.

I also think that there is still the misconception, at some levels, that researchers and teams shouldn’t be thinking about the communication of their work until later in the process or indeed towards the end. However, whoever leads on communications needs to engage with stakeholders as early as possible to ensure relationships are cemented and that ideally decision-makers have buy in.

And finally, well even if they could do all of the above, donors frequently do not have sufficiently flexible mechanisms and incentives to support a more appropriate response, as discussed in a recent ODI background note: Promoting evidence-based decision-making in development agencies.

So faced with all this doom and gloom, what can be done? While workshops can still be useful, in RAPID, we are now trying to incorporate them where possible, as part of a wider and longer involvement in a project, and one where ideally we are involved from the beginning. For example, we are currently working on a two year project with the International Initiative for Impact Evaluation (3ie) on communications support to their grantees, knowledge management development (at an organisational level) and another three year project on monitoring and evaluating grant policy influence. The latter is in consortium with three other regional organisations: CEPA (Sri Lanka), CIPPEC (Argentina) CommsConsult (Zimbabwe).  It is an exciting, though we recognise, rare opportunity to work at different organisational levels to do some thinking, develop tools, research and capacity work in a ‘quality learning laboratory’. Support will be provided by locally based teams working in context, prioritising face-to-face engagement (which does include workshops!), but also using online engagement where necessary. All of this will hopefully help to ensure better impact, longevity and buy in through stronger, more collaborative relationships between researchers and policy-makers, and from our side, better contextual knowledge.

And for other projects, where we are working with smaller organisations and donor budgets, we are trying to ensure that there is additional support around the workshops through mentoring, field trips, local partners and we will certainly take on board the recommendations put forward by Enrique and Martine.  And sharing evaluation findings in early discussions with donors can make a big difference. An organisation I am working with decided to implement more face-to-face support, because the donor read and assimilated the recommendations from another project evaluation report.

Communications capacity development is a constant learning process and there is no best-case, winning magic formula. But nor should there be – because good support is so dependent on the organisation, project, participants and the context, and just ‘shoehorning’ a ready-made approach or template is not going to work. This report contains some useful principles to guide new forms of support and to encourage donors, think tanks and consultants alike to not fall into the same traps of short-term support that frequently only deliver mediocre results. And above all, interventions are far more likely to become embedded into the life of a project (and hopefully beyond) if they are part of the project from the beginning and not left as an afterthought.

[Editor's note: Vanesa Weyrauch's response will come out on Wednesday but if you'd like to join the conversation with a post of your own, please send us an email or tweet. See the original post for inspiration: Developing research communication capacity: lessons from recent experiences]

Jun 11

11 Comments

Developing research communication capacity: lessons from recent experiences

[Editor's note: This is the first of four blogs on the subject: Research communications support: why do donors, think tanks and consultants keep making the same mistakes?, Research communications support: why do donors, think tanks and consultants keep making the same mistakes?, Capacity development: it is time to think about new ways of generating and sharing knowledge, and INASP’s reflections on lessons from recent research communication capacity building experiences. Join the debate.]

Donors spend millions every year trying to build the capacity of researchers to communicate their work more effectively. Unfortunately, most of it goes on one-off workshops and attempts to get them to do things they are clearly uninterested in. Sometimes it feels that lessons are hardly ever learned. But sometimes opportunities come about that let us reflect and learn.

Last year, ODI and INASP asked Martine Zeuthen and me to review their efforts to build the capacity of a series of IDRC funded research programmes in Africa. We assessed each one separately and then brought both reviews together in the synthesis below. We found, among other things, that a lot more time needs to be dedicated to planning the interventions.

I am now trying the recommendations listed below in a project I am working on this year with four Latin American think tanks. I’ll report back on how it goes.

Lessons (more detail on these lessons, the recommendations below, and the approaches themselves, is provided in the document in Scribd or GoogleDocs):

  • The best laid plans… In both cases, as well as in other cases consulted for the purpose of this review, the interventions did not go as planned.
  • An expression of interest does not always imply commitment: Although the grantees had expressed their interest in being involved in the projects several were not engaged in learning and did not change their approach to research communication as a consequence. In one case one of the grantees expresses that their involvement was based on the impression that the project appeared to be important for IDRC and ODI. In other words, their participation was driven more by an interest in being part of such initiative to satisfy donor demands rather than in the initiative itself.
  • Researchers have other interests and pressures besides communications: Most researchers are often more interested in researching than communicating. Additionally, while an individual project may be a priority for the donor or for the lead partner, it is unlikely to be so for individual organisations or researchers. As a consequence, any activities that are not seen to directly support their core business are unlikely to be given the priority they demand to be effective.
  • Face-to-face is better than virtual, but the web is a good alternative
  • If it is not done at the beginning, then it is probably too late: In all cases the project came about as a final activity for the grantees, added to the project with only months to go. Furthermore, while the support provided was intended to lead to a communication strategy, there were no additional funds to implement such a strategy. As a consequence, researchers had few incentives to engage more than necessary.
  • The right people matter: The ambition was for the people receiving the support to then go on and train or mentor other members of their networks or organisations. Unfortunately, those who participated where not always the right people for this objective. Senior researchers, network coordinators, and even communicators may be excellent candidates to make use of any skills learned  but that does not necessarily make them the most appropriate ‘trainers of trainers’.
  • Local or regional facilitators and mentors: INASP’s approach involved using regionally based facilitators and mentors. This had a particularly positive effect on the project. The partners learned from the mentors and enjoyed discussing the specific challenges that they were facing with regional professionals. Conversely, ODI was able to connect with the grantees it was supporting only after visiting their offices, and concerns about the consultants’ lack of familiarity with their context were raised.
  • No one is starting from scratch: All the grantees, to different degrees, have some sort of research communication capacity. In some cases, their personal and professional networks ensure greater levels of impact than any formal research communication strategy could ever promise. Furthermore, many communication tactics and channels that are common for developed countries or the United Kingdom, and that ODI and INASP are more familiar with, may not be appropriate for the grantees’ contexts.

Recommendations:

  • Start early –right from the beginning: Developing the capacity to communicate should not come as an afterthought. Funders must plan this right from the start and service providers like ODI and INASP should be careful about being involved if this is not the case.
  • Confirm demand before starting: Even before signing a contract, the service providers should contact the grantees and effectively treat them as clients; inquiring as to their interests, concerns, and commitment to the initiative. The service providers must be very clear regarding the time and resources that they will have to allocate to the process. They must also discuss, at length, who are the most appropriate people to be directly involved and what will be their responsibilities.
  • More than a needs assessment: really understand the organisation and its context: The service provider should start by either spending time with the organisation or hosting the relevant people. Above all, the service providers need to understand the culture of the organisations and the policy contexts they seek to affect. This is not something easily achieved through a remote diagnostic.
  • Consider who is the most appropriate source of expertise: It may be that the organisations conducting the assessment are not necessarily the most appropriate when it comes to delivering the support. Would they limit their recommendation to the services they can offer?
  • Build on strengths: The service providers should seek to either improve what they already do or introduce new channels or tactics that build on those that they are comfortable with. This is likely to make a bigger impact than if the consultants bring along an entirely foreign and all-encompassing new approach.
  • Focus on the organisation rather than on single projects: Support should be aimed at strengthening the organisation’s capacity and not just a single project’s visibility. This is likely to attract the support of senior managers that is crucial for any change to take hold within the organisation. The project itself can be used as a pilot to text the new tactics or channels proposed.
  • Earmark funds to implement whatever strategy they develop: It is unlikely that the organisations will dedicate the necessary time to develop a strategy or plan unless they know that there will be funds available to implement it. Just as the service providers are not helping for free, it is unlikely that these researchers will be able to dedicate the necessary time to the initiative unless their time is covered. The service provider should therefore make sure that there are sufficient funds for this purpose. On the other hand, if the organisation has the funds but is not willing to allocate them to this purpose this should be seen as a sign that there is little buy-in from the leadership.
  • Maximise peer-to-peer exposure: Depending on the kind of skills being shared and the individuals involved, the donors and service providers should attempt to ensure that people with the right experience deliver the support. Researchers, for example, are more likely to respond to other researchers; communication officers to communication officers; and managers to managers. This means that it is possible that the service providers will have to look beyond their organisations for the right expertise. Instead, they may act as facilitators and help the organisations find the most appropriate people for their needs.

Read the full report here:

View this document on Scribd


Follow

Get every new post delivered to your Inbox.

Join 3,400 other followers