Nov 12
Julia Day and Adrian Ely pull back the curtain on their New Manifesto Project and how it was used to communicate highly technical research approaches on the social dimensions of science and technology.
Mar 30
[Editor’s note: Jeff Knezovich is the Policy Influence and Research Uptake Manager for the Future Health Systems research consortium, and a frequent contributor to On Think Tanks. This is the first in a series of posts examining the role of policy briefs in communicating research. For a more up to date critique on the final paper see: “A policy brief is a piece of paper. It doesn’t DO anything on its own”]
There is no doubt that policy briefs are an important tool for research communications, especially for policy-oriented think tanks. A recent study from CIPPEC found that 80% of think tanks surveyed across Latin America, Africa and Asia produced some form of policy brief.
‘Yeah, but do policy briefs actually DO anything? Do they actually have impact?’
As someone with ‘policy influence’ in my job title, and as someone who trains researchers and other research uptake professionals on the development of policy briefs, it’s a question I get a lot.
In fact, I remember in January 2011 sitting in Delhi at the headquarters of the International Initiative for Impact Evaluation (3ie), and being posed that exact question by their Senior Advocacy and Policy Officer, Christelle Chapoy. The answer is, of course, both incredibly simple and incredibly complicated.
The simple answer
A policy brief is a piece of paper. It doesn’t DO anything, and is therefore unlikely to have impact on its own.
This is something I try to remind people of all the time when discussing policy influence and research uptake. In fact, most communication outputs by themselves probably aren’t very impactful. It is a shame, then, that these outputs tend to be listed as deliverables in contracts with funders and thus tend to become viewed as an ‘end’ rather than a ‘means to an end’.
Here’s where it get’s a bit more difficult. Policy influence requires a concerted effort to engage target audiences at different times with different information and messages – which is why putting together a clear strategy is such an important first step. The type of engagement necessary will depend on the context within which one is working and might depend on a number of factors including the stage of the policy cycle, how controversial a topic is, the capacity of the policy makers to understand evidence and research, the strength of the evidence, etc. (see the RAPID framework for a good list of things to consider).
Getting something on the policy agenda, for instance, requires different information (and a different strategy) than improving the implementation of an established policy. This, in turn, requires a variety of targeted communication products.
Probably even more important is some sort of human interaction – never underestimate the power of a cheeky pint (or more culturally appropriate informal meeting, whether it be tea or coffee, or even a breakfast) with your key audience. In that context, passing them a policy brief might be an easy way for them to recall parts of your discussion, or act as a way to establish credibility in your discussions.
The (even more) complicated answer
OK, so we know that it probably takes more than a piece of paper to influence policy. That said, can some policy briefs be more effective than others? Absolutely.
At a seminar earlier this week at IDS entitled, ‘Research, Knowledge, Action – What can a policy brief really achieve?’, researchers from IDS and 3ie (Penelope Beynon, IDS; Marie Gaarder, formerly of 3ie; and Edouardo Masset, IDS) presented some initial findings from a research study looking into the utility of policy briefs as a tool for research uptake. And I was glad to see an attempt to answer the question posed over a year ago.
Using a randomised control trial, or RCT (which presented its own series of challenges), they tested to see what sort of actions a relevant audience might take. They were particularly interested in digging deeper into a finding from Jones and Walsh (2008) that indicated that nearly 80% of researchers and policy makers in the science, technology and innovation field valued the opinion of the researcher in addition to just presenting basic facts. In other words they were looking at the role of personal authority and credibility.
After inviting participation from some 70,000 individuals through IDS and 3ie contact lists, 807 participants self-selected to participate in the study (which, says a lot right there!). Participants were generally well educated and interested in the field of agriculture and development. Participants self-identified their level of influence at various levels of policy – some indicated they had more and some said they had less.
They divided the sample into four groups. The control group was given a policy brief from IDS vaguely related to the topic of the study. The first experimental group was given a two-page brief that summarised findings from a systematic review on agriculture and nutrition, particularly looking at bio-fortification and home-based farming. To another group they circulated the brief and attached a two-page opinion piece by Lawrence Haddad, Director of IDS. For a third group, they circulated both the brief and the opinion, but this time from an unnamed IDS research fellow.
I’ll leave it to the researchers to publish a more thorough analysis of their findings, but a few points from their study I found particularly interesting:
It’s clear that randomised control trials can tell us something about the utility and use of policy briefs. But I would also argue that RCTs are not the only arbiter of good knowledge. As such, in the next post I will reflect more on my past working with and developing policy briefs for a range of audiences, to see what experience can tell us about how to make policy briefs more effective.
Jun 23
By Jeff Knezovich, Policy Influence and Research Uptake Manager for the Future Health Systems Research Programme Consortium*
In the world of evidence-based policy-making, we often struggle with its evil twin: policy-based evidence-making.
Proponents of the former dream of a world in which a problem is identified, those in charge commission rigorous research, several options are presented, the pros and cons of each are weighed and the best choice is enshrined into policy (with the assumption that said policy is then implemented and there is further monitoring and evaluation of its implementation to make tweaks and improve upon the policy).
Anyone familiar with policy making – whether within governments, companies, organisations or social institutions – will know that such an ideal policy cycle is far from reality. More often we see someone with a value or interest in a particular topic looking to build a case for a particular solution, which may become policy if the case is compelling enough. These issue ‘champions’ may be nefarious schemers looking after their own best interest, but case building is equally prevalent among advocacy organisations and politicians working for the greater good.
Case building is a conscious, active, purposive and biased process. Researchers like to pretend, of course, that bias is beneath them and that the scientific method and peer review guard against this. But research into human psychology would indicate otherwise. Even when trying our best to be analytically objective, the human subconscious falls prey to something known as ‘confirmation bias’.
Confirmation bias expresses itself in different ways among humans, but effectively it can be boiled down to a subconscious predisposition to find or give more weight to evidence/data/information that supports a pre-existing belief or pre-existing knowledge. In research this may mean seeking data that supports a given hypothesis (or conversely, ignoring evidence that doesn’t support the hypothesis). Don’t believe me? Try this little activity to see confirmation bias in action.
In the real world, this means that we often try to support our beliefs by going to information sources that already support our view. This is what makes political punditry of the likes of Fox News (on the right) and MSNBC (on the left) in the U.S. so compelling. They do the heavy lifting of interpreting reality to fit in with pre-conceived worldviews so viewers don’t have to.
Indeed it turns out that, not only do we tend to seek information that already agrees with us, but information sources (or knowledge brokers) that we can trust to do the same. And as Eli Pariser points out, the increasing use of advanced web technologies that use algorithms to filter out ‘irrelevant’ information means that people don’t even have to make conscious decisions to seek out trusted information – and might make it even more difficult for people actively trying to challenge their existing beliefs.
This poses a serious problem for those of us seeking to influence others to change attitudes and beliefs: how do we get inside a target stakeholder’s filter bubble? If it was difficult to do in real life, it’s that much more so when we’re trying to game an algorithm.
I suggest several strategies:
Apr 19
‘Capoeira mata um’, or perhaps more accurately, ‘capoeira foi morto por um’ – at least on one sunny day last summer.
By Jeff Knezovich, Policy Influence and Research Uptake Manager for the Future Health Systems Research Programme Consortium*
Now if you’re like 99.9% of readers of this blog, you’re probably wondering a) why Enrique let me do a guest blog, and b) what in the world Brazilian Portuguese has to do with austerity communications. Let me explain.
‘Zum zum zum’ is a popular song in the capoeira circuits. Indeed it is so popular that you might recognise it from certain Mazda adverts. Zoom zoom.
The first line literally means ‘capoeira kills one’, but that’s not what this story is about. It’s about a time where ‘capoeira was killed by one’, how that has changed the development communications landscape, and what lessons development policy entrepreneurs can draw from the famed Brazilian martial art/dance.
When the Conservative-led coalition came into government in the UK last May, one of the first targets in their crosshairs was ‘profligate’ Labour spending, which they argued had left the country in dire economic straits. ‘Communications’, synonymous with spending, quickly became a dirty word across Whitehall. And, although they promised to protect – and even increase – aid spending, the Secretary of State for International Development, Andrew Mitchell, made value for money of British aid a clear priority. Among other things, that meant cuts to a cherished Labour objective: making the argument for aid to the British public.
Indeed, who could argue with cutting aid funds to a ‘Brazilian dance troupe’ in Hackney (a neighbourhood in East London)? In one of his first ministerial speeches, Mitchell made clear that these sorts of activities would no longer be tolerated and that the aid argument would be won not on explaining it to audiences at home but by improving lives abroad.
By mid-2010 the UK, not just the newly rebranded UKaid, entered an age of ‘austerity communications’. Government websites were among the first to be scrutinised. As it turned out, the UK Trade and Investment website, in what is frankly a crude measure, cost the government nearly £12/visitor, and that’s discounting staff and operating costs. Hardly value for money, by any definition.
Such costs called for a rationalisation of government websites, an edict that has trickled down through the ranks of DFID. In the most recent advice given to its large portfolio of research consortia, it was suggested that no programme should have a standalone website. Recommendations have also emerged that no money should be going to promoting large programmes as brands independent from their host organisations, and that hosting events that cost over £20,000 require cabinet-level approval.
While the value for money of these arbitrary rules is dubious at best, the push for austerity communications should be welcomed by development researchers, research communicators, knowledge intermediaries and policy entrepreneurs alike. Just as it is an incorrect assumption that less polished looking communication activities are cheaper (just ask the 2012 Olympic committee), it is equally untrue that communication has to be expensive. An unhealthy economy has emerged in the research communication field: from expensive and self-indulgent websites to exorbitant per diems for participation in events (which may soon be considered bribery in certain circumstance under new UK legislation) to paying for media placement.
My friend and former colleague, Nick Scott from ODI has spoken widely about free and low cost online tools that can help establish and bolster an online presence, so I will instead broaden the discussion in the rest of this post to how communications has the opportunity to be more effective in these tight times.
Ironically, capoiera’s existence today is a shining example of massive impact with limited resources. Capoeira emerged from slaves of African origin working the sugarcane plantations of Brazil in the 1600s. As a martial art, slaves used it for self-protection, to escape and to defend Quilombos (informal settlements of escaped slaves and others living outside the law). As capoeira was a clear threat to the Portuguese slave owners, it was outlawed, forcing capoeiristas to disguising the practice as a form of traditional dance. And perhaps at a most basic level, this clandestine approach of obfuscating traditional approaches to research communications will be necessary, but only when they are the most appropriate techniques to reach an objective.
Ultimately I hope that these new rules force us to change rather than conceal. And here, capoiera offers more lessons to inform an innovative approach to research communications.
There are several styles of capoiera, the two most popular being capoira regional (pronounced ‘hey-shu-nal) and capoiral angola. Capoeira regional is the newer, flashier side of capoeira, with rodas usually going at a quicker pace and with more jumps, spins and kicks. The more traditional angola style is comparatively slow place and low to the ground, with combatants usually keeping at least one hand touching the ground at all times. Both styles are popular, but capoeira angola is considered the more difficult. It is a reflective and strategic style and requires greater control – consider it the chess of the martial arts world. And perhaps these two styles represent the difference between research communications and marketing as it was promoted under the Labour government (capoiera regional) and the era of austerity communications (capoiera angola).
There are a few principles operating in capoeira angola: 1) conserve energy and maintain endurance; 2) use the slow pace to develop an understanding of the opponent and use that understanding to defeat her/him; 3) exploit opportunities and make every attack count. Development communications would do well to abide by these principles.
1) Conserve energy and maintain endurance: As Enrique has noted elsewhere, think tanks and research organisations that chase visibility at the cost of substantive research and influence do so at their own peril. The fact is that we are operating with finite resources and there is an opportunity cost associated with pursing any given engagement activity. To that end, we must recognise that substantive influence does not happen overnight. We need to be prepared to invest in long term strategies that focus on building relationships and trust – neither of which is founded on glossy brochures.
2) Understand the opponent: At its least, austerity communications should give us time to pause and reflect on how policy influence and research uptake actually occur in our individual contexts. Maybe getting an article into a journal with the highest impact factor isn’t going to change practice on the ground. Maybe the long research publication isn’t the best choice in Cambodia, where most business and politics is transacted verbally. Maybe the flashy website that woos donors isn’t the right option to reach researchers in the D.R. Congo where internet penetration is notoriously low.
Additionally, a good understanding of our audience allows us to extend a ‘being there’ strategy from the web to other forms of communication. Beyond thinking of where in the web world your audiences are spending their time, also think through: What publications are your target audiences already reading? What media do they already engage with? What events are they already attending? Spending effort getting into these spaces may be much more valuable than simply creating more of your own spaces and spending resources to market them. Enrique’s recent post on ‘confirmation bias’ should be a good reminder of this – people are predisposed to agree with evidence from a source they already trust.
3) Make every attack count: Value for money doesn’t necessarily mean spending less money, it means spending it wisely. Instead of a throw-everything-at-the-wall-and-see-what-sticks approach to communication (which can be particularly valuable when working in complex environments as long as there are in-built learning mechanisms), under austerity communications we will likely need to be more selective in our communications activities. So when an opportunity does arise, and we do think that it is the right intervention for the right objective, go ‘all in’ and put significant resources behind it.
In a review of DFID’s recommendation to spend 10% of funds on communication activities for certain types of programmes that Enrique and I both participated in a few years ago, we found that some programmes were taking the advice literally and cascading the 10% funding throughout all of its interventions – but some research is more communicable than other research. Austerity communications will require a greater investment in horizon scanning (and tools that facilitate this), and then taking every advantage of opportunities as and when they do arise.
*[This is the first of I hope many more contributions from practitioners and experts in the field of think tank management, communications, funding, etc. If you would like to recommend someone please contact Enrique Mendizabal on enrique@mendizabal.co.uk]
Theme: Linen by The Theme Foundry
