What do policymakers want?

21 May 2014

[Editor’s note: this post has been edited to include a recent study from Australia.]

Three recent reports on the views of policymakers have been making the rounds in influential blogs and online spaces.

From the United States

The first is a report on what White House policymakers in the US want from researchers that first came to my attention via Duncan Green’s blog. The paper: What Do Policymakers Want From Us?, by Paul Avey and Michael Desch, found, interestingly that:

policymakers do regularly follow academic social science research and scholarship on national security affairs hoping to draw upon its substantive expertise. But put into question the direct relevance of the most scientific approaches to international relations.

And, to the surprise of many, their fundings challenge:

the “trickle down” theory that basic social science research eventually influences policymakers.

The authors try to answer a question that many think tanks have asked themselves (but rarely actually ask policymakers);

What, precisely, do the most senior national security policymakers want from international relations scholars?

They arrive at a number of interesting findings. Some are well known by now but other, I think, are worth highlighting:

Theory is not a bad word: I remember, before “Theory of Change” became all the rage, being asked by DFID to remove the word “theory” (in relation to brief mention to theories of political change) from a 15 page How to Note on how to plan and monitor policy influencing strategies (I was also asked to cut the note as it was too long).  DFID and other agencies have employed lots of ‘knowledge brokers’ to digest theory for policymakers and protect them from the hard world of academic research. The problem, however, aren’t the theories or the complex ideas that researchers often deal with. The problem is jargon!

While policymakers do use theory they are skeptical of much of academic social science which they see as jargon-ridden and overly focused on technique, at the expense of substantive findings.

Smart policymakers are not researchers best friends (at least not for those who like to claim impact): A few months ago, Philipe Martin wrote about an interesting situation among the think tanks of the TTI. Apparently, the policymakers that knew the think tanks better also distrusted them more. In other words, the more one knows about something the more one is skeptical or critical about those who claim to know about it, too.

Not surprisingly, rank in government is often negatively associated with tolerance for sophisticated methods; more striking, in our view, is that level of education also has that same negative correlation, indicating that it is those most familiar with those theories and techniques who are most skeptical of them.

It is about people -and ideas, but mostly people: In the world of intelligence services the most important contributions to policy/decisions happens not via ‘briefs’ or ‘documents’ but conversations. From that post:

At the National Intelligence Council, I came to think that, for all the technology, strategic analysis was best done in person. I came to think that our real product weren’t those papers, the NIEs (National Intelligence Estimates). Rather they were the NIOs, the National Intelligence Officers –the experts, not the papers.

The paper on the White House agrees:

Finally, policymakers believe that the most important contributions scholars can make are not as direct policy participants or trainers of aspiring government employees, but rather as informal advisors or creators of new knowledge. However, severe time constraints limit their ability to use such scholarship in any but its’ very briefest presentation.

From Britain

The second paper comes from Britain and deal with the role that the media, especially printed newspapers, play in reaching and influencing policymakers. Earlier this year I organised an event in Lima in which we asked a journalist, a politician, and a researcher how they preferred to access information. The political panelist was very clear in the role that the media played in shaping his agenda -even his agenda for the day.

This paper by Colin Talbot and Carole Talbot at Manchester University: Sir Humphrey and the professors: What does Whitehall want from academics? Makes this very same point.

The findings include the following:

How do they access expertise?

Research reports and newspapers and magazines top the ways in which policymakers like to access expertise.

Most interestingly, though, policymakers where asked about what disciplines they found more useful. Not surprisingly, the findings show that more applied or applicable disciplines came on top. Many think tanks tend to organise themselves along disciplines (economics, law, social sciences, etc.) and therefore fail to capture policymakers’ own preferences.

The report offers an interesting approach to explore this issue in other contexts.

From Australia

The third paper is one recently published via the LSE Impact of Social Sciences blogAustralian survey indicates policy-makers still have major reservations about assigning priority to academic research by  Michele FergusonBrian HeadAdrian Cherney and Paul Boreham.  In fact, the Australian report is a whole project that includes several papers: Are Australian policymakers interested in social science research?

Among their publications is included a survey:

the responses from the survey indicated that academic research, while valued and considered relevant, is not being used by the majority of staff in policy decision-making. Policy staff at both federal and state levels indicated major reservations about assigning high priority to academic research in their policy development work and only 16 percent of respondents reported that university research results have regularly influenced changes in policies developed by their unit.

But most interesting is the difference in the perception of influence between policymakers and researchers. This table speaks for it self:

Academics certainly think quite highly of themselves. My own experience working with think tanks in developing countries would tend to confirm this. The thousands of case studies that research funders like to commission (and think tanks like to produce) overestimate the role that research plays on policymaking.

The Australian researchers provide an excellent list of key results that are worth publishing:

Policy-makers

94% – Indicated that the internet is an important means of obtaining research information82% – Reported difficulties in accessing full-text versions of academic articles and reports

81% – Agreed that the policy-making process is driven by budgetary considerations

71% – Perceived that urgent day-to-day issues take precedence over “long-term” thinking

71% – When using academic research, consider it a high priority that the research findings are unbiased

61% – Reported that in the last 12 months they had written a policy-related document that drew on academic research

Academic researchers

84– Agreed that academic reward systems do not adequately recognise dissemination of work to non-academic end-users81– Indicated that the amount of time needed to invest in coordinating the work between different partners in research collaborations is potentially problematic

80% – Find there are different research orientations between academics and external partners

77% – Agreed that research partnerships have provided them with opportunities for their research to have an impact on policy and practice

73– Agreed that the academic requirement to publish in peer-reviewed journals inhibits a focus on policy and practitioner audiences

All papers are worth reading in detail. There have been previous efforts in international development think tanks to undertake similar surveys. Unfortunately, they have never been large enough nor sufficiently robust to yield much more than anecdotal evidence.

The Australian study in particular converse a number of angles and issues. It also provides excellent recommendations (published in the LSE blog).

It is quite telling that they are:

  1. Common sense things: 1) to be influential, academic research must be accesible, 2) all actors must take the time and effort to build and maintain relationships, 3) choose the best channels and tools and be consistent, 4) create opportunities to bring policymakers and academics together.
  2. They should be part of the day to day job of research centres by now. I am amazed at funders still hiring consultants and ‘encouraging’ their grantees to make their research more accesible, or build relationships with the media or with policymakers. These should have been incorporated into think tanks’ and even more academically leaning research centres’ core mandates.