Force Field Analysis can be used to plan, monitor and evaluate the decisions made by think tanks involved in influencing interventions. It provides the user with intervention options, rich information about the context, and a good sense of progress. An excellent tool for ex-ante evaluations.
Posts tagged ‘learning’
Donors are increasingly concerned with studies that address the real and immediate problems faced by these countries -and in particular by the poorest among them. As a consequence a crucial component of any knowledge body is being missed: learning about others.
[Editor's note: This is the first post by Hans Gutbrod, Director of the Think Tank Initiative. I'd like to welcome him to onthinktanks.org and look forward to his ideas and reflection on think tanks. Over the next few days and weeks we'll also be sharing some of the videos from the sessions organised at the TTI's exchange mentioned in this blog post.]
Bringing together more than 100 think tank executive and research directors from more than 25 countries, the Think Tank Initiative (TTI) Exchange in Cape Town, in June 2012 probably was one of the largest gatherings of think tanks from the global South held to date. Some of the substance of the event has already been covered, in various ways, by lively Twitter commentary. Short videos from the sessions, highlighting particular topics, will be made available in the coming days and weeks.
So what continues to stand out from this event in hindsight? Here are some personal reflections, not intended as a conclusive summary, but as points of discussion.
For think tanks to make a difference, they have to attract exceptional staff. Thomas Carothers, from the Carnegie Endowment for International Peace, has made this point. It is obvious, but not trivial: think tank research needs to be both substantive and hedged, smart and safe, and, on some occasions, cautious and bold. As think tanks trade on their authority, they are particularly vulnerable when making claims. Raymond Struyk highlights this risk in the first page of his classic book on managing think tanks, describing this unsettling scenario:
A report on a high-visibility and urgent problem is sent to the Ministry of Finance with significant flaws in the statistical analysis. These flaws are discovered by an analyst from another organization after the report has been widely distributed. The think tank loses significant credibility with the government and other clients.
Not many people are good at responding thoughtfully and quickly, while getting everything right. Peer review processes can prevent errors, but given the pressure, any first draft has to be solid, and review steps need to be executed with great care.
That means not only do think tank leaders need to be remarkable (and there were a lot of exceptional people at the TTI Exchange), but they also need a remarkable team behind them. The success of a think tank depends on building a deeper team, on recruiting exceptional staff, and getting exceptional people to work together. Core funding is thus critical, as it allows think tanks to attract and retain exceptional staff. Some innovative funding arrangements notwithstanding, core funding remains a key feature for the type of local research that can improve lives, because it makes high-performing think tank teams possible.
Connecting to Conversations
Even with great teams, unnoticed loneliness at the top can be a severe challenge: since the leaders of think tanks rarely engage peer-to-peer on management questions, their loneliness is often profound. With a few exceptions, leaders mostly figure things out by themselves, maybe with a board or their personal circle. This loneliness sets them apart from people in other professions such as doctors, accountants, lawyers, or even managers in the private or public sector, who regularly discuss and enhance professional practice.
Yet the loneliness can go unnoticed, since the leader of any think tank will engage extensively with fellow researchers and policymakers, clients, maybe diplomats, scholars from abroad, journalists, interns and students. After six years at CRRC, my email address book had accumulated more than 4400 contacts. Yet except for two extended conversations that Goran Buldioski from the Think Tank Fund made possible, I not once – not a single time – in those six years, sat down with someone else who was running a research organization to exchange experiences on how we do things. And to me, the remarkable thing was that I didn’t even realize, until the TTI Exchange, that I had not discussed how to run a research organization with anyone other than my colleagues. Bringing think tank leaders together helps to overcome this loneliness, and gives them an opportunity to begin conversations and connect.
Communities of Practice
Following from this, I think it’s fair to say that there is no fully established community of practice – defined by Wenger as a “group of people who share a concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis” – for running think tanks, especially not in the South. There are some personal links, and there are well-established practices of research in relevant academic disciplines, but these are different from the practices of generating policy research with the aim of improving people’s lives – perhaps as different as physics is from engineering.
If policy research in the South is seeking to have a greater impact, it’s probably worth cultivating habits of sharing more, deepening knowledge and expertise, and interacting on an ongoing basis. These are the kind of habits that are well established in other professions. Patent lawyers, insurance actuaries, heart surgeons and project managers alike – they all set up a trade publication, ways of exchanging information, and they get together regularly, to discuss how to do things. The formalization is not all happy: there is tomfoolery in most jamborees, but if you only pick up a handful of better practices, tell a few peers what has worked for you, and identify the colleagues you will call for advice when things get sticky in the office, you serve yourself and the profession, as well as the people your profession serves.
In the case of successful policy research in the South, strengthening a nascent community of practice could probably go a long way toward making better policies stick. On our end, at TTI, we are certainly thinking about how to cultivate such a community of practice. We have quite a few ideas on how to do this, but welcome ideas and input from others.
This is also why I’m contributing these reflections here in this blog. Onthinktanks.org serves as an excellent aggregator for many of the issues that are worth debating in the community. To broaden the debate, we will be making videos from the TTI exchange available in the coming weeks, and hope they find a good audience. Check our website or follow us on Twitter.
In addition to what has already been published on this blog, do you have any other thoughts on what we can do to cultivate a community of practice for policy research and think tank management?
I rarely write about individual think tanks -mainly because I do recognise that, given think tanks’ complexities, it is very easy to get it wrong. But on this issue I am using the case of Peru mainly to help in presenting the argument with clarity. Just for the record, and hopefully to get a reaction to this idea, I think this has implications for initiatives such as AusAID’s Knowledge Sector Support Programme, the Think Tank Initiative, and the Think Tank Fund. But it is perfectly relevant for other research funders and their grantees looking for ‘impact’ to report on.
I want to address the fact that the university based think tanks is not at the top of most think tank funders’ priorities. And even when they are funded (and they are) this is done as if they were independent (is any really independent?) think tanks and with the explicit objective of enhancing their capacity to influence policy -very little else seems to matter. This is a shame in part because these think tanks are probably better able than any others to promote the development of evidence informed policy capacities -see Kirsty Newman’s post on this: Policy influence versus evidence-informed policy.
But it is also the case that we are missing out on an important function that these think tanks can fulfil particularly well: to train researchers for other think tanks.
It just occurred to me that while I was studying economics at the Universidad del Pacifico in Peru I, as pretty much all my fellow students, spent some time working at the university’s think tank, CIUP. Some spent just a few months involved in one or two projects while others took up positions as long term interns and research assistants. I stayed at CIUP after graduating as did some of my peers. Others, who had worked at CIUP before graduation went on to work for other local think tanks such as GRADE, MacroConsult, Apoyo, etc. Of all those who stayed at CIUP after graduation not all became full-time researchers there. Some stayed but many left to join other these other think tanks -as well as public and private organisations in Peru and abroad. Some have gone back after years working in other centres and sectors.
The point is that CIUP’s link to the university and its education responsibilities -which can be seen by some funders as a constraint on their independence- in effect led to it ‘subsidising’ the development of the capacity of its competitors’ (or comparators’) staff (which they would have otherwise had to develop themselves -although I must point out that GRADE, Macro and other centres do develop the capacity of their young researchers but I would argue that there is at least an initial investment made by the university based centres that provides a good base to build on). I have come across the more extreme situation in several countries where there are no university think tanks: young researchers telling me that they joined a think tank after graduation because they had not learned how to do research at university; and think tank directors have confided that they have to build the capacity of their young recruits whose research skills are close to nil.
Of course in Peru there are other such university think tanks and so CIUP shares this role. But in many countries there are not that many good universities with strong and respected think tanks where students can first learn to do research and practice the skills they will need later on. And in other cases there simply are no university think tanks at all (or those that exist are not really fulfilling this role) and so graduates and think tanks are left without the skills they need. Additionally, without the opportunity to practice policy research, many graduates are simply unaware that think tanks can be an exiting and interesting career path.
When I say that without CIUP-like organisations young graduates would not have the opportunity to learn how to do policy research and analysis, I am speaking from experience: this is where I learned about policy analysis and had the chance to practice it and get the taste for it. My peers and I may have learnt lots of information (theories, models, facts) in class but would not have had the chance to apply it to real and practical questions had it not been for the think tank and the opportunities the university gave us to work with the researchers at CIUP on real policy research and analysis.
By supporting university based think tanks, therefore, funders can create positive externalities that can have critical effects across the entire knowledge sector -as well as on other public and private policy players. This support can come in the form of:
- Institutional funds to encourage more students to participate in policy research and analysis initiatives with senior researchers. These projects can offer students the opportunity to experience the various aspects of think tank work: data collection, analysis, project management, communications, policy engagement, fundraising, etc.
- Travel grants for top students to spend a few weeks or their summer holidays in think tanks in the US, Europe or elsewhere (and my usual disclaimer applies: NOT international development think tanks).
- Seminar series on think tanks and the roles they can fulfil as a way of encouraging them to get involved.
Of course, where no university think tanks are present or where their capacity is quite low a more concerted effort should be made to either rebuild them or establish closer links between universities and non-university think tanks. In Zambia, for example, the student chapters of the Economic Association of Zambia facilitates internships in think tanks for their members. But if funders provided funds, via the think tanks, to support this the effect this could have on the future capacity of think tanks would be enhanced.
In Serbia, too, the Belgrade Centre for Security Studies has a fantastic internship programme informally linked to the Faculty of Political Science. They don’t just use their interns for data collection or analysis (or coffee-making) but have also designed a lecture series that includes sessions on think tanks, research methods, and communications. Most of their interns then go on to work for other organisations in Serbia and take what they learned with them.
This kind of experience is invaluable for encouraging a greater number of highly competent young graduates to join think tanks and for giving them the skills that they and their future employers will need.
A couple of weeks ago, as part of the Think Tank Initiative Exchange, I participated in a panel on successful policy engagement alongside Nancy Birdsall of the Center for Global Development and Frannie Leautier of the African Capacity Building Foundation to discuss approaches to policy influence.
I shared a research project we are carrying out in Grupo FARO. Our goal is to look into what some think tanks participating in the initiative consider to be policy influence and how they go about achieving it. We also hope it will contribute to a growing literature on the political economy of research uptake that is slowly, but surely, being adopted by researchers from the developing world.
We based our analysis on twelve stories of policy influence written and kindly shared with us by think tanks from Africa, Asia and Latin America. The stories involve a wide variety of topics, from education reform and poverty reduction to affirmative action in the private sector and shaping electoral processes.
To analyse this great diversity of stories we used a framework that considers, on the one hand, strategies that think tanks carry out to influence policy and, on the other hand, the contexts in which they work.
For the strategies, we examined three central aspects involved in participating in policy processes:
- The stage(s) of the policy cycle in which think tanks participated;
- The different types of evidence they employed. We based our analysis on previous work by Reimers and McGinn to identify whether the evidence came from more academic or more action research; and
- The roles think tanks played during those given episodes, whether advising decision-makers, facilitating dialogue among stakeholders, or advocating for a certain proposal or position.
However, strategies do not come out of thin air; they are (supposedly) calculated responses to complex contexts. To demonstrate this relationship we classified the types of problems think tanks face. We borrowed Hoppe’s framework, which looks into two dimensions of a policy problem:
- The level of certainty regarding relevant knowledge for the policy process; and
- The level of agreement on relevant norms and values.
The intersection of these two dimensions yields four general types of policy problems that think tanks may encounter. Let me share an example: Let’s say a think tank is working on health care systems. If the think tank faces a setting where there is agreement that a country must establish universal health care coverage as a right, and has decided on a model to provide coverage to all, let’s say a completely public model, then it is a structured problem. In such cases a think tank can help in putting forth specific proposals on how to implement that decision or monitoring the programs to see their actual impact.
If there is no agreement on whether health care is a right but all parties agree that there should be private provision of services, it would be a partially structured problem with disagreement on values. If stakeholders agree on universal coverage, but not on whether the best solutions should include public or private providers, then it would be a problem characterized by a lack of certainty on knowledge.
Timing is perfect to use this case. What type of problem is the health care reform in the United States? This discussion could sparkle an interesting debate, but I will leave that to be your homework.
There are also unstructured problems. For example, trying to initiate a peace process after a civil war is likely to be a very unstructured problem, as there is little to no knowledge on what happened during the war or the consequences it had and will have on the country, and little to no agreement on how to move forward.
Redefining policy influence
Influence on what? What is it that think tanks hope to achieve?
Although when discussing policy influence we usually focus on outputs, such as concrete change of laws or policies, there is a growing need to see beyond this limited vision of influence. In contexts of uncertainty, disagreement or mistrust, it is unlikely that resulting policies will be well-designed and effectively and sustainably implemented. In such circumstances, think tanks, instead of trying to achieve policy change, should aim to establish a minimum of agreement among stakeholders to begin along the road for positive, viable policy change. In this sense, changes in the public agenda, values, power structures, and institutions should also be included in the definition of policy influence.
Second, how do think tanks carry out this mission of change?
Think tanks traditionally take a tactical approach, seeing influence as a set of chores carried out to ensure that policymakers hear and adopt ‘my’ solution. Unfortunately, this leads us to prioritise the marketing and communication of ideas over efforts to bring stakeholders together to create real and collective change. After reviewing the cases, we are more convinced than ever that true influence is best described as exercising leadership.
Leading is not pushing ‘my’ idea forward, or the idea of a political party, or donor, or other external stakeholder; leading means mobilising people to collaborate and work towards a solution. It is a collective effort in which think tanks play a very active rather than passive role. The think tanks’ cases studied exert important influence by, for example, posing tough questions or leading hard conversations.
Think tanks are political actors
Another significant conclusion we were able to draw from this analysis is that think tanks are political actors, far more than they consider themselves to be. The analysis shows that think tanks, even in the most structured scenarios, often go beyond the advisory role, bringing new voices into the discussion and choosing not to advise behind closed doors.
In less structured contexts, where evidence and knowledge play a secondary role, think tanks have become what Robert Heifetz calls “leaders without authority” by bringing to the surface concealed issues such as discrimination or generating dialogue among political parties at moments of high political polarisation.
Think tanks navigate in complex and changing contexts. These realities at times require them to advocate a technical solution and other times to promote a learning process to change perspectives and encourage new practices. Coming back to the title of this post and the document, it is about leading, it is about learning jointly with society; not an easy task, but that some are willing to pursue.
This report is just an initial draft that we are continuing to develop. We look forward to hearing your comments on the framework and analysis as we finalise our study.