Skip to content

Posts tagged ‘learning’

Think tanks and communication workshop: a learner’s perspective

In this post, Palash Sanyal from the Centre for Policy Development in Bangladesh reflect on the communications workshop organised in Dhaka last month. He was one of the participants, representing CPD's communications team. In this post he identifies key lessons and highlight some insights from the sessions and the discussion.

Read more

How did leading US Think Tanks fare in 2012? Analysis by numbers

Hans Gutbrod analyses how 20 leading US think tanks have developed over 2012. Seven of them are doing very well, while four of them are not exactly comfortable, at least not in financial terms. Analysis and detailed spreadsheet available.

Read more

The onthinktanks interview: Dr Asep Suryahadi

In this interview, SMERU's Director, Dr Asep Suryahadi, describes his motivations for joining the think tank, the centre's history, and its current and future challenges. Pak Asep explains that as a think tank in Indonesia SMERU must balance a number of sometimes competing expectations from multiple stakeholders.

Read more

A cheaper and more useful alternative to ROI for a think tank: Force Field Analysis

Force Field Analysis can be used to plan, monitor and evaluate the decisions made by think tanks involved in influencing interventions. It provides the user with intervention options, rich information about the context, and a good sense of progress. An excellent tool for ex-ante evaluations.

Read more

Advice to Think Tank Startup: do not do it alone

Hans Gutbrod outlines some ideas that should be considered when thinking of setting up a new think tank. He argues that planning and learning from others are critical for success. And do not forget that management will matter as much as the quality of your research.

Read more

How to overcome the knowledge gap: study the rich

Donors are increasingly concerned with studies that address the real and immediate problems faced by these countries -and in particular by the poorest among them. As a consequence a crucial component of any knowledge body is being missed: learning about others.

Read more

Reflections on Bringing Think Tanks Together: a Community of Practice?

[Editor's note: This is the first post by Hans Gutbrod, Director of the Think Tank Initiative. I'd like to welcome him to onthinktanks.org and look forward to his ideas and reflection on think tanks. Over the next few days and weeks we'll also be sharing some of the videos from the sessions organised at the TTI's exchange mentioned in this blog post.]

Bringing together more than 100 think tank executive and research directors from more than 25 countries, the Think Tank Initiative (TTI) Exchange in Cape Town, in June 2012 probably was one of the largest gatherings of think tanks from the global South held to date. Some of the substance of the event has already been covered, in various ways, by lively Twitter commentary. Short videos from the sessions, highlighting particular topics, will be made available in the coming days and weeks.

So what continues to stand out from this event in hindsight? Here are some personal reflections, not intended as a conclusive summary, but as points of discussion.

Stability Matters

For think tanks to make a difference, they have to attract exceptional staff. Thomas Carothers, from the Carnegie Endowment for International Peace, has made this point. It is obvious, but not trivial: think tank research needs to be both substantive and hedged, smart and safe, and, on some occasions, cautious and bold. As think tanks trade on their authority, they are particularly vulnerable when making claims. Raymond Struyk highlights this risk in the first page of his classic book on managing think tanks, describing this unsettling scenario:

A report on a high-visibility and urgent problem is sent to the Ministry of Finance with significant flaws in the statistical analysis. These flaws are discovered by an analyst from another organization after the report has been widely distributed. The think tank loses significant credibility with the government and other clients.

Not many people are good at responding thoughtfully and quickly, while getting everything right. Peer review processes can prevent errors, but given the pressure, any first draft has to be solid, and review steps need to be executed with great care.

That means not only do think tank leaders need to be remarkable (and there were a lot of exceptional people at the TTI Exchange), but they also need a remarkable team behind them. The success of a think tank depends on building a deeper team, on recruiting exceptional staff, and getting exceptional people to work together. Core funding is thus critical, as it allows think tanks to attract and retain exceptional staff. Some innovative funding arrangements notwithstanding, core funding remains a key feature for the type of local research that can improve lives, because it makes high-performing think tank teams possible.

Connecting to Conversations

Even with great teams, unnoticed loneliness at the top can be a severe challenge: since the leaders of think tanks rarely engage peer-to-peer on management questions, their loneliness is often profound. With a few exceptions, leaders mostly figure things out by themselves, maybe with a board or their personal circle. This loneliness sets them apart from people in other professions such as doctors, accountants, lawyers, or even managers in the private or public sector, who regularly discuss and enhance professional practice.

Yet the loneliness can go unnoticed, since the leader of any think tank will engage extensively with fellow researchers and policymakers, clients, maybe diplomats, scholars from abroad, journalists, interns and students. After six years at CRRC, my email address book had accumulated more than 4400 contacts. Yet except for two extended conversations that Goran Buldioski from the Think Tank Fund made possible, I not once – not a single time – in those six years, sat down with someone else who was running a research organization to exchange experiences on how we do things. And to me, the remarkable thing was that I didn’t even realize, until the TTI Exchange, that I had not discussed how to run a research organization with anyone other than my colleagues. Bringing think tank leaders together helps to overcome this loneliness, and gives them an opportunity to begin conversations and connect.

Communities of Practice

Following from this, I think it’s fair to say that there is no fully established community of practice – defined by Wenger as a “group of people who share a concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis” – for running think tanks, especially not in the South. There are some personal links, and there are well-established practices of research in relevant academic disciplines, but these are different from the practices of generating policy research with the aim of improving people’s lives – perhaps as different as physics is from engineering.

If policy research in the South is seeking to have a greater impact, it’s probably worth cultivating habits of sharing more, deepening knowledge and expertise, and interacting on an ongoing basis. These are the kind of habits that are well established in other professions. Patent lawyers, insurance actuaries, heart surgeons and project managers alike – they all set up a trade publication, ways of exchanging information, and they get together regularly, to discuss how to do things. The formalization is not all happy: there is tomfoolery in most jamborees, but if you only pick up a handful of better practices, tell a few peers what has worked for you, and identify the colleagues you will call for advice when things get sticky in the office, you serve yourself and the profession, as well as the people your profession serves.

In the case of successful policy research in the South, strengthening a nascent community of practice could probably go a long way toward making better policies stick. On our end, at TTI, we are certainly thinking about how to cultivate such a community of practice. We have quite a few ideas on how to do this, but welcome ideas and input from others.

This is also why I’m contributing these reflections here in this blog. Onthinktanks.org serves as an excellent aggregator for many of the issues that are worth debating in the community. To broaden the debate, we will be making videos from the TTI exchange available in the coming weeks, and hope they find a good audience. Check our website or follow us on Twitter.

In addition to what has already been published on this blog, do you have any other thoughts on what we can do to cultivate a community of practice for policy research and think tank management?

A case for funding university-based/linked think tanks: more and better staff for others

I rarely write about individual think tanks -mainly because I do recognise that, given think tanks’ complexities, it is very easy to get it wrong. But on this issue I am using the case of Peru mainly to help in presenting the argument with clarity. Just for the record, and hopefully to get a reaction to this idea, I think this has implications for initiatives such as AusAID’s Knowledge Sector Support Programme, the Think Tank Initiative, and the Think Tank Fund. But it is perfectly relevant for other research funders and their grantees looking for ‘impact’ to report on.

I want to address the fact that the university based think tanks is not at the top of most think tank funders’ priorities. And even when they are funded (and they are) this is done as if they were independent (is any really independent?) think tanks and with the explicit objective of enhancing their capacity to influence policy -very little else seems to matter. This is a shame in part because these think tanks are probably better able than any others to promote the development of evidence informed policy capacities -see Kirsty Newman’s post on this: Policy influence versus evidence-informed policy.

But it is also the case that we are missing out on an important function that these think tanks can fulfil particularly well: to train researchers for other think tanks.

It just occurred to me that while I was studying economics at the Universidad del Pacifico in Peru I, as pretty much all my fellow students, spent some time working at the university’s think tank, CIUP. Some spent just a few months involved in one or two projects while others took up positions as long term interns and research assistants. I stayed at CIUP after graduating as did some of my peers. Others, who had worked at CIUP before graduation went on to work for other local think tanks such as GRADE, MacroConsult, Apoyo, etc. Of all those who stayed at CIUP after graduation not all became full-time researchers there. Some stayed but many left to join other these other think tanks -as well as public and private organisations in Peru and abroad. Some have gone back after years working in other centres and sectors.

The point is that CIUP’s link to the university and its education responsibilities -which can be seen by some funders as a constraint on their independence- in effect led to it ‘subsidising’ the development of the capacity of its competitors’ (or comparators’) staff (which they would have otherwise had to develop themselves -although I must point out that GRADE, Macro and other centres do develop the capacity of their young researchers but I would argue that there is at least an initial investment made by the university based centres that provides a good base to build on). I have come across the more extreme situation in several countries where there are no university think tanks: young researchers telling me that they joined a think tank after graduation because they had not learned how to do research at university; and think tank directors have confided that they have to build the capacity of their young recruits whose research skills are close to nil.

Of course in Peru there are other such university think tanks and so CIUP shares this role. But in many countries there are not that many good universities with strong and respected think tanks where students can first learn to do research and practice the skills they will need later on. And in other cases there simply are no university think tanks at all (or those that exist are not really fulfilling this role) and so graduates and think tanks are left without the skills they need. Additionally, without the opportunity to practice policy research, many graduates are simply unaware that think tanks can be an exiting and interesting career path.

When I say that without CIUP-like organisations young graduates would not have the opportunity to learn how to do policy research and analysis, I am speaking from experience: this is where I learned about policy analysis and had the chance to practice it and get the taste for it. My peers and I may have learnt lots of information (theories, models, facts) in class but would not have had the chance to apply it to real and practical questions had it not been for the think tank and the opportunities the university gave us to work with the researchers at CIUP on real policy research and analysis.

By supporting university based think tanks, therefore, funders can create positive externalities that can have critical effects across the entire knowledge sector -as well as on other public and private policy players. This support can come in the form of:

  • Institutional funds to encourage more students to participate in policy research and analysis initiatives with senior researchers. These projects can offer students the opportunity to experience the various aspects of think tank work: data collection, analysis, project management, communications, policy engagement, fundraising, etc.
  • Travel grants for top students to spend a few weeks or their summer holidays in think tanks in the US, Europe or elsewhere (and my usual disclaimer applies: NOT international development think tanks).
  • Seminar series on think tanks and the roles they can fulfil as a way of encouraging them to get involved.

Of course, where no university think tanks are present or where their capacity is quite low a more concerted effort should be made to either rebuild them or establish closer links between universities and non-university think tanks. In Zambia, for example, the student chapters of the Economic Association of Zambia facilitates internships in think tanks for their members. But if funders provided funds, via the think tanks, to support this the effect this could have on the future capacity of think tanks would be enhanced.

In Serbia, too, the Belgrade Centre for Security Studies has a fantastic internship programme informally linked to the Faculty of Political Science. They don’t just use their interns for data collection or analysis (or coffee-making) but have also designed a lecture series that includes sessions on think tanks, research methods, and communications. Most of their interns then go on to work for other organisations in Serbia and take what they learned with them.

This kind of experience is invaluable for encouraging a greater number of highly competent young graduates to join think tanks and for giving them the skills that they and their future employers will need.

Influencing policy: it’s about learning and leading for change

[Editor’s note: Written by Andrea Ordoñez, Grupo FARO’s Director of Research; follow her @AOrdonez]

A couple of weeks ago, as part of the Think Tank Initiative Exchange, I participated in a panel on successful policy engagement alongside Nancy Birdsall of the Center for Global Development and Frannie Leautier of the African Capacity Building Foundation to discuss approaches to policy influence.

I shared a research project we are carrying out in Grupo FARO. Our goal is to look into what some think tanks participating in the initiative consider to be policy influence and how they go about achieving it. We also hope it will contribute to a growing literature on the political economy of research uptake that is slowly, but surely, being adopted by researchers from the developing world.

We based our analysis on twelve stories of policy influence written and kindly shared with us by think tanks from Africa, Asia and Latin America. The stories involve a wide variety of topics, from education reform and poverty reduction to affirmative action in the private sector and shaping electoral processes.

To analyse this great diversity of stories we used a framework that considers, on the one hand, strategies that think tanks carry out to influence policy and, on the other hand, the contexts in which they work.

For the strategies, we examined three central aspects involved in participating in policy processes:

  • The stage(s) of the policy cycle in which think tanks participated;
  • The different types of evidence they employed. We based our analysis on previous work by Reimers and McGinn to identify whether the evidence came from more academic or more action research; and
  • The roles think tanks played during those given episodes, whether advising decision-makers, facilitating dialogue among stakeholders, or advocating for a certain proposal or position.

However, strategies do not come out of thin air; they are (supposedly) calculated responses to complex contexts. To demonstrate this relationship we classified the types of problems think tanks face. We borrowed Hoppe’s framework, which looks into two dimensions of a policy problem:

  • The level of certainty regarding relevant knowledge for the policy process; and
  • The level of agreement on relevant norms and values.

The intersection of these two dimensions yields four general types of policy problems that think tanks may encounter. Let me share an example: Let’s say a think tank is working on health care systems. If the think tank faces a setting where there is agreement that a country must establish universal health care coverage as a right, and has decided on a model to provide coverage to all, let’s say a completely public model, then it is a structured problem. In such cases a think tank can help in putting forth specific proposals on how to implement that decision or monitoring the programs to see their actual impact.

If there is no agreement on whether health care is a right but all parties agree that there should be private provision of services, it would be a partially structured problem with disagreement on values. If stakeholders agree on universal coverage, but not on whether the best solutions should include public or private providers, then it would be a problem characterized by a lack of certainty on knowledge.

Timing is perfect to use this case. What type of problem is the health care reform in the United States? This discussion could sparkle an interesting debate, but I will leave that to be your homework.

There are also unstructured problems. For example, trying to initiate a peace process after a civil war is likely to be a very unstructured problem, as there is little to no knowledge on what happened during the war or the consequences it had and will have on the country, and little to no agreement on how to move forward.

You can read a more complete analysis in the first draft of our paper and my presentation at the Think Tank exchange at the bottom of this post, but let me first summarise some of our conclusions.

Redefining policy influence

Influence on what?  What is it that think tanks hope to achieve?

Although when discussing policy influence we usually focus on outputs, such as concrete change of laws or policies, there is a growing need to see beyond this limited vision of influence.  In contexts of uncertainty, disagreement or mistrust, it is unlikely that resulting policies will be well-designed and effectively and sustainably implemented. In such circumstances, think tanks, instead of trying to achieve policy change, should aim to establish a minimum of agreement among stakeholders to begin along the road for positive, viable policy change. In this sense, changes in the public agenda, values, power structures, and institutions should also be included in the definition of policy influence.

Second, how do think tanks carry out this mission of change? 

Think tanks traditionally take a tactical approach, seeing influence as a set of chores carried out to ensure that policymakers hear and adopt ‘my’ solution. Unfortunately, this leads us to prioritise the marketing and communication of ideas over efforts to bring stakeholders together to create real and collective change. After reviewing the cases, we are more convinced than ever that true influence is best described as exercising leadership.

Leading is not pushing ‘my’ idea forward, or the idea of a political party, or donor, or other external stakeholder; leading means mobilising people to collaborate and work towards a solution. It is a collective effort in which think tanks play a very active rather than passive role. The think tanks’ cases studied exert important influence by, for example, posing tough questions or leading hard conversations.

Think tanks are political actors

Another significant conclusion we were able to draw from this analysis is that think tanks are political actors, far more than they consider themselves to be. The analysis shows that think tanks, even in the most structured scenarios, often go beyond the advisory role, bringing new voices into the discussion and choosing not to advise behind closed doors.

In less structured contexts, where evidence and knowledge play a secondary role, think tanks have become what Robert Heifetz calls “leaders without authority” by bringing to the surface concealed issues such as discrimination or generating dialogue among political parties at moments of high political polarisation.

Think tanks navigate in complex and changing contexts. These realities at times require them to advocate a technical solution and other times to promote a learning process to change perspectives and encourage new practices. Coming back to the title of this post and the document, it is about leading, it is about learning jointly with society; not an easy task, but that some are willing to pursue.

This report  is just an initial draft that we are continuing to develop. We look forward to hearing your comments on the framework and analysis as we finalise our study.

The paper:

View this document on Scribd

The presentation:

View this document on Scribd

Understanding the demand for World Bank research within the Bank

A year or so ago, DFID asked us (in ODI) to assess whether its research and evaluations were being used by DFID staff. Harry Jones and I developed an approach that focused not on the studies themselves (the supply) but on the way that DFID staff made choices and the roles or functions that evidence played in those. Evidence, we assumed, could come from different sources, one of which could be DFID research and evaluations. We did not, however, want to bias our study by focusing on them. And we felt that to say anything useful about the use that these had within DFID we had to do it in a wider context of all other inputs to decision making.

A recent study from the World Bank takes a different approach. Martin Ravallion’s study: Knowledgeable bankers ? the demand for research in World Bank operations, focuses on the demand for World Bank research. It does, however, share some of our conclusions:

The methods used affect demand:

Today’s research priorities may well be poorly matched with the issues faced by practitioners in these sectors. For example, the current emphasis on randomized trials in development economics has arguably distorted knowledge even further away from the hard infrastructure sectors where these tools have less applicability (Ravallion, 2009). Making the supply of research more relevant to the needs of development practitioners would undoubtedly help.

Absorptive capacity is crucial:

The differences across units in the demand for the Bank’s research are correlated with the incidence of PhDs and economists, suggesting that internal research capacity in operational units helps create absorptive capacity for knowledge in those units.

Researchers need to make an effort, too, however:

The slope of the relationship between perceived value and familiarity with research is positive but significantly less than unity, suggesting frictions in how the incentive for learning translates into knowledge. The responsiveness of researchers and the timeliness and accessibility of their outputs are clearly important to how much learning incentives lead to useful knowledge.

This study also finds two models that explain how research affects decisions:

In the first, they have a demand for knowledge that does not stem from its direct bearing on their work. Much development research is a public good. Practitioners might read research findings to better understand the world in which they work, even when that understanding is essentially irrelevant to the specifics of that work.

Alternatively, in the second model, research has a direct value in the work of practitioners—such as by informing project choices at the entry stage and assessing impacts later on—and research findings are sufficiently relevant and accessible to assure that practitioners become well informed.

We found a few more options, depending on the type of decisions that staff had to make:

  • In some cases, evidence generation was incorporated into the policy cycle
  • In others, evidence was used to make small incremental changes and corrections to ongoing policies and programmes
  • In other cases, evidence had to catch up to events
  • And in others, more often than not, it was used to make sense of political demands

We concluded that DFId was better at using the evidence and learning from it. These are two different things. It was also:

much better at using research and evaluation findings during or as part a project cycle than in more complex and emergent decision making processes.

In other words, it was better at working with a consultant than with an academic (or so I liked to think about it) -and this resonates with Ravallion’s finding of the importance of staff capacity.

This, in turn, points towards a possible mismatch between the ideals and realities of lesson- learning in DFID. For example, research is largely done outside the organisation by increasingly larger consortia with clear incentives to communicate to audiences other than DFID. The incorporation of their findings into DFID policymaking processes then depends on these programme‟s communications capacities, intermediaries (both technology based and knowledge brokers), and DFID staff themselves –who, according to the study are under increasing time pressures that reduce incentives towards engaging with research and evaluation processes and the analysis of their evidence and findings.

To us this meant that DFID was pushing research away from itself making it a foreign concept. Its efforts to bring it back by hiring Knowledge Brokers (PhDs) to mediate did not seem to fit with what emerged as the more effective model:

The system emerging from this is one where intermediaries between research and evaluation and policy and practice play a significant role.

In other words, learning in DFID (of the kind that promotes the incorporation of analysis into decision making and the development of a learning organisation) is more akin to a system with fewer intermediaries and more direct relations between users and producers of knowledge.

I liked this conclusion because it fits nicely with a belief that we need to pay more attention to people in this business of international development.

Follow

Get every new post delivered to your Inbox.

Join 5,161 other followers