Skip to content

Posts tagged ‘DFID’

A peer-to-peer approach to supporting think tanks in Zambia

In this post, Guy Lodge and Will Paxton write about a new initiative to support think tanks in Zambia: the Zambian Economic Advocacy Programme (ZEAP). This initiative, comparable to the much larger Think Tank Initiative and Knowledge Sector Initiative (in Indonesia), aims to improve the quality of economic policy debate in the country by relying on the role of experienced peers to support a group of think tanks. It offers a promising alternative to much more expensive models.

Read more

Zambia is the place to be for new ideas on think tanks

Zambia has emerged as a fertile ground for new think tanks. Innovative donors, an enlightened political leadership, and committed individuals offer interesting lessons.

Read more

Hans Gutbrod: new head of the Think Tank Initiative

Almost half a year ago I wrote some unsolicited recommendations for the future head of the Think Tank Initiative. This week I’ve had the chance to share some of them with the incoming Program Manager: Hans Gutbrod. Over the last few days here in Cape Town he has also expressed his own views on twitter and in conversations with think tank directors and researchers at the event. Today, after a long day at the TTIexchange in South Africa I sat down with him for a quick chat about his motivations and expectations.

Why did you apply for the job? 

I am passionate about making research work, but making it work in a way and in places where it’s even more important that research gives citizens and their countries a voice. And this is what the Think Tank Initiative is about.  Think tanks, at their best, anchor expertise in a country and allow research to make a difference in the long term. But nothing lasts without the right individuals and institutions (that the TTI is supporting).

What is your think tank experience?

I worked for almost 6 years with a research organisation focused on the delivery of empirical research: a Fact Tank. (Hans was Regional Director at Caucasus Research Resource Center.) In a context where even the basic numbers were contested (unemployment levels, levels of destitution, etc.) there was a need for this approach. A fact tank can be a think tank. This strategy worked well for a highly polarised context and our centre did this work rather well and gave citizens (who were finally being counted) a voice via our research.

Can think tank make a difference in unfriendly contexts?

Even in difficult contexts when the constraints faced by think tanks in different situations are sizeable, the opportunities are there too. Small policy fixes can make huge differences to the majority of citizens. But to be effective under these circumstances, think tanks need to be highly responsive and in a way contribute to create their own opportunities for success. Once this happens, success is cumulative, melting problems away once you get going.

Do you have a definition of a think tank?

Ask me in 3 months. It is important to make sure that we have one that in inclusive whilst meaningful. But if I was to outline some principles: it would have to be interested broadly in the public good, be non-for-profit, and it should apply judgement to generate the appropriate type and level of knowledge that can feed into policy and practice.

What do you mean by appropriate?

That it responds to the context. That it addresses the problem: is it ignorance or disfunctionality? These require different types of knowledge. In doing so they need to behave like entrepreneurs. When people try to generate knowledge that adds value they take certain risks; risks similar to those taken by entrepreneurs. They must also develop and sustain influencing processes; similar to the enterprises taken on by entrepreneurs.

But back to the definition. Think tanks, at least in the idealised version, start with a concern about an issue and research is at the core of their response to it. In other words, research is central to their credibility and identity.

You are right. When deciding what is and what is not a think tank we can ask ourselves what we think of when we think of an organisation. Amnesty International, for instance, may do research but that is not the first thing that comes to mind when I think of it. 

I know you’ve had only a few days to get to know the grantees and the initiative but  is there anything that you have found particularly appealing? 

Difficult to pick… Maybe that in 10 to 15 years it is possible that the idea of think tanks in the ‘south’ will be seen as normal and central as they are in more developed economies. This is now possible because of the internet. From my own experience I know how central it was for us to succeed in a difficult context. Even in hard circumstances we had access to information about our role models half way across the world. They were only a few seconds away; closer than the bookshelf.

That the discussions we have had today are now available to people on the internet in a radical development.

And what about concerns? 

Leadership is critical to success. But the kind of leadership that is necessary in research in one with an element of productive paranoia. Leadership is an authority claim and for this claim to be credible it needs to be rigorous. So leaders in the initiative need to ask themselves again and again what works and what doesn’t and why. They need to surround themselves by an environment of discipline pluralism.

And the conversation, I expect, will continue. It has been (so far; the exchange still has another day to go) a great opportunity to learn more about think tanks and their staff. I will continue to blog about some of the main issues that emerged over the course of the plenaries and workshops (and coffee breaks) over the next few weeks. And I hope, of course, to welcome Hans to again.

DFID’s new approach to long-term funding for think tanks

I have been asking about the details of these projects for some time and was half expecting they would be published more publicly but in the end I found them in DFID’s project database. (I wonder if there should be a rule that all publicly funded projects must be announced when signed -by funder and sub-contractor or grantee.) Anyway, these project are important for our understanding of the funding mechanisms available to think tanks as well as of the relationship between donors and these organisations. I do hope that as they progress, lessons will be shared about the approach as much as of the projects’ outcomes.

The Overseas Development Institute has been awarded an accountable grant (£9,750,000 over 4 years) to: generate improved evidence and analysis for managing risk and resilience. The grant follows from a 7 year Partnership Programme Agreement that is now focused entirely on NGOs but that provided ODI with just over £1 million a year until 2010. This is a new important project because it recognises that think tanks cannot be expected to behave as if they were non-for-profit consultancies.

DFID is negotiating these accountable grants with other UK-based think tanks (IDS and IIED are mentioned in the ODI business case document and their grants are supposed to be in process -but since I could not find them I must assume they have not been agreed yet). A similar accountable grant has been awarded to the U.S.based Center for Global Development (£4,038,000 over 3.5 years).The project documents say quite a bit about the aid industry’s and one of its top donor’s attitudes towards think tanks.

The business case for ODI’s accountable grant provides some detail about this. I really liked that in it, DFID seems to acknowledge that think tanks are valuable institutions in their own merit. Take, for example, the following quotes regarding DFID’s view of think tanks (they are the same for the CGD business case):

Think tanks continue to make important contributions to development thinking and are an important source of ideas and analysis for development agencies such as DFID.

In the past, similar work by think tanks has impacted positively on DFID’s policy development and operational guidance to our country offices. There is an ongoing need to ensure that DFID continues to benefit from the latest ideas and policy-focused research and analysis of leading thinkers in development globally, including those within the UK.

Think tanks benefit and complement DFID’s and wider development community’s work:

  • Their research, analysis, and advice helps us to make informed choices, generates challenge, and helps to generate and change ideas into policy practice;
  • Their global knowledge work covers key niche areas, such as low carbon technology;
  • They are an independent voice in policy debates and a well-regarded producer of a global public good, in the form of knowledge and ideas;
  • They cover areas of policy for which DFID does not hold adequate capacity in-house (such as urbanisation); and
  • They offer access to new networks for example through links and partnerships with individuals, researchers and other policy think tanks in emerging economies

These quotes are important because they not only recognise the value of think tanks but also outline several functions for which they are valuable. It is clearly not just about policy influence (or policy change): they access networks and policy spaces, they provide a reserve workforce on issues that policymaking bodies may not have their own in-house capacity, audit policymakers, etc.

Unfortunately, this all appears to be forgotten in the rest of the documents as the business cases then go on to describe projects that look more like a contracts than a grants:

DFID’s support will be structured around support for predefined outputs and expected results directly taking forward critical policy issues in development. This Accountable Grant is intended to consolidate under a single framework all policy work with DFID’s Policy Division during the four year period.  Unrestricted core funding is not provided.

Even the logframe (e.g. for ODI) feels the same and the results chains (or what DFID now calls theories of change) for each of the flagship projects demand quite a stretch of imagination and an over-simplification of reality (but this is what business case templates do to good ideas). The range of ways by which think tanks can contribute to society (and policy) is narrowed to the more limited liner ideal of evidence based policymaking.

Luckily, there is a provision, in the case of the ODI grant, to recommend new issues to study that may emerge over time. But it will be interesting to see how this is put into action.

In both business cases, in the risks section, one of the risks with a medium probability and medium likely impact is the duplication of work between ODI/CGD and the other think tanks supported by DFID. The suggested mitigation strategy is to have regular coordinating meetings between the think tanks. Think tanks in developing countries (as well as think tanks working on mainstream domestic policy issues in developed countries) might find this strange. What is wrong with a bit of overlap? After all, we cannot expect all think tanks in a country to agree with each others. So duplication (or triplication, even) can lead to more ideas being put forward and a far more interesting and pluralistic debate to develop.

If there were overlap and all the think tanks supported had the chance to study similar issues then DFID would have more options to choose from. And this can only be good. Unfortunately, the debate (sometimes quite heated) that think tanks in developing countries might encounter on a daily basis is not so common in the aid industry in the UK. Not all of DFID thinks this way, though: last year I had the chance to work with DFID Zambia to develop a programme to promote public economic policy debate.

On a slightly more positive note, page 16 of the CGD business case deals with how to assess value for money. The document says:

Neil MacDonald evaluated many of DFIDs Programme Partnership Agreements (“PPA”) in January 2011, including PPAs with think tanks such as ODI and IIED. He set out that “Policy change is essentially a high-risk activity because the outcome is beyond the control of the influencing organizations. Policy change may be slow and, given changes of political climate, is not linear and may go into reverse”. Hence, he recommended that, “Organizations undertaking policy influencing would be well-advised to develop elaborated theories of change with results chains that will allow achievement of meaningful milestones short of the intended outcome.” It is important that DFID builds on these lessons: it is likely that monitoring research activities, the dissemination of ideas, and the influence those ideas have had on policy actors, will be the primary mechanism through which DFID evaluates the effectiveness of the Accountable Grant.

It might be worth noting that Neil MacDonald did not evaluate ODI’s nor IIED’s PPAs -this was done by the organisations themselves under the supervision of David Lewis and Tom Forsyth from LSE- but this is beside the point. The problem is that theories of change are rarely as elaborate as he suggests they should be; although CGD’s focus on specific policy audiences and policies provides a way to avoid the usual shallowness of most theories of change. If you compare CGD’s and ODI’s results chains you find that the former’s are slightly more (although not in all cases) specific about the policy audiences or policies that need to change and how they must change along the chain. This is good practice.

To finish I’d like to recommend that in the future the theories of change are presented up-front. It should be clear, I think, that the strategy (the proposed activities) should respond to the theory of change -and not the other way around.

In summary, this is an interesting development in the way that DFID funds its think tanks (although CGD is probably safely beyond its sphere of influence). The PPA provided ODI with funds to transform its management and communications capacity -that took well over a decade. Several Research Programme Consortia in which ODI and the other think tanks have participated have also helped them to develop longer term research programmes. The accountable grand model could offer another layer to the fabric of funding for think tanks in the U.K. But does it create more or less independence? And will it lead to a more informed (and debated) policy? Time will tell.

Understanding the demand for World Bank research within the Bank

A year or so ago, DFID asked us (in ODI) to assess whether its research and evaluations were being used by DFID staff. Harry Jones and I developed an approach that focused not on the studies themselves (the supply) but on the way that DFID staff made choices and the roles or functions that evidence played in those. Evidence, we assumed, could come from different sources, one of which could be DFID research and evaluations. We did not, however, want to bias our study by focusing on them. And we felt that to say anything useful about the use that these had within DFID we had to do it in a wider context of all other inputs to decision making.

A recent study from the World Bank takes a different approach. Martin Ravallion’s study: Knowledgeable bankers ? the demand for research in World Bank operations, focuses on the demand for World Bank research. It does, however, share some of our conclusions:

The methods used affect demand:

Today’s research priorities may well be poorly matched with the issues faced by practitioners in these sectors. For example, the current emphasis on randomized trials in development economics has arguably distorted knowledge even further away from the hard infrastructure sectors where these tools have less applicability (Ravallion, 2009). Making the supply of research more relevant to the needs of development practitioners would undoubtedly help.

Absorptive capacity is crucial:

The differences across units in the demand for the Bank’s research are correlated with the incidence of PhDs and economists, suggesting that internal research capacity in operational units helps create absorptive capacity for knowledge in those units.

Researchers need to make an effort, too, however:

The slope of the relationship between perceived value and familiarity with research is positive but significantly less than unity, suggesting frictions in how the incentive for learning translates into knowledge. The responsiveness of researchers and the timeliness and accessibility of their outputs are clearly important to how much learning incentives lead to useful knowledge.

This study also finds two models that explain how research affects decisions:

In the first, they have a demand for knowledge that does not stem from its direct bearing on their work. Much development research is a public good. Practitioners might read research findings to better understand the world in which they work, even when that understanding is essentially irrelevant to the specifics of that work.

Alternatively, in the second model, research has a direct value in the work of practitioners—such as by informing project choices at the entry stage and assessing impacts later on—and research findings are sufficiently relevant and accessible to assure that practitioners become well informed.

We found a few more options, depending on the type of decisions that staff had to make:

  • In some cases, evidence generation was incorporated into the policy cycle
  • In others, evidence was used to make small incremental changes and corrections to ongoing policies and programmes
  • In other cases, evidence had to catch up to events
  • And in others, more often than not, it was used to make sense of political demands

We concluded that DFId was better at using the evidence and learning from it. These are two different things. It was also:

much better at using research and evaluation findings during or as part a project cycle than in more complex and emergent decision making processes.

In other words, it was better at working with a consultant than with an academic (or so I liked to think about it) -and this resonates with Ravallion’s finding of the importance of staff capacity.

This, in turn, points towards a possible mismatch between the ideals and realities of lesson- learning in DFID. For example, research is largely done outside the organisation by increasingly larger consortia with clear incentives to communicate to audiences other than DFID. The incorporation of their findings into DFID policymaking processes then depends on these programme‟s communications capacities, intermediaries (both technology based and knowledge brokers), and DFID staff themselves –who, according to the study are under increasing time pressures that reduce incentives towards engaging with research and evaluation processes and the analysis of their evidence and findings.

To us this meant that DFID was pushing research away from itself making it a foreign concept. Its efforts to bring it back by hiring Knowledge Brokers (PhDs) to mediate did not seem to fit with what emerged as the more effective model:

The system emerging from this is one where intermediaries between research and evaluation and policy and practice play a significant role.

In other words, learning in DFID (of the kind that promotes the incorporation of analysis into decision making and the development of a learning organisation) is more akin to a system with fewer intermediaries and more direct relations between users and producers of knowledge.

I liked this conclusion because it fits nicely with a belief that we need to pay more attention to people in this business of international development.

Ideas for the next Head of the Think Tank Initiative (and other think tank funds), if I may

In a few months, a new Head of the Think Tank Initiative will take on the responsibility of leading a new phase of a programme (see the second annual report) that has the potential of not only changing the fortunes of its grantees but also the way in which international development (and domestic) funders support think tanks all over other world.

In view of this, as well as an upcoming launch of its second annual report, I offer some ideas (unsolicited, but hopefully welcomed) to consider. I hope these are also relevant and useful to other think tank funds and initiatives that are directly and/or indirectly supporting think tanks and policy research centres across the developing world.

Clearly, I do not expect that these recommendations will be implemented right away -and at the same time I do not wish to imply that these are terribly original and that they have not been considered already. But I think it would be good for all of us to keep them in mind and maybe debate them a bit more in the coming months. In all cases, the hoped-for outcome is that the initiative’s work may be taken on by others and its effects (the positive ones) will be greatly strengthened, multiplied, and sustained in the long-run.

Be a thought leader

This is not a new sector but it is certainly not a very popular one. In the world of developing country think tanks there are not many minds and voices out there. This position, I believe, offers an enviable opportunity to contribute to the effort to develop the sector and our own understanding of it. As time goes by, any opportunity found for writing and publishing could certainly add value to our work, think tanks, and their funding. Goran Buldioski‘s blog, for example, is a perfect example what I’d hope to see more of across all think tank and research funders. Another one is Jeffrey Puryear whose account of think tanks in Chile does not cease to provide new insights into the sector.

Leverage domestic funds

Unless national and even local governments, businesses, and philanthropists take on the responsibility of funding research and think tanks in their own countries and communities, initiatives like the TTI will never see the end to their work. Many of the organisations funded by the TTI have been funded by various IDRC programmes in the past. In several cases, their funding helped to set them up many decades ago. None of these think tanks (not even the most established) have been able to ‘graduate’ from foreign funding and I would expect that the same fate awaits the younger ones.

There are serious problems with relying on international funds for research. First, researchers end up paying more attention to what donors are interested in than what their countries need. Second, they are quick to adopt discourses and processes that often lack relevance for contexts. And finally, to list only three, they become dependent on political (and funding) cycles on which they have no influence what so ever.

The TTI provides an opportunity to break this dependence that has not existed before. This time around there is a coalition of influential and interested international donors (IDRC, the William and Flora Hewlett Foundation, the Bill & Melinda Gates Foundation, the UK Department for International Development (DFID) and the Directorate-General for International Cooperation (DGIS) of the Netherlands). There are significant funds available to back their commitment (CAD113 million). The think tanks supported constitute an important community (12 in East Africa, 12 in Latin America, 11 in West Africa, and 16 in South Asia). And the media and governments have paid attention -it is not just think tanks who have noticed what is going on.

I think that the TTI, ideally working with other think tank and research funders, could use these sources of power and influence that it has so successfully developed during its first two/three years of existence, to target not just think tanks but also potential domestic funders. How to do this? I’ve thought of some ideas but I would expect that it would be necessary to think about which ones may be more appropriate for each think tank fund and its partners -and of course this would depend on each country.

  • Channel future funds (first some and progressively more) through national research councils or trust funds, set up independently from the government or any other policy actors, and governed by independent national boards to ensure autonomy. TTI funds could serve as a gravitational force for the funds of other international and national research funders who will feel greatly reassured by their presence. Better coordinated, funds would be more effectively allocated.
  • This model could be used to nudge national governments into topping up or matching the funding provided by international funders and, slowly, shift the balance in favour of domestic funds. Not so long ago the Canadian government engaged in a discussion with the Peruvian government to do just that -but the Peruvian government did not step-up to the challenge (if only there had been a bit more pressure?).
  • Mobilise global and northern philanthropists to convince their peers. Why not invite Jim Balsillie (I have a feeling he would do it) from Research in Motion and who set up CIGI to a series of dinners and seminars with wanna-be philanthropists in developing countries? Or host a show-and-tell event in Canada, Washington, or London to bring them together with other philanthropists. There are fantastic stories from developing countries, too –the Gateway House, for example, is a new Indian think tank funded by Indians. In China the government is investing heavily on think tanks. Why not work with them to convince their wealthy peers in Asia, Africa and Latin America to do the same? Better yet, use the Soros, the Gates, and the Hewletts to help set up new foundations in developing countries. Why not make them TTI funders, too? This would make it a truly global initiative.

The leveraging of funds should also consider how to mobilise other research funders in some countries where the focus should really be elsewhere: on universities and the development of human capital.

Unleash the power of your data

The TTI, like other such funds and initiatives that support think tanks and research centres, has and continues to gather invaluable information about think tanks from all around the world. This data offers unparalleled opportunities for research and learning -something I know is on the initiative’s agenda. Inevitably, some will have to be used with care as much of the information may be private but, where possible, I would encourage the new Head to allocate resources to make it public and easily available.

I expect that soon, think tank scholars, think tanks themselves, students, and the media will be using it and developing our common understanding on these organisations. Of course, just as the data is public so should any products that are based on it. Some ideas:

  • A competition could be established to encourage new studies; maybe the opportunity to present them at annual meetings or other international conferences.
  • A collaboration with an academic journal could be set up to publish a special issue of studies based on the data.
  • Incentivise TTI grantees’ researchers to undertake research on think tanks (using their own data as well as that of their peers) -see ‘amplify the voices of thinking directors’, below.
How great would it be if the data published by the TTI could be strengthened by data from other initiatives (e.g. the Think Tank Fund or the go to think tank index or DFID’s Research Programme Consortia, etc.)? Independently, some think tanks may want to add their own information to the database and thus support the development of a useable (and useful) knowledge base on the subject. So far all we have are lists and directories.

Fund new think tanks and start-ups (take risks on new big ideas)

Up until now, the TTI has funded organisations with a history (not always too long) and, hopefully, an established reputation. I think that, while necessary, this could have the potential undesired effect of limiting intellectual debate in some of the grantee countries. If all follow the same safe approach to funding, by funding the strongest organisations the donors could be closing the space of new ideas coming out of other centres. After all, good ideas are not the property of large and old think tanks; anyone can have them.

There is also the issue that there being so few think tanks in some countries, the ones that exist have developed a sort of monopoly on the production of research. And control the prices and quality of knowledge.

TTI could fund start-ups around new big (which is not the same as expensive) ideas that need that little extra help to develop -in a context of more domestic funding this could lead to new sustainable think tanks and programmes. Start-ups do not need to be expensive -this is common the response I get to this suggestion, before it is dismissed. A couple of friends and I have been working to set up a forestry think tank in Peru and working hard to keep its set-up costs very low. How? First, we have decided to focus the work of the think tank on analysis and synthesis before moving on to more long-term research -this means starting with a small team and a small research budget. Second, we propose to set it up first as a programme within an existing research centre so that we can avoid the usual costs involved with setting up a new organisation (e.g. renting office space, hiring staff, sorting out legal issues, etc.). To make sure that the think tank is autonomous from the start we are putting together an independent board and will call it ‘centre’ rather than programme or project (sounds silly, but I think this matters). Third, we have developed a draft plan that explicitly calls for a reduction on any foreign funding and increase in funding from local sources (in the form of grants) -in fact, right from the beginning the initiative has the support of the private sector.

If the think tank proves to be successful (and makes a contribution to the sector) then we would seek funds (local) to set it up as an independent body. But only if. Like any start-up there is always an element of risk.

Some of the recommendations suggested under ‘leveraging domestic funds’ apply to this idea but a focus needs to be given to funding innovations or existing initiatives such as JCTR’s the Basic Needs Basket or CIPPEC’s Agenda Presidencial. This is as close to a win-win situation as one can get: the funds would be rewarding organisations that have already invested their own money and effort. Another way is to include newer and smaller organisations in the next round of grants ; which to some degree has happened already.

Don’t be afraid to drop a few think tanks along the way

There is no reason why a think tank must exist if it does not fulfil its functions -and funders should not fund if they are not happy with what’s produced. If the think tanks (and their funders) set out realistic objectives there is no reason why the TTI grantees should not be able to meet them. All too often, I believe, donors (and other northern ‘intermediaries’) are afraid to walk away from a southern organisation. There is a mistaken sense of responsibility for them.

In the end, this simply closes the space for new initiatives, rewards poor practice, and creates dependence.

And taking risks (see the previous point) involves accepting some losses: but nothing ventured, nothing gained, as they say.

Bypass Aid

Not entirely, of course, but I would encourage the new Head of the TTI to look for inspiration and allies among northern and southern non-aid think tanks. These are the organisations that deal, on a daily basis, with domestic issues (economics, trade, agriculture, health, education, etc.) in developed countries and on non-aid issues (security, foreign policy, right and left-wing politics, etc.) in developing countries.

I think that the TTI grantees have more to learn from them than from northern international development think tanks, NGOs and development consultancies. International development organisations tend to work in the world of ‘marginal’ politics (aid, after all, represents a negligible portion of donor country national budgets); on the other hand, domestic ones, like developing country think tanks, deal with ‘mainstream’ politics.

IPAR Rwanda, one of the TTI grantees, has a very interesting collaboration with IPPR, in the UK, going on. I would encourage more of these links -but also between think tanks in different ‘southern’ regions, such as the Think Tank Fund initiative to link Eastern European think tanks with their peers in Latin America and South East Asia.

Encourage and amplify the voices of your thinking Directors and thinktankers

This is not to dismiss the important work undertaken by all others, but many think tank directors (for instance people like Orazio Belettinni and Simon Maxwell, both of whom have contributed to this blog) have taken up the challenge of managing think tanks and chosen a critical thinking route, too. Rather than ‘getting on with it’ they have decided to turn their grants and jobs into an opportunity to think about their organisations and their roles. Their experience, research, and the lessons they are sharing with other directly relevant to other think tanks. The same is true for other staff (researchers or communicators) in think tanks.

The TTI could encourage more of this, possibly by:

  • Providing small bits of additional funding for studies on think tanks by think tanks.
  • Setting-up a simple blog for the initiative or an online space; or encouraging them to write for other spaces (like this one?)
  • Pairing-up their most promising candidates (and I stress: promising ones -those who make the effort) with peers in more developed (or more successful -if this is possible to determine) think tanks.

Expand the community

As the think tanks settled into the initiative and their new grants, the TTI could, slowly (and I stress, slowly -and carefully), invite non-TTI grantees to its meetings and events (regional and global).

  • Maybe, they could be included in some of the evaluations (could they be considered as counterfactual?) and research commissioned by the TTI.
  • They could be invited to regional learning events or trainings such as the recent Latin American think tanks meeting held in Lima.
  • Future learning events could be organised in coordination with other initiatives, thus maximising opportunities for networking and sharing.
Core support need not be only interpreted as ‘cash’. In-kind knowledge contributions can be equally valuable for think tanks and for think tank directors. A good idea can create as much ‘space for manoeuvre‘ as funding.


There are other more micro-level ideas that could be considered relating to capacity development, communications, and links to non-TTI think tanks. But those presented here, I hope, will provide some inspiration for a successful and meaningful intervention.

And, of course, I hope these ideas and recommendations are equally useful and relevant for other initiatives.

More information on the Head of the Think Tank Initiative post and how to apply.

Best of luck.

Theories of change: an annotated review of documents and views

A few weeks ago, Jeff Knezovich, asked the members of the evidence based policy in development community for some feedback on the way in which Theories of Change (ToC) were being used in planning, monitoring and evaluation of policy research interventions. The response was quite encouraging and many resources were shared.

In my view, ToCs can be a powerful planning, monitoring and evaluation tool but often interventions focusing on research based policy influence forget to base their ‘theories’ –which I have called change pathways elsewhere–  on actual research of how (policy) change happens.As a consequence, the ToC sounds and looks brilliant -perfectly logical- but it is terribly irrelevant for the context in which it will be deployed: e.g. online communication in Sierra Leone (where there is hardly any electricity) or media dissemination in Zambia (where the public and private newspapers have polarised opinion so much that it would be difficult to get a neutral take at politically sensitive evidence, or having he same ToC for interventions in Latin America, Africa and Asia (or even within Latin America Africa and Asia).

Anyone attempting to develop a ToC would do well do first reflect on how policy change happens -and by this I do not mean a quick two hour session in a workshop. This should be the most important part of any planning process. And do not be afraid to be challenged. It is not supposed to be easy. If it was, we would not be spending all this money (and it is a lot of money) to try to influence policy.

In this post I am collecting some of comments and resources shared in the community as well as other that I have found in other sites (for example, a post by Kris Putnam-Walkerly in early 2010 with 10 great resources).

Here is a synthesis of the resources and the comments from the contributors to the online debate (if you have any more to add please do) and from other sources too (references included):

Simon Hearn:

My favourite paper on this subject is by Doug Reeler of CDRA, not so much of a how-to but he lays out some very interesting theories of how social change happens.

There’s also these links:

Jennifer Morfín, offered some key readings:

Francisco Perez offered a reflection:

I just want to write a little note on the theory of change issue from my 11 years working on impact assessment of Public Programs and NGOs.[Note, Francisco is based in Nicaragua.]

Theory of change is a basic element of every development intervention; indeed what we have found is that those programs with a week theory of change, tend to be just wasted money. A second important issue is that these projects have no-clear cut scenarios and variables to monitor changes in the context.

Now, for public policy influence, we clearly need a map of actors; however, our argument should come from a theory of change supported by evidence. For instance, if we promote a land management plan in order to protect water basins. Thus our argument has a theory of change behind; we are saying that if we have an appropriated use of soils, we will not have water scarcity. So, even if we do focus on incidence only, we should have a strong and sticky argument or idea to promote with a theory behind it.

A note from me: What Francisco is suggesting as well is that the Theory of Change –in this case, the pathway of actions, outputs, outcomes and impacts expected- need to be based on a sound theory of how change actually happens. And this is no simple matter.

Lori Heise, also contributed with some personal experience:

I think one of the challenges Jeff is alluding to is how best to apply these notions to the particular program structure that he is referring to, which is a DFID’s research programme consortium.  These are large 5-6 year collaborations between research institutions that commit to work jointly on a particular theme or challenge (like health impacts of climate change — or the one I lead, which is tackling structural drivers of HIV).  The problem is that the goal of the program is to influence policy and practice through generating and applying new knowledge.  The collaboration is rather diffuse and depends on leveraging funds from many other sources.  Such projects require a theory a change about how you believe evidence influences policy change in a particular setting and/or at the global level.  The reality is that evidence is seldom the defining feature driving policy in my experience, although of course our goal is to work toward having it be a greater factor in the future.

So, is DFID’s theory of change for the RPC design wrong? Or at least in need of a revision?

Tionge Saka reminded us that:

Weiss (1998) explains that the term program theory refers to the mechanisms that mediate between the delivery (and receipt) of a program and the emergence of outcomes of interest. There are two kinds of theory and these are, program theory and implementation theory. These two intertwine in the evolution of the program and the combination of these two is called program’s theories of change. A programme theory usually includes

  • Programme inputs such as resources and organizational auspices
  • Programme activities which represent the manner in which the program is being implemented
  • Interim outcomes – that is the chain of responses the activities elicit, which are expected to lead to
  • Desired results

Rick Davies shared Patricia Rogers new book: Purposeful Program Theory: Effective Use of Theories of Change and Logic Models

David McDonald recommended:

An outstanding book on this topic has recently been published:

Funnell, SC & Rogers, PJ 2011, Purposeful program theory: effective use of theories of change and logic models, Jossey-Bass, San Francisco, CA that comes with a companion website.

From Regina Gyampoh-Vidogah:

According to ActKnowledge, a Theory of Change defines all the building blocks required to bring about a long-term goal. ‘Like any good planning and evaluation method for social change, it requires participants to be clear on long-term goals, identify measurable indicators of success, and formulate actions to achieve goals.’

And the toolkit: Theory of Change: A Practical Tool For Action, Results and Learning, prepared by Organizational Research Services

Donna Loveridge also noted that:

As with many terms theory of change can mean different things. The evaluation literature has quite a bit on theories of change and while the ideas have been around for several decades they are getting greater currency/more discussion in the last few years.

  • Huey Chen (1990: 40) defines theory as a ‘set of interrelated assumptions, principles, and/or propositions to explain or guide social action’.
  • van der Knaap (2004) refers to theory as the collection of assumptions, norms and values regarding the causal links between a program’s actions and the outcomes. These definitions differ from the standard positivist-scientific theories.
  • Shadish (1987: 95) describes program theory as ‘hunches and intuition built on common sense and on accumulated professional wisdom and experience about the nature of social programs and how they change’.

Theories may relate to social science theories but may not. In relation to research/knowledge, perhaps ideas or assumptions around research influence and use might be worth exploring (I am more familiar with the literature on evaluation use but can see that there are probably similarities. – see Kirkhart, K. E. (2000) Reconceptualizing evaluation use: an integrated theory of influence. New Directions for Evaluation, 88).

If research/knowledge is linked to policy influence, I have found this document quite useful. Maybe another area to look at is knowledge and learning.

Other references that you may find useful include:

R2D offered a synthesis of the resources here as well as others. Some of the ones that they added on their own include:

A guide to monitoring and evaluating policy influence.The following article written by Harry Jones at ODI has a very useful introduction to the three  of the most common approaches to theories of change: causal path, dimensions of influence and actor-centred theories.

Keystone have developed some useful resources on how to develop a theory of change, plus they also provide a useful template to start developing your own theory.

  • Theory of Change guide A guide to developing a theory of change as a framework for inclusive dialogue, learning and accountability for social impact.
  • Theory of Change template This interactive PDF template allows you to input information directly into it to build your theory of change.

The Social Framework developed by Rick Davies is an actor centred approach that attempts to map pathways to change through different actors and their relationships to each other.

Predating all of this, the Philanthropy411 Blog posted 10 great resources for creating a theory of change in March 2010:

For general information about what a Theory of Change is and some examples:

  1. Theory of Change As A Tool For Strategic Planning introduces the use of the Theory of Change approach for planning community-based initiatives using examples from the The Wallace Foundation Parents and Communities for Kids (PACK) initiative.
  2. Theory of is a collaborative project of the Aspen Institute and ActKnowledge, offering a wide array of resources, tools, tips, and examples of Theory of Change.
  3. ActKnowledge is currently piloting Theory of Change Online (TOCO), a free, web-based application to create Theories of Change and to learn more about the methodology.
  4. They’ve also provided a guided example of how one Theory of Change was developed.
  5. You Can Get There From Here: Using a Theory of Change Approach to Plan Urban Education Reform” by James Connell and Adema Klem gives an overview and an example in the field of education

For useful manuals, facilitators’ guides, and tools to create a Theory of Change:

  1. The International Network on Strategic Philanthropy has a Theory of Change Tool Manual.
  2. Theory of Change: A Practical Tool for Action, Results and Learning” was created under the guidance of Tom Kelly (@tomkaecf) at the Annie E. Casey Foundation.
  3. The Aspen Institute’s Roundtable on Community Change created “The Community Builder’s Approach to Theory of Change,” which is a practical guide for facilitators, including what to do before and during meetings with stakeholders, suggested participants, and recommended materials.

And to better understand the difference between a Theory of Change and a Logic Model check out:

  1. GrantCraft created “Mapping Change: Using a Theory of Change Approach to Guide Planning.” (BTW, GrantCraft has produced terrific guides on all aspects of grantmaking, so you should definitely check them out)
  2. Theories of Change and Logic Models: Telling Them Apart” is a helpful PowerPoint presentation.
One of the comments to Kris’ blog, I think, deserves special attention (by Holger):

We are living in a world that is ruled by laws of complexity and dominated by an ever increasing degree of uncertainty. However, projects are still managed from a reductionist and mechanistic viewpoint. So far, there have only few simple and applicable models that help organizations to look at their change projects from a complexity viewpoint.

That’s not a model, and even less so a theory. I am more interested in a meta model that helps people to find their own theory of change. For that, we have developed the Change Journey (

The Change Journey is a radical approach to change. It is based on the paradigm that change in organizations is not a linear path from A to B. We offer a tool for developing a specific change model: The Change Journey Map. The Map is inclusive – which means whatever tools and models and theory you are used to can be incorporated.

Corruption free think tanks

The work of think tanks is never straight forward. Researchers do not exist in an institutional vacuum with no links to the real world. They must access information that is often kept behind closed doors, engage with policy makers whose agendas are controlled by often unscrupulous people, etc.

This, however, does not mean that think tanks must bend or break the rules to work. Jeff Knezovich’s post in this blog last week made a passing reference to the new UK Bribery Act. I am sure there are similar such laws in other countries so this is equally relevant to organisations working outside DFID’s sphere of influence.

According to the Bribery Act:

What is a bribe? All payments of bribes, no matter how small or routine, or expected by local customs, are illegal. You are breaking the law whether you give or receive a bribe. Unlike some anti-bribery laws, the Bribery Act applies to bribes paid both to public officials and within non-public operations. A bribery offence is committed if the intention of the briber is that the person being bribed improperly performs his/her duties. Improper performance will arise if it is intended that, by paying the bribe, the recipient of the bribe would be expected to act otherwise than in good faith, an impartial manner or in accordance with a position of trust. Expectations are judged by UK, not local, standards.

And it is particularly relevant for organisations who claim to be promoting transparency and accountablity.

Here is an anecdote that I have enjoyed telling for quite some time:

I had been invited to a meeting of the new grantees of a programme seeking to promote transparency and accountability in policymaking. I was there to talk about policy influencing. A past grantee had been invited to talk about their own experience in doing so. They used schools monitoring data to make recommendations to improve the quality of education in Guatemala. The process involved working with the school monitors who collected the data that they then used to do the necessary analysis. As he was describing this he was interrupted by one of the participants who asked how much they had paid the monitors. The presenter was a bit confused. Nothing, he said. They were doing their job, we just asked them for the information, which is public anyway. Again, the man who interrupted responded by saying that they should pay them because they were acting as their research assistants.

Before the presenter could say anything a debate started among the participants. Researchers from different parts of Africa intervened saying how it would not be possible for them to do any of this without paying them for the information; some argued that this was right and normal, others that this was not advisable -although the reason given was that these small incentives would translate into large payments once the project got to the top and tried to influence senior policy makers ‘there would not be any money left in the project.’ At this point, the representatives of the donors left the room -they clearly did not want to be part of the conversation. Good timing too because the presenter interrupted to say that he did not want to judge but what they were suggesting was not possible in his country because, to put it bluntly, it was corruption. 

No, it is just an incentive, said someone. 

So if you choose not to look the other way, Transparency International provides some useful recommendations based on the finding that :

among NGOs anti-bribery procedures are either poor or non-existent. This is often explained by the difficult circumstances in which NGOs are operating on the ground. Paying a bribe is seen as the only way to get things done.

TI has published  some very useful tips for fighting corruption:

Conduct a risk assessment: where is your organisation exposed to a high risk of bribery – and how effective are its anti-corruption policy and management systems?

Introduce a zero-tolerance policy: put in place a headline policy that notes the damage that corruption does to your goals and mission, the importance of strong internal anti-bribery systems and makes it clear that the organisation does not tolerate bribery in any form. Anything less will provide a weak defence under the Bribery Act.

Information gathering: it is important to know whether bribes are being paid by your employees, agents or partners – and if so where, how much, and how frequently. This information is crucial if the organisation is to implement a zero-tolerance policy and, where necessary, try and ‘design out’ bribery from future projects or operations. Paradoxically, creating such a paper trail may provide evidence in a prosecution. However, such information gathering would probably be regarded as part of an ‘adequate procedure’, and therefore failure to assess the extent of bribery in an organisation might create a liability for senior managers and directors who could be accused of ‘consent and connivance’ by turning a blind eye.

Put in place robust anti-bribery systems: having in place ‘Adequate Procedures’ is the only defence to protect an organisation against corporate liability under the Bribery Act. TI produces a 20-point checklist for companies to assess anti-corruption procedures. Although it is aimed at companies, it is also relevant to NGOs. TI is seeking funding to develop NGO- specific tools.

Training and support: implementing effective anti-bribery systems can be a difficult process, and employees and partners may feel vulnerable and ill-equipped, especially in a transition phase from one way of doing things to another. Proper training and support is a vital part of this process.

Zoom zoom zoom, capoeira mata um: communications in the age of austerity

‘Capoeira mata um’, or perhaps more accurately, ‘capoeira foi morto por um’ – at least on one sunny day last summer.

By Jeff Knezovich, Policy Influence and Research Uptake Manager for the Future Health Systems Research Programme Consortium*

Now if you’re like 99.9% of readers of this blog, you’re probably wondering a) why Enrique let me do a guest blog, and b) what in the world Brazilian Portuguese has to do with austerity communications. Let me explain.

Zum zum zum’ is a popular song in the capoeira circuits. Indeed it is so popular that you might recognise it from certain Mazda adverts. Zoom zoom.

The first line literally means ‘capoeira kills one’, but that’s not what this story is about. It’s about a time where ‘capoeira was killed by one’, how that has changed the development communications landscape, and what lessons development policy entrepreneurs can draw from the famed Brazilian martial art/dance.

When the Conservative-led coalition came into government in the UK last May, one of the first targets in their crosshairs was ‘profligate’ Labour spending, which they argued had left the country in dire economic straits. ‘Communications’, synonymous with spending, quickly became a dirty word across Whitehall. And, although they promised to protect – and even increase – aid spending, the Secretary of State for International Development, Andrew Mitchell, made value for money of British aid a clear priority. Among other things, that meant cuts to a cherished Labour objective: making the argument for aid to the British public.

Indeed, who could argue with cutting aid funds to a ‘Brazilian dance troupe’ in Hackney (a neighbourhood in East London)? In one of his first ministerial speeches, Mitchell made clear that these sorts of activities would no longer be tolerated and that the aid argument would be won not on explaining it to audiences at home but by improving lives abroad.

By mid-2010 the UK, not just the newly rebranded UKaid, entered an age of ‘austerity communications’. Government websites were among the first to be scrutinised. As it turned out, the UK Trade and Investment website, in what is frankly a crude measure, cost the government nearly £12/visitor, and that’s discounting staff and operating costs. Hardly value for money, by any definition.

Such costs called for a rationalisation of government websites, an edict that has trickled down through the ranks of DFID. In the most recent advice given to its large portfolio of research consortia, it was suggested that no programme should have a standalone website. Recommendations have also emerged that no money should be going to promoting large programmes as brands independent from their host organisations, and that hosting events that cost over £20,000 require cabinet-level approval.

While the value for money of these arbitrary rules is dubious at best, the push for austerity communications should be welcomed by development researchers, research communicators, knowledge intermediaries and policy entrepreneurs alike. Just as it is an incorrect assumption that less polished looking communication activities are cheaper (just ask the 2012 Olympic committee), it is equally untrue that communication has to be expensive. An unhealthy economy has emerged in the research communication field: from expensive and self-indulgent websites to exorbitant per diems for participation in events (which may soon be considered bribery in certain circumstance under new UK legislation) to paying for media placement.

My friend and former colleague, Nick Scott from ODI has spoken widely about free and low cost online tools that can help establish and bolster an online presence, so I will instead broaden the discussion in the rest of this post to how communications has the opportunity to be more effective in these tight times.

Ironically, capoiera’s existence today is a shining example of massive impact with limited resources. Capoeira emerged from slaves of African origin working the sugarcane plantations of Brazil in the 1600s. As a martial art, slaves used it for self-protection, to escape and to defend Quilombos (informal settlements of escaped slaves and others living outside the law). As capoeira was a clear threat to the Portuguese slave owners, it was outlawed, forcing capoeiristas to disguising the practice as a form of traditional dance. And perhaps at a most basic level, this clandestine approach of obfuscating traditional approaches to research communications will be necessary, but only when they are the most appropriate techniques to reach an objective.

Ultimately I hope that these new rules force us to change rather than conceal. And here, capoiera offers more lessons to inform an innovative approach to research communications.

There are several styles of capoiera, the two most popular being capoira regional (pronounced ‘hey-shu-nal) and capoiral angola. Capoeira regional is the newer, flashier side of capoeira, with rodas usually going at a quicker pace and with more jumps, spins and kicks. The more traditional angola style is comparatively slow place and low to the ground, with combatants usually keeping at least one hand touching the ground at all times. Both styles are popular, but capoeira angola is considered the more difficult. It is a reflective and strategic style and requires greater control – consider it the chess of the martial arts world. And perhaps these two styles represent the difference between research communications and marketing as it was promoted under the Labour government (capoiera regional) and the era of austerity communications  (capoiera angola).

There are a few principles operating in capoeira angola: 1) conserve energy and maintain endurance; 2) use the slow pace to develop an understanding of the opponent and use that understanding to defeat her/him; 3) exploit opportunities and make every attack count. Development communications would do well to abide by these principles.

1)    Conserve energy and maintain endurance: As Enrique has noted elsewhere, think tanks and research organisations that chase visibility at the cost of substantive research and influence do so at their own peril. The fact is that we are operating with finite resources and there is an opportunity cost associated with pursing any given engagement activity. To that end, we must recognise that substantive influence does not happen overnight. We need to be prepared to invest in long term strategies that focus on building relationships and trust – neither of which is founded on glossy brochures.

2)    Understand the opponent: At its least, austerity communications should give us time to pause and reflect on how policy influence and research uptake actually occur in our individual contexts. Maybe getting an article into a journal with the highest impact factor isn’t going to change practice on the ground. Maybe the long research publication isn’t the best choice in Cambodia, where most business and politics is transacted verbally. Maybe the flashy website that woos donors isn’t the right option to reach researchers in the D.R. Congo where internet penetration is notoriously low.

Additionally, a good understanding of our audience allows us to extend a ‘being there’ strategy from the web to other forms of communication. Beyond thinking of where in the web world your audiences are spending their time, also think through: What publications are your target audiences already reading? What media do they already engage with? What events are they already attending? Spending effort getting into these spaces may be much more valuable than simply creating more of your own spaces and spending resources to market them. Enrique’s recent post on ‘confirmation bias’ should be a good reminder of this – people are predisposed to agree with evidence from a source they already trust.

3)    Make every attack count: Value for money doesn’t necessarily mean spending less money, it means spending it wisely. Instead of a throw-everything-at-the-wall-and-see-what-sticks approach to communication (which can be particularly valuable when working in complex environments as long as there are in-built learning mechanisms), under austerity communications we will likely need to be more selective in our communications activities. So when an opportunity does arise, and we do think that it is the right intervention for the right objective, go ‘all in’ and put significant resources behind it.

In a review of DFID’s recommendation to spend 10% of funds on communication activities for certain types of programmes that Enrique and I both participated in a few years ago, we found that some programmes were taking the advice literally and cascading the 10% funding throughout all of its interventions – but some research is more communicable than other research. Austerity communications will require a greater investment in horizon scanning (and tools that facilitate this), and then taking every advantage of opportunities as and when they do arise.

*[This is the first of I hope many more contributions from practitioners and experts in the field of think tank management, communications, funding, etc. If you would like to recommend someone please contact Enrique Mendizabal on]

on Monitoring and Evaluating Influence

Harry Jones from RAPID just published a background note written as a part of a project that we worked on last year with Jeremy Clarke to develop a How to Note on M&E of Policy Influencing for DFID (the HTN is forthcoming but maybe this presentation on M&E of policy influence I did for the Think Tank Initiative may be of interest). The project was based on the lessons learned (and the limits of) on a review of DFID’s health policy influencing and a How to Note on Policy Influencing I worked on with Matt Gordon and Jessica Proust based on the RAPID Outcome Mapping Approach (ROMA).

From Harry’s paper:

This paper provides an overview of approaches to monitoring and evaluating policy influence, based on an exploratory review of the literature and selected interviews with expert informants, as well as ongoing discussions and advisory projects for policy-makers and practitioners who also face the challenges of monitoring and evaluation. There are a number of lessons that can be learned, and tools that can be used, that provide workable solutions to these challenges. While there is a vast breadth of activities that aim to influence policy, and a great deal of variety in theory and practice according to each different area or type of organisation, there are also some clear similarities and common lessons.

Rather than providing a systematic review of practice, this paper is intended as a guide to the topic, outlining different challenges and approaches, with some suggestions for further reading.

Also very useful is CIPPEC’s manual for M&E and KM of policy influence (in Spanish).


Get every new post delivered to your Inbox.

Join 7,065 other followers