I was listening to a podcast on Free Will, determinism, indeterminism, etc, and it got me thinking about two fundamental questions related to current efforts and ideas focused on monitoring and evaluating think tanks: who is responsible for a think tank’s influence? and what are they actually responsible for?
Attempting to answer them led to two further issues: a question: when should think tanks claim influence? and, finally, a conclusion: any claims of influence are political acts; they are claims of power over others.
This post is one of those posts that develops as I write it. It is probably longer than it need to be but, in a way, the fact that it has been so difficult to write reflects the complexity of the challenge at hand.
Who really is responsible for the influence that a think tank may have?
It seems to me that the most honest answer is that, probably, not the think tank. Here are a few reasons for this (I am sure there are a lot more):
The original sources of power lie outside the think tank and in the past
Think tanks are small players in a much larger, historical, and complex game. Nowhere in the world are they the most powerful players and everywhere in the world they get their power (credibility, mandate, legitimacy) from others. In contexts where the State rules, think tanks get their mandates from the State (e.g. China). Where elites are in control, elites give think tanks their mandate (e.g. the US). Where donors are the big fish in the pond, donors are the source of their power (e.g. any Aid dependent country). If the right wing is in power, right wing think tanks will be more powerful; if the left is in power, left wing think tanks will be more powerful; and if people are tired of ideological fighting then the ones that shun away from consistent and potentially ideological narratives are likely to stay on top.
Think tanks are either pawns or parts of bigger players or systems. They are purposely or accidentally manipulated by others with more power or made to play a role determined (purposely or not) by them.
Not even the biggest think tanks can claim otherwise. Brookings’, Heritage’s and any of the other large and powerful US think tanks’ access to Congressional hearings or committees is controlled by whoever controls Congress. And they, the Congress men and women who ‘choose’ to invite them or not to these spaces are controlled by the money and the people that put them there (or elected them); or by the game of politics itself. Brookings’ own legitimacy is greatly down its endowment and long history: the power that its current researchers have is greatly based on the hard work of their predecessors’ work over the last century.
The same applies to other ‘sources of power’: access to policy spaces, to resources (e.g. to databases), to the right staff, to the necessary funds, to knowledge about the context they are attempting to influence, etc. is defined by others -and in the past.
Perception is everything
I am always getting into arguments with researchers in think tanks who say that it is possible to be unbiased. My view is that bias is defined by others, not by the think tanks, and since everyone has a bias (we all have ideologies) then every think tank and every researcher is biased (to various degrees).
What matters then is the perception of impartiality or rigour; and this is never universal nor stable. Even Brookings is labeled as ‘liberal’ by the conservative right in the United States. In Peru, without any proof, Universidad del Pacifico is seen to be on the right and Catolica on the left. As the public mood shifts from the right to the left and back, these think tanks’ position shifts, too, even if they haven’t themselves.
Credibility therefore is determined by others. There is only so much think tanks can do in this respect. They can do a good job -they can do a brilliant job even- but being believed, becoming a trusted source of ideas, is ultimately out of their control. If policymakers or the media trust a think tank, this is more likely to be based on their own backgrounds, on the opinions of others, on whether the ideas proposed by the think tank are aligned to those they already believe in, on personal and professional networks, etc.
Influence may be a factor of think tanks’ capacity, too; but are they responsible for it?
Of course there are ‘internal’ factors that can explain the influence a certain organisation may have. Some call these ‘endogenous’ factors. But how endogenous are they, really? Is our capacity really ours to claim?
Good leaders matter when it comes to influence, but are good leaders developed by the think tanks? Unlikely. It is more likely that think tank directors developed their leadership competencies at university, in previous jobs, from peers, etc. God knows what they went through and how these experiences shaped them.
Good researchers are another factor: but these are impossible to conjure out of nothing. They come from good universities (somewhere in the world) and maybe from other think tanks. Even if some of the researchers’ skills were developed ‘in-house’ they would have been the consequence of good mentoring (by good mentors who are likely to have learnt to be good mentors outside of the think tank), good fundamental skills (from university), good training programmes (developed by good HR managers -who are unlikely to have been trained in the think tanks themselves), etc.
We could argue the same thing about communicators, finance managers, HR managers, administrators, board members, etc. It is impossible to say anything meaningful about the influence of a think tank (the entire organisation) without granting them an important portion of the claim. And their capacity is likely to the be the consequence of investments made (in education, health, infrastructure, technology, etc.) by people and organisations decades ago; and these are the effect of institutional (political, economic, social, cultural, etc.) arrangements sometimes centuries in the making.
A think tank’s capacity is therefore not endogenous at all. It did not grow from within the think tank like some sort of benign tumour. Capacity is a factor of forces outside the think tank’s control: they are exogenous.
If we are honest, anyone could have done it… eventually
Current efforts to measure the claim of influence rarely ever ask: what would have happened if think tank A had not done the study that was used in Z policy decision? When think tanks and their funders claim an organisation’s contribution (which is the same as partial attribution -so in essence trying to measure attribution and contribution is the same thing) they fail to assess alternative scenarios.
One possible scenario is that change would have happened anyway. A policymaker could have decided to do what the think tank suggested without the need for the think tank’s evidence. The fact that the think tank provided evidence for it does not explain the decision. Even if the policymaker referred to the study the cause-effect relationship cannot be automatically established as the study could have been used ex-post, simply to illustrate or legitimate a decision already made.
Another scenario is that the policymaker did in fact first think about the idea after reading about it in a think tanks’s study. But in this scenario, like in most cases, there were other think tanks and organisations saying the same thing. In this case (besides the fact that the think tank’s access, choice of research, communications, etc. will have been determined by others) it is not possible to say that the policymaker did not hear about it from someone else -or would not have heard about it from someone else if the think tank had not been there int he first place.
Since an individual think tank entirely cannot control when it gets the money to do a study, or whether its researchers and communicators use the money efficiently, or who pays attention to what it has to say and when, etc., it cannot therefore claim success (of being heard) all for itself.
Yet another scenario is that the change would have happened but much more slowly. Change could still take place but by other means and in other ways. Can a single think tank claim success for something that could have taken place in the future? Maybe it could say that it helped to speed up the change -but that would be it.
Now, add to these scenarios the fact that there are lots of other organisations (with their own influences) attempting to influencing every single decision at all times. How could a single think tank claim success for what is nothing more than the consequence of many more decisions and choices made by others?
Here is a quote I like: On the evaluation of a given think tank in affecting an important change in public policy, Murray Weidenbaum, in his book ‘The competition of ideas: the world of the washington think tanks‘, reports that John Hamre (from the Center for Strategic International Studies, CSIS) is reminded:
of the case of Anna Nicole Smith’s baby, where several men volunteered that they were the father. However, in the case of public policy there is no equivalent of a DNA test to declare the parentage with a comparable degree of certainty.
- Think tanks are not responsible for political choices
Another important point is that think tanks are not responsible for political decisions: and policy, even the most technocratic of policies, is nothing more than a set of political choices. Sure, think tanks can recommend and even lobby for change but someone else is responsible. Regardless of the political system, this responsibility formally falls on elected or unelected politicians and public servants -with more or less accountability. And I would expect think tanks, more than any other organisation, would want this to remain the case. But, if we follow the same line of analysis as with the think tanks, the origin of their power is likely to lie somewhere else; and so are the reasons for the choices make.
Even if a policymaker ‘copy pastes’ a think tank’s proposal (and we should not be happy about this) the decision to ‘copy paste’ would have been the policymaker’s own: 100% his or her decision (with all the influences he or she is subject to). A bad choice (and let us not forget that think tanks can get it wrong quite a lot of the time) is likely to cost the policymaker his or her job while the researchers will no doubt be busy with the next project.
So, again, the think tank could not claim any success for itself for something that it is not responsible for. This is particularly true when a think tank has provided advice as part of a consultancy agreement. Assessing influence in these cases is quite useless.
What are think tanks responsible for?
Am I saying that think tanks are not responsible for anything? That their leaders and staff can wash their hands off all responsibility? Certainly not. Think tanks and their staff can still control their immediate environment (so there is some free will after all): they are responsible for doing a good job just like a bank manager is responsible for balancing the books at the end of the day or a bus driver is responsible for getting passengers from A to B safely and on time. They, too, face external forces that may affect their work but we do not just let them off the hook.
Outcomes are helpful for planning and learning but not so much for evaluation
When we plan we take outcomes (e.g.uptake and policy change) into account but this does not mean that we should be evaluated against them. I think that we all agree that the context of think tanks matters a lot more than anything think tanks can do when it comes to influence. Even before my Free Will analogy we would have accepted this as truth.
If think tanks are not ultimately responsible for the policy decisions (or any decision) made by anyone who participates in the policy process then what would be the point of assessing whether an outcome happened or not as the consequence of what the think tank did?
The only things that would be legitimate to assess are if the choice of strategy the think tank made (with its resources and in its context) made any sense in relation to the outcomes sought and if the strategy was implemented appropriately (with its resources and in its context).
Of course, it is important to asses if an outcome take place or not. But not to pin them down to a particular organisation’s strategy but because said organisation ought to be taking into account any changes in their context, including their objectives, and reflecting them in their strategies accordingly.
Turning inputs into outputs
A good job, for a think tank, is doing good research and communicating it properly. They must do this without wasting resources, following certain principles (depending on their founding and emerging values), ensuring their sustainability, strengthening their environment, etc. Most importantly, they are responsible for making sure that their work, given their context, resources, history, etc., makes sense: is relevant, useful, insightful, helpful, etc..
In other words, think tanks must be good at turning inputs into outputs. Since the inputs are somewhat beyond their control we cannot hold them responsible for them. E.g. if there are no good researchers available we cannot really make them responsible for having poor researchers; and if funders only provide short-term project funding we can’t really say they are responsible for their poor fundraising. But we can hold them responsible for using the researchers and the money they do have access to effectively; and this means producing the best possible outputs.
Influence is not enough; the manner in which one seeks to influence matters, too
Briefly, think tanks are not soulless machines. They are staffed and supported by people with values and these values can be judged. In my view (and this view may not be held by others) think tanks ought to seek influence in a manner that supports and strengthens their environments: e.g working with and not against or around political parties, universities, the media, NGOs, the private sector, etc.
Influencing in private, again, in my view, should not be rewarded. Engaging with others and attempting to win an argument in public, while much harder, ought to be encouraged.
But do not take my view alone. Think tanks themselves are best placed to make that call; but they will have to state their values upfront. And this is something that be can be held responsible for fulfilling.
When should (or shouldn’t) claims be made?
Inevitably, claims will be made. I am not that naive. Think tanks are under pressure from their funders to ‘demonstrate’ their influence and so many go ahead and make lots of claims regarding their influence. But there are some problems with this.
For starters, I think that think tanks that are receiving funds to develop their capacities to be more influential shouldn’t be making claims of their fantastic influence. If they claim to be influential then why would they need support? Maybe it would be more appropriate to say that they could be more influential if they were better at X, Y, Z; and then ask support for that.
Think tanks that are working to develop their research capacity should definitely not claim influence or else they would be accepting that their influence is based on poor research; and this is certainly not something a think tank would want to claim.
Those that are working to develop their communications and influencing capacities would also find themselves in a bit a pickle if they also claimed to be quite successful already; unless they want to say, publicly, that they do not need to invest in their communications capacity since they way they do it right now is good enough.
Try getting into a university programme by writing a cover letter in which you say you are perfect; if you don’t have anything to learn then why join the programme?
Claims of influence should not be made; or made very carefully. But if they have to be made then they should be made by think tanks which already have the necessary and sufficient capacity to turn their inputs into outputs.
Be careful: claims are political
Claims must be made with great care because these are not just academic exercises: they are political acts. By claiming that a policy decision is down to something I did or said, I am in fact claiming to have power over others. In essence, I am claiming, taking, power to myself.
When a think tank publishes a story of change in which it attributes that change to something it did, or when it undertakes a cost benefit analysis to calculate its return to investment like some donors have started to do themselves, it is also saying that it has power to make others do things.
In some cases this may be true, but should think tanks say so? Should they step into this role? Are they ready for the consequences and the reactions from their peers, policymakers, the media, politicians and the public?
I, for one, would demand that, if a think tank claims responsibility over a policy decision, it should be subjected to the same kind of oversight that we subject policymakers and politicians.