{"id":1642,"date":"2013-11-26T13:02:41","date_gmt":"2013-11-26T18:02:41","guid":{"rendered":"https:\/\/onthinktanks.org\/articles\/\/"},"modified":"2016-01-23T13:04:18","modified_gmt":"2016-01-23T18:04:18","slug":"this-year-instead-of-ranking-think-tanks-lets-think-about-them-more-carefully","status":"publish","type":"post","link":"https:\/\/onthinktanks.org\/articles\/this-year-instead-of-ranking-think-tanks-lets-think-about-them-more-carefully\/","title":{"rendered":"This year, instead of ranking think tanks lets think about them more carefully"},"content":{"rendered":"

David Roodman<\/a>\u00a0and\u00a0Julia Clark<\/a>\u00a0from the Center for Global Development have posted a very interesting reflection on the now unfortunately famous McGann think tank ranking. In\u00a0Envelope, Please: Seeking a New Way to Rank Think Tanks<\/a>, they offer an alternative to a global ranking exercise.<\/p>\n

By now most people working in or for think tanks must have received a few (more than a few, actually) email reminders to submit nominations for the ranking. I have even heard of \u00a0emails ‘threatening’ a fall in the rankings for those think tanks not willing to participate. But this is still no more that rumour.<\/p>\n

Criticism to the ranking on this blog has been rather consistent (See:\u00a0on\u00a0rankings<\/a>,\u00a0Another year, another ranking of think tanks (and surprise surprise, Brookings is still the\u00a0best)<\/a>,\u00a0Goran\u2019s recommendations on think tank\u00a0rankings<\/a>,\u00a0The mighty influence of think tanks (in the\u00a0US)<\/a>,\u00a0And the winner is: Brookings \u2026 but, once again, the loser: critical\u00a0analysis<\/a>, and\u00a0The Go to Think Tank Index: two\u00a0critiques<\/a>). Even before, when I was working at ODI, I felt that the effort put into this ranking exercise could make a more important contribution elsewhere (I should stress that McGann’s expertise on think tanks is not under question here: in fact I wish he used his time and the time of his many assistants more productively). The process, and the ranking itself, is in my view (and that of others)\u00a0inherently flawed<\/a>:\u00a0It confuses visibility with influence on the substance of policy and politics.<\/p>\n

Roodman and Clark offer an alternative: not a ranking but an exercise in attempting to measure the aspects of think tanks and their actions that can be measured. This may considered by some as being too cautious; but their caution is based on experience:<\/p>\n

Our experience with\u00a0building policy indexes<\/a>\u00a0such as the\u00a0Commitment to Development Index<\/a>makes us keenly aware of the limitations of any such exercise. Think tank profile is not think tank impact. Fundamentally, success is hard to quantify because think tanks aim to shift the thinking of communities. But the operative question is not whether we can achieve perfection. It is whether the status quo can be improved upon. Seemingly, it can be: it only took us a few weeks to choose metrics and gather data, and thus produce additional, meaningful information.<\/p><\/blockquote>\n

And this meaningful information has provided ample opportunities for a meaningful discussion. So what the McGann ranking has failed to do year after year, Roodman’s and Clark’s exercise has managed in a single post. The authors identify four key methodological issue that could open up several lines of very interesting reflection (I quote in full to encourage others to engage with their own reflections and maybe suggest alternative solutions to the challenges they faced):<\/p>\n

\n