Think Tank Effectiveness – An outsider view

1 December 2014
SERIES Think tanks: definition and terminology 17 items

[Editor’s note: this post was written by Neeta Krishna, Associate Professor, Father C Rodrigues Institute of Management Studies, Navi Mumbai. It is part of a series of articles commissioned by the Aditi project at CSTEP which aims to capture information/ anecdotes/data from various Think Tanks. The main objective of Aditi is to discuss institutional issues of common interest such as funding, human resources, research capacity, institutional issues of policy research etc.] 

This article discusses several effectiveness issues of Think Tanks focusing on Indian Think-Tanks. It does not present a theory or model of effectiveness, nor does it suggest a measurement framework.

In a market economy a business organization’s profit is a pretty good measure of its performance over time. If a company makes a good profit year after year it suggests that its products provide value to customers who are ready to pay more for them than the cost of resources used by the company; that the company manages its business quite well, and is performing efficiently and effectively.

Think Tanks are largely ‘non-profit’ organizations. Aimed as they are at better informed public policy they lack a convenient easy-to-measure variable like profit, to gauge their performance. Even if their recommendations are not accepted by policy makers, they increase the number of options brought to the table or raise the quality of the policy debate, that they have achieved something. Therefore getting an assessment of their effectiveness and value created by them can confound stake-holders, including leaders and funders.

So what makes a Think Tank effective? Is it specific ways of functioning and attributes? Or end results achieved? This article picks and mixes some ideas from The Think Tanks & Civil Societies Program’s 2013ranking of the world’s Think Tanks (2013 GLOBAL GO TO THINK TANK INDEX) and from Think Tank Initiative (TTI) a multi-donor program offering funding and other support to promising Think Tanks in developing countries.

The 2013 Think Tank index suggests the following indicators to assess a Think Tank’s impact.

  • Resource Indicators – ability to obtain human and financial resources and proximity to policy makers;
  • Utilization Indicators – reputation with media and policy influencers,
  • Output indicators – policy proposals, ideas, publications, conferences, briefings etc.
  • Impact indicators – primarily reflects the degree to which a Think Tank’s work has actually influenced policy.

TTI selects ‘promising’ Think Tanks that have a good track record, independence and transparency in functioning, commitment to using research, and a national level focus on broad social and economic policy issues. A 2013 report evaluating the TTI program indicates the following as some improvement areas for these Think Tanks: research quality, policy linkages and overall organizational performance.

Borrowing from these ideas this article will discuss some important key attributes and outputs and outcomes of effective Think Tanks.

Attributes of effective think tanks

The following can be identified as attributes of effective think tanks:

Capable: Policy research organizations must have organizational capability to produce relevant and reliable research. They require leaders with vision, committed both to their domains and to producing high quality evidence based knowledge for policy and social change. They also require qualified and competent researchers to understand policy imperatives for research, frame problems, dialogue with policy actors, undertake robust research and do something fruitful with their research to produce positive policy outcomes.

Independent: Most Think Tanks claim to be ‘independent’ implying that they are free to pursue research in the policy area of their choice, to choose the methods, scale and scope of the research, to communicate research findings and recommendations, advocate policy options, and to allocate resources without approvals from outside agencies including governments.

Absolute independence is unrealistic and impractical. Think Tanks will depend on funders and, perhaps, policy makers and implementers.   A little dependence may be a good thing as it could shape more relevant policy research agendas, secure allocation of resources for more worthy research causes. It becomes a matter of concern, however, when the dependence rises to a degree that research, analysis, views and recommendations are tailored to meet the expectation of the agency the Think Tank depends on.

  • Funding & Independence: A major source of dependence is funding, discussed later in this article. An institution completely dependent on a few donors or paying ‘customers’ for its research, will feel the pressure to meet the expectations of these stake-holders. This may take the form of bowing down to overt pressure by the funder or client or, perhaps even ‘self-censorship’ by the researcher, which could be a bigger issue. A newspaper writes about a Government of India Ministry which chose not to publish a report on a matter it had asked a Delhi based Think Tank to study, apparently because the study report supported the views of another government department with which the client Ministry was feuding. The article claims that this is one of many such incidents of the cost of giving unpopular advice.

Transparent – Within and Outside: Institutions that are transparent hold their activities and methods open to scrutiny and questioning, and thereby provide a more inclusive basis for change and improvement. Transparency in functioning implies that information is shared within the Think Tank, including basis for decisions on which projects to pursue, allocation of resources and work, reward, open discussion on what went well and what did not, and so on. It could also imply that the Think Tank shares more information in the public domain.

Some elements of transparency are embodied in the research process itself. Think Tanks hold conferences to refine and shape research agendas at the start of a project, and to disseminate findings at the end of a project. Questions are raised, research defended, perhaps modified or changed. Reports and policy briefs are often freely available on their web-sites, some with a considerable amount of detail on the methodology and findings.

Some Think Tanks like the Centre for Policy Research (CPR) and  NCAER (both from India) make their financial statements available to the public, permitting a public scrutiny of where they receive funds from and how they use them. They also publish very detailed reports of their activities.

Outputs and outcomes

Research Quality: The final, but difficult to measure, test of high quality Policy Research is the degree to which it enables a Think Tank to execute its vision or purpose. Easier to define, however, are some important criteria with regard to a Think Tank’s research, making it more likely that the research will result in better policy.

  • Policy Relevance: Research by a Think Tank should be relevant for policy issues and public engagement in the national and perhaps, international development agenda. The domain of public policy is huge, but in every nation/society some issues emerge as priorities. The best Think Tanks pick up issues of salience; perhaps those inadequately served by other organizations; some even out of the public eye or treading on delicate issues, upsetting powerful interest groups.

In an emerging economy like India, trying to extract a demographic dividend from its immense population, equitable growth is an area of concern.

  • Reliability & Credibility: Research might be relevant, but is the information it provides reliable and valid? Raghuram Rajan, Governor of the Reserve Bank of India speaks about ‘clean, trustworthy studies’ in connection with India’s NREGA (National Rural Employment Guarantee Act) , hinting, therefore that research by research institutions may sometimes be inferior, and so not of much use to policy makers.
  • Interdisciplinary Responses: Since public issues cut across different policy areas and a particular public policy could affect people in different ways how do Think tanks ensure that their research can provide an interdisciplinary response to complex issues?

Large Think Tanks with wider policy horizons would have specialists from different disciplines to provide a greater interdisciplinary perspective. Their challenge would be to get specialists from different disciplines to provide a specialist perspective, while integrating efforts across disciplines to execute the research and generate solutions. Yet even these large institutions might choose to work in policy areas, for example policies related to Climate Change, requiring solutions often beyond the capability of a single institution. For smaller Think Tanks with fewer members the challenge is greater.

Policy linkage, impact, and influence

According to Donald Abelson, Professor of Political Science at University of Western Ontario, Canada, many Think Tanks in the U.S. are actively involved in the policy-making process – participating both directly and indirectly in policy-making as policy-makers to turn to them for policy advice. On the other hand several Think Tanks in many parts of the world are quite removed from the policy making process. A paper by IFPRI (Nigeria) claims poor research-policy linkages in Nigeria, and assigns reasons as:

  • Lack of high-quality research – despite the presence of many Policy research institutions.
  • Lack of interaction between policymakers and researchers, so research findings don’t get linked to policy related problems.
  • Inadequate use of the information for policy making by policy makers.

So what does it take to achieve policy influence at different stages of the policy making cycle? It takes a strong research base, reaching out to all actors relevant for the policy at each stage of the research process – before, during and post-research. The Centre for Civil Society for instance, reaches out to both policy makers and beneficiaries of its School Vouchers agenda.

Some other factors that strengthen policy linkage, impact and influence are discussed below.

Communication: The output of Think Tanks is largely knowledge and ideas. It is useful only if espoused or at least heard as a viewpoint by policy makers and society.   Unless well communicated in a manner which conveys the facts, recommendations and ideas, using the right channels, to policy makers and society, the most robust research and brilliant idea is not of use.

Visibility: High visibility in public debate in relevant policy areas and themes has been cited as a feature of strong policy linkages. Visibility in the media is one example: In just the three days, between July 29 & 31, 2014, articles on current and engaging themes by CPR’s Faculty and Researchers appeared in the prestigious centre pages in two of India’s leading dailies.

Visibility takes other forms including speaking engagements at high profile conferences, seminars, talks; appearances as experts on television talk-shows, even showing up on top in a search engine search on the internet!

Mass Media & Public Opinion The public and sometimes even policy makers learn about policy issues through the news media. Also, policy makers are often influenced by public opinion, and public opinion can be influenced by coverage in the news media. Clearly, therefore, the mass media plays a role in shaping a Think Tanks’ influence so Think Tanks must pay attention to their appearance in and coverage by mass media.

Some Think Tanks, like Centre for Civil Society (CCS) enlist volunteers from members of civil society thereby getting high quality manpower at no significant cost, and shaping public opinion as citizen volunteers get engaged with and speak to friends and family about their work and the institution.

Credibility & Reputation How well a Think Tank’s work is regarded by policy makers and policy relevant actors? Do they seek its inputs? Does it have a reputation for sound research, insightful analysis and thoughtful and creative suggestions? Does it have the best and most highly regarded researchers on its team?

Linkages with Policy Makers: Credibility, visibility, good communication can serve to facilitate a Think Tank’s links with policy makers. These links are important at all stages of the policy research process.

Many Indian Think Tanks have been founded or are led by people who have served as senior government officials. Several senior government functionaries, including the current National Security Advisor have been associated with a New Delhi based Think Tank. (This is unusual; it is more customary for former government officials to join Think Tanks). Either way it provides access between Think Tanks and government based on personal networks, reputation and understanding of needs and mindsets.

Networking with & Developing Other Institutions Since public policy is bound to require interdisciplinary responses, sometimes across geographies, Think Tanks which are able and collaborate with other good Think Tanks could increase their ability to produce more significant work. The Track 2 diplomacy initiative mentioned in Part II in this series, involving the collaboration of one Indian and one Pakistani Think Tank, is an example.

Policy Outcomes: The proof of the pudding, eventually, is in the eating. And Think Tanks which are able to influence policy and social change outcomes can be said to have achieved their purpose if such change is what they seek. CCS’ 2012-13 report on its Jeevika Campaign in support of street vendors’ right to livelihood suggests that its efforts yielded fruit in the form of laws being passed in two states which were part of their campaign.

Other elements of organisational performance

If Think Tanks are to achieve their purpose they need to secure and effectively manage resources. Two key resources – financial and human, are discussed here. The financial information reported here has been obtained from the published annual statements for the year 2012-13.

Securing Finance: Think Tanks must secure funds to pay for their activities. India ranks fourth worldwide, in terms of the number of Think Tanks in a country (Global Think Tank Survey 2013). Most of these require funding. Funds may be sought from domestic and international funding agencies including the government. Many of the best Think Tanks receive considerable funding from funders and government agencies. It is, however, a competitive arena and government funding seems to be shrinking.

Think Tanks which have diverse sources of funding face a lower risk in case a funder withdraws support, making it less dependent on a particular source of funds.

To reduce dependence on short-term flow of funds, some Think Tanks have built a substantial corpus which is invested and provides a regular stream of income that can be used for long term institution building activities. CPR’s   corpus on March 31, 2013 was more than three times the year’s expenditure. Earnings from the corpus alone, if prudently invested, could cover about a fourth of its expenses. NCAER (India), a larger Think Tank had a corpus amounting to around fifteen months’ expenditure and secure earnings on it would be likely to cover a smaller proportion of expenditure.

Indian Think Tanks have been quite unsuccessful (with a few exceptions) in raising funds from Indian business which has donated millions of dollars to American universities and more recently to American Think Tanks seeking to set up base in India. According to a report, one such foreign Think Tank was able to raise an amount from Indian business which is approximately equal to the corpus of the top three Indian Think Tanks.

Why doesn’t Indian business support Indian Think Tanks? Opinions vary; some reflecting attitudes of expediency in Indian business people; others take the view that Indian Think Tanks do not adequately cater to the interests and concerns of Indian business.

There is, however, a new breed of right thinking Indians with the wealth to provide funds and the will to support good policy research organizations that can help usher in a better future through evidence based public policy.

Effective management of resources

Utilization of Financial Resources: Ray Struyk, author of ‘Managing Think Tanks’ who has worked with dozens of Think Tanks believes that very few Think Tanks do a good job of Financial Management by ensuring that resources go where they are intended, costs are controlled, allocation of overheads is fair. He suggests that cost information should be provided to Think Tank Managers in a timely manner, there should be detailed budgeting for the institution’s activities, a system for tracking and allocating expenses and keeping track of manpower utilization, since labor is typically 70 % of cost.

Fund Management a quick glance at a leading Indian Think Tank’s published income statement for 2012-13 indicates that It was holding a huge cash balance (around six months’ operating expenses) on the last day of the financial year. Unless this is a chance occurrence these funds could have been put to more productive use either earning higher interest or perhaps supporting a higher level of activity. Cash apart, its unspent funds did not seem to be earning as much as they could have, though more information is required to conclude this. The point is that Think tanks might not be doing enough thinking on managing money and therefore not making the best use of their financial resources.

Managing Human Resources: The ability to attract, develop, motivate and retain talent is a critical success factor for Think Tank effectiveness. Think Tanks require top intellectual talent, and talent is their only real asset. Such talent, according to the head of a research institution in India, operates in a global market, making it difficult for Indian Think Tanks with funding constraints and possible anchoring to ‘University Grants Commission pay scales’ to attract and retain the kind of talent they require, settling perhaps, for what is available instead.

On think tank staff motivation and engagement

Ray Struyk offers several suggestions. A key to staff motivation is providing innovative and policy relevant work, an enriched job content and chance for growth. This is easy to prescribe and less easy to implement. It means, at the least, that the institution should have a flexible but robust performance assessment system that aligns individual and organizational goals and performance. It requires that senior Think Tank staff have goal setting, performance review and developmental conversations with the junior staff in their charge.

A word for the future

Think Tanks are organizations, which like biological systems grow and evolve. They may do this well or badly; a Think Tank may thrive, grow and increase its sphere of influence or struggle and become irrelevant, perhaps perish. The best Think Tanks in the world have obviously managed this well, starting relatively small. In South Asia, Institute of Governance Studies (IGS) in Bangladesh has described its transition from a higher education and academic research organization to a more policy-engaged Think Tank. India has a number of promising and performing Think Tanks, most of which are quite small. Investments by their leaders today in shaping the growth and evolution of their institutions in the future, will yield rich dividends to the Think Tank’s members and employees and   build an invaluable asset for the nation and its people.