{"id":1370,"date":"2013-10-07T18:35:29","date_gmt":"2013-10-07T23:35:29","guid":{"rendered":"https:\/\/onthinktanks.org\/articles\/\/"},"modified":"2016-01-20T18:37:03","modified_gmt":"2016-01-20T23:37:03","slug":"quality-control-a-few-options-for-think-tanks","status":"publish","type":"post","link":"https:\/\/onthinktanks.org\/articles\/quality-control-a-few-options-for-think-tanks\/","title":{"rendered":"Quality control: a few options for think tanks"},"content":{"rendered":"<p><em>[Editor&#8217;s note: you can read more about\u00a0<a title=\"Peer review: experimenting with think tanks\" href=\"https:\/\/onthinktanks.org\/2014\/06\/16\/peer-review-experimenting-with-think-tanks\/\">peer review systems\u00a0<\/a>for think tanks and\u00a0<a title=\"Stocktaking on Research Quality at the Think Tank Initiative Exchange\" href=\"https:\/\/onthinktanks.org\/2015\/03\/05\/stocktaking-on-research-quality-at-the-think-tank-initiative-exchange\/\" target=\"_blank\">research quality control<\/a>.]<\/em><\/p>\n<p>Last month I visited the Think Tank Fund in Hungary. It was my first time there and I got to meet the whole team (I think). The whole week turned out to be a mini festival of discussion and debate on think tanks that I hope will be repeated soon. Throughout the week we came back to a few issues and questions. One such question is the matter of research quality control (to which I would also add communications quality control, and, why not management, too? just &#8220;quality control&#8221;, then.) As a way of kicking off the conversation I thought I would share a short reflection on this subject. A few years ago I had to interview candidates for a communications manager. We asked all the candidates about quality control: what would you do to ensure the high quality of the outputs produced? We expected (and got) answers that focused on systems and processes. This particular candidate (who got the job) said: if you want good quality outputs then hire competent people. I will make sure my team is made up of competent people. There is no substitute for having good researchers, communicators and managers. All the systems and processes in the world will be useless without competence.\u00a0<a title=\"Improving the quality of a think tank\u2019s publications: Lessons from\u00a0CIPPEC\" href=\"https:\/\/onthinktanks.org\/2012\/10\/24\/improving-the-quality-of-a-think-tanks-publications-lessons-from-cippec\/\" target=\"_blank\">What can organisations do to address quality control<\/a>? In particular, what can small and resource-strapped organisations do? Here are some ideas:<\/p>\n<ul>\n<li>In-house quality control: Many think tanks rely on in-house systems to ensure that their outputs and activities fulfil some minimum standards. This demands a few things:\n<ul>\n<li>Capacity within the organisation: someone needs to have the right skills and experience to act as a quality controller. This may be more likely for research quality but communicators and managers are often on their own.<\/li>\n<li>Time to provide feedback and support to researchers, communicators and managers. It is all very well to say that something is not up to standards but what really matters is that this is followed up by advice on how to improve it.<\/li>\n<\/ul>\n<\/li>\n<li>A combination of in-house and outsourced quality control: Most think tanks would prefer to compliment their in-house quality checks with external inputs from peer reviewers or even academic boards. This can be useful for some of their outputs: e.g journal papers, working papers, and maybe for flagship reports. This is appropriate but more expensive and think tanks need to make sure that they budget accordingly. Also, external reviewers demand that think tanks have the capacity to manage the process as it is very likely that they will take longer than expected to deliver. And if there is one thing that think tanks cannot afford is to miss their deadlines. Missing a deadline can mean missing a one hour opportunity window; years of work down the drain.<\/li>\n<\/ul>\n<p>The following ideas are a bit more experimental. I am just putting them out there to see if there is any reaction to them:<\/p>\n<ul>\n<li>Peer-to-peer peer review (think tank level): Think tanks in a country or in a region could come together to offer each other peer review support. In a country or region, think tank A could offer think tank B peer review support -in exchange for the reciprocal service. This is actually how it works in many developed countries so I am always surprised about claims that think tanks would be turned off by fear that their competitors would steal their ideas. More likely, opposition to this idea has to do with embarrassment. For this to work, however, there has to be a string relationship of trust between the think tanks and the researchers involved.<\/li>\n<li>Peer-to-peer peer review (researcher level): I think this is more likely to work as researchers tend to know each other and often support each other&#8217;s work -even if they compete for funding and access. Think tanks communities in developing countries are too small for researchers to afford not being nice to each other. Much harder to find is the support that directors and managers could get from their peers in the think tank community.\u00a0<a title=\"Supporting think tanks series: Economic Policy Research Institutes in Sub-Saharan\u00a0Africa\" href=\"https:\/\/onthinktanks.org\/2013\/06\/10\/supporting-think-tanks-series-economic-policy-research-institutes-in-sub-saharan-africa\/\" target=\"_blank\">Few directors and managers in fact ever think about their work<\/a>\u00a0or would consider asking for help.<\/li>\n<li>Learn from others: An alternative to asking for advice is to search for it. All think tanks have access to other think tanks&#8217; work (locally and internationally). It should be possible for a researcher or communicator to know if his or her work is up to standard. Managers may have to seek out peers more directly, but this is not impossible. In all cases, this approach demands honesty and humbleness. Some ways to do this include:\n<ul>\n<li>Find a few ideal think tanks (domestic and international) and compare the quality of their work to your own. Place a paper on housing policy by NIESER or Brookings (or a local think tank) next to a paper on housing policy by your think tanks and compare. You may not be able to be as &#8216;good&#8217; but you can certainly strive to and move in the right direction.<\/li>\n<li>Watch or attend the events, keep an eye on the best publication designs, websites and use of digital tools of the most progressive and active think tanks. Think tanks always know who are &#8216;very good at communications&#8217; so it should not be hard for them find some to compare their own work to.<\/li>\n<li>Funders could (but think tanks should) support the formation of a culture of public events that maximise the opportunities of think tanks in a city to come together and exchange ideas, &#8216;spy&#8217; on each other, build relationships, etc. These events (like the ones that any think tanker can attend in London or Washington on a daily basis) provide an invaluable opportunity to learn from each other.<\/li>\n<\/ul>\n<\/li>\n<li>Practice, practice, practice (repeat, repeat, repeat): I have written about<a title=\"A Zambian think tank start-up: a possible\u00a0model\" href=\"https:\/\/onthinktanks.org\/2013\/03\/13\/a-zambian-think-tank-start-up-a-possible-model\/\" target=\"_blank\">\u00a0PMRC in Zambia<\/a>. They have a great range of publications and other communication tools developed over a year of intensive practice. The model we developed together was simple: a few outputs that would be used on a monthly basis for every project. By repeating the same outputs over and over again they would get to practice and continuously improve their quality. Each time, too, they were able to introduce a new\u00a0<a title=\"Communication options for think tanks: channels and\u00a0tools\" href=\"https:\/\/onthinktanks.org\/2012\/08\/20\/communication-options-for-think-tanks-channels-and-tools\/\" target=\"_blank\">channel or tool<\/a>. The best way to learn how to do research and write a good paper is to practice doing research and writing papers. Young researchers need practice. Too many think tanks I have come across favour long term research projects in which their younger staff have little or nothing to do when instead they could be:\n<ul>\n<li>Writing literature reviews (and annotated bibliographies): This can help to give the researchers a good understanding of several issues, they can help with future research (by them, their bosses or other researchers), they are communication tools in their own right, and can help them to learn from others (by reading the work of others). Any down time in a young researcher&#8217;s week should be focused on something like this.<\/li>\n<li>Play with databases: When I started working in research about 15 years ago my colleagues and I used to spend quite a bit of our time putting together cross-tabs and running regressions with data bases just for the fun of it. And I mean for the fun of it: what is the relationship between education of the mother and education of the father per quintile? what is the relationship between years of education for urban adults and size of the household? We had no use for these but it helped us to learn how to use STATA and how to ask questions.<\/li>\n<li>Allow communicators and young researchers to develop\u00a0<a href=\"http:\/\/ttdatavis.onthinktanks.org\" target=\"_blank\">data visualisations<\/a>, manage personal or organisational Twitter handles, organise short 1-2 hour lunch time events, etc. These things do not cost much (I recently organised a couple of events in Peru with local think tanks that cost peanuts) and even tiny events for a few people can allow communicators to practice and get better at things for those big and important moments.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>[Editor&#8217;s note: you can read more about\u00a0peer review systems\u00a0for think tanks and\u00a0research quality control.] Last month I visited the Think Tank Fund in Hungary. It was my first time there and I got to meet the whole team (I think). The whole week turned out to be a mini festival of discussion and debate on [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_relevanssi_hide_post":"","_relevanssi_hide_content":"","_relevanssi_pin_for_all":"","_relevanssi_pin_keywords":"","_relevanssi_unpin_keywords":"","_relevanssi_related_keywords":"","_relevanssi_related_include_ids":"","_relevanssi_related_exclude_ids":"","_relevanssi_related_no_append":"","_relevanssi_related_not_related":"","_relevanssi_related_posts":"","_relevanssi_noindex_reason":"","footnotes":""},"tags":[215,313],"acf":[],"_links":{"self":[{"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/posts\/1370"}],"collection":[{"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/comments?post=1370"}],"version-history":[{"count":0,"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/posts\/1370\/revisions"}],"wp:attachment":[{"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/media?parent=1370"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/tags?post=1370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}