{"id":920,"date":"2014-06-23T21:22:21","date_gmt":"2014-06-23T21:22:21","guid":{"rendered":"https:\/\/onthinktanks.org\/?p=920"},"modified":"2016-01-06T18:09:10","modified_gmt":"2016-01-06T18:09:10","slug":"what-are-peer-review-systems","status":"publish","type":"post","link":"https:\/\/onthinktanks.org\/articles\/what-are-peer-review-systems\/","title":{"rendered":"What are peer review systems?"},"content":{"rendered":"<p><em>[Editor\u2019s note: This is the third\u00a0of a series of posts on\u00a0<a href=\"http:\/\/wp.me\/pYCOD-1wM\" target=\"_blank\">a peer review mechanism pilot for think tanks<\/a>. It has been edited by Andrea Ordo\u00f1ez as part of the\u00a0<a title=\"On Think Tanks is looking for guest editors (a pilot)\" href=\"https:\/\/onthinktanks.org\/2013\/10\/18\/on-think-tanks-is-looking-for-guest-editors-a-pilot\/\" target=\"_blank\">Guest Editor initiative\u00a0<\/a>launched by On Think Tanks last year. If you are interested in being a Guest Editor please get in touch.]<\/em><\/p>\n<p>As shared in the\u00a0<a href=\"http:\/\/wp.me\/pYCOD-1wM\" target=\"_blank\">introductory post<\/a>, the principle of peer review is simple. Its implementation, however, is not at all that straightforward (Review an\u00a0<a href=\"http:\/\/cdn.knightlab.com\/libs\/timeline\/latest\/embed\/index.html?source=0AsQO8QeiwHnEdGFjanFZR18tWEhDOVB0Wk5SZmlqaHc&amp;font=Bevan-PotanoSans&amp;maptype=toner&amp;lang=en&amp;height=650\">interesting timeline of the concept<\/a>).<\/p>\n<h3>Objectives and limitations<\/h3>\n<p>The first critical question is why we carry out peer review processes. There is not one answer for this, so let\u2019s explore some of them.<\/p>\n<p><strong>Gatekeeping<\/strong>\u00a0\u2013 By filtering and screening material, the peer review system is the way in which journals have traditionally decided on what to publish or what not to publish. The reviewers usually face the following recommendation choices:<\/p>\n<ul>\n<li>to unconditionally accept the manuscript or the proposal,<\/li>\n<li>to accept it in the event that its authors improve it in certain ways,<\/li>\n<li>to reject it, but encourage revision and invite re-submission,<\/li>\n<li>to reject it outright.<\/li>\n<\/ul>\n<p>Peer reviews also have a place in grant making processes and academic conferences presentations. At the end of the day, the objective is to distribute limited space, time and resources: not everybody can get funding, get published or present at conferences.<\/p>\n<p><strong>Maintain reputation<\/strong>\u00a0\u2013 It has been argued\u00a0<a href=\"https:\/\/royalsociety.org\/events\/2014\/reinventing-science-publishing\/\">that peer review processes were popularized by the Royal Society to protect its reputation<\/a>. Reputation is thus based on the quality of those involved in the review process, which is why being part of a journal editorial board is a highly regarded position among researchers. Furthermore, the decision of publishing or not is not only based on the editor\u2019s judgment, but includes that of others.<\/p>\n<p><strong>Quality control<\/strong>\u00a0\u2013 Peer review processes are supposed to ensure that research papers are consistent: that an appropriate methodology is used, that conclusions presented are backed by the research undertaken, and that other relevant knowledge on the subject is correctly acknowledge (basically, a good literature review).<\/p>\n<p><strong>Capacity development<\/strong>\u00a0\u2013 although this aspect is not the main concern of the world of publishing, I believe that receiving input about one\u2019s work is a valuable opportunity to improve one\u2019s work. This usually requires a capacity to interpret and critically analyse comments holistically. Many think tanks I know introduce peer review as a mechanism to support researchers, even when peer review is not used for the previous purposes.<\/p>\n<p>Although the peer review process is a keystone in the academic world, it is not perfect. At the end of the day, the responsibility of the research paper is the authors\u2019. Fraud and plagiarism cannot be detected by a reviewer. A peer review is no guarantee of the validity of the research presented; that responsibility lies with the authors. In fact,\u00a0<a href=\"http:\/\/www.nature.com\/news\/publishers-withdraw-more-than-120-gibberish-papers-1.14763\">Springer recently retracted 120 papers that were actually computer-generated nonsense<\/a>. Authors can act unethically, and reviewers may not have enough information to judge those actions.<\/p>\n<h3>Criticisms of the peer review processes<\/h3>\n<ul>\n<li>Some have argued that peer review processes are biased. Researchers that cite the right authors, that maintain conservative views and follow traditional methods might get more published, some believe. It is hard to assess this, as the peer review documents are usually not public.<\/li>\n<li>Santiago Basabe recently wrote a\u00a0<a href=\"http:\/\/www.patiodesociales.com\/2013\/10\/publicando-en-revistas-cientificas-los.html\">great reflection on the bias on \u2018good data\u2019<\/a>. This particularly affects countries with deficient information systems. As a result, countries with better data are more studied than those with poor data. Although we want more knowledge of these countries, this bias reduces the interest of researchers, who see their chances of getting published diminished even if their work is methodologically sound with the information currently available.<\/li>\n<li>The way peer review processes are carried out now delays publication significantly. This, some argue, is detrimental to scientific inquiry and it may have an impact on the diffusion of such knowledge, as authors\u00a0<a href=\"http:\/\/www.politicsandideas.org\/?p=1632\">move quickly into new topics<\/a>.<\/li>\n<li>Peer review is highly connected to the \u201cpublish or perish\u201d paradigm of modern science. Are the high expectations on publishing responsible for the growing manipulation of peer review processes? These high expectations on publishing are pushing the limits of the system. Researchers are encouraged to create various papers out a research project and longer term processes are discouraged. As\u00a0<a href=\"http:\/\/www.theguardian.com\/science\/2013\/dec\/06\/peter-higgs-boson-academic-system?utm_content=buffer4bb80&amp;utm_medium=social&amp;utm_source=twitter.com&amp;utm_campaign=buffer\">Peter Higgs argues<\/a>, today he might not have the space to do the work that involved studying the Higgs boson in the sixties.<\/li>\n<\/ul>\n<p>This brief overview of the peer review processes shows how this simple concept, in practice, gets quite complex. This is not meant to discourage researchers from participating in such endeavours but to do so with a critical perspective.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Andrea Ordo\u00f1ez writes about a new peer review mechanism for think tanks that the Think Tank initiative has piloted recently. She has produced a series of posts that explore the approach, consider possible lessons learned, and outlines recommendations for future efforts. In this post, Andrea discusses peer review systems and the particular challenges faced in the implementation of the Think Tank Initiative&#8217;s Peer Review System.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_relevanssi_hide_post":"","_relevanssi_hide_content":"","_relevanssi_pin_for_all":"","_relevanssi_pin_keywords":"","_relevanssi_unpin_keywords":"","_relevanssi_related_keywords":"","_relevanssi_related_include_ids":"","_relevanssi_related_exclude_ids":"","_relevanssi_related_no_append":"","_relevanssi_related_not_related":"","_relevanssi_related_posts":"","_relevanssi_noindex_reason":"","footnotes":""},"tags":[182],"acf":[],"_links":{"self":[{"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/posts\/920"}],"collection":[{"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/comments?post=920"}],"version-history":[{"count":0,"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/posts\/920\/revisions"}],"wp:attachment":[{"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/media?parent=920"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/onthinktanks.org\/wp-json\/wp\/v2\/tags?post=920"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}