Comprehensive examination of the peer review process in academic medicine: Towards reaching unbiased decisions - Editorial.

•The implementation of double- or triple-blind review practices will ensure that authors with worthwhile and prominent research will have fair and equitable review regardless of their prominence in the field.•Improving the quality of our reviews and raising the standings of our publishing authors. This makes way for healthy competition and a drive to produce high quality research.•It is our responsibility to limit or eliminate bias by promoting impartiality and increasing the level of transparency between the editorial teams and authors, allowing peer review to be more inclusive, instructional, and equitable.

Background

The peer review process is at the core of the scientific enterprise, but it is not unique to this field. Athletes depend on a team of referees, gymnasts seek perfect scores from unbiased judges, and academics heavily rely on peer review to adjudicate which scholarly activity is worthy of publication. As explained in Elsevier's Peer Review “The peer review system exists to validate academic work, helps to improve the quality of published research, and increases networking possibilities within research communities.” [1] The public expects a peer-reviewed article to be a trusted piece of work that meets the standards of the academic community. In reality, there is a flaw in the peer review system - bias. We aim to provide an argument for the value in developing standardization for the peer review.

The peer review process

Studies have shown that there is evidence of reviewers displaying clear bias [2] and moreover, it has been demonstrated that lower institutional rank was associated with lower rates of recommendation from reviewers [3]. Another study by Sen-Crowe et al. assessed the relationship between the number of peer-reviewed publications in surgical journals in authors that were part of their editorial boards. They identified a distinct correlation between the number of publications by editorial team members and the number of peer-reviewed publications by their affiliated journals [4]. In addition, ad hominem biases, a bias for or against a person based on personal jealousy, friendship, or sympathy for the author's situation; can occur when a reviewer is aware of the author's identity. As mentioned by Shatz, “If a reviewer knows something about the author's work, [they] may extrapolate from previous impressions to the work being considered now.” [5] This can create discord in the community, and removes trust from what is considered the gold standard in determining scientific value.

The vast majority of researchers believe peer review is the best way to assess the scientific value of a manuscript. A survey on the perception of peer review from 2008 found that 93% disagree with the claim that peer review is unnecessary and 85% believe peer review benefits scientific communication [6]. This study was performed again in 2015, the authors reported no change in satisfaction and there continues to be broad support for peer review since their original study. Survey participants ranked improving the quality and originality of published papers as the most important functions of peer review [7].

However, reviewer bias may work against authors of different genders, minorities and from less prestigious institutions. The single blind review is the most widely used model for peer review. In single blind reviews, reviewers know the name and affiliation of the paper's author, but the reviewer's information is hidden from the scientific authors. The goal of single-blind is to prevent the author from influencing the reviewer's critique of the manuscript. This allows reviewer critique without fear of repercussion. While serving this purpose, single blind reviews do not protect authors from reviewers' bias. The single-blind review process allows bias in favor of authors from known, favored, or respected institutions, when the only issue should be scientific merit. A double blind review, where the reviewer has no information on the study authors or institution would limit this bias. An analysis of manuscripts submitted to a computer scientific society conference found that single-blind reviewers bid preferentially on papers from top universities compared to a double blind review and they gave better reviews on papers from authors and institutions of higher standing [8]. Okike et al. found that reviewers recommended acceptances at higher rates to prestigious authors and institutions when that information was known [9]. Bias that prevents the best research from being published or discussed stops science from progressing. Improving the quality of our reviews and raising the standings of our authors makes way for healthy competition and a push towards new heights for academia.

Potential solutions and future recommendations

A simple method of improvement is the widespread implementation of double-blinded reviews. Under such a process, peer reviewers do not have access to the names of the authors or the institutions from which the paper originated. In the previously mentioned Ware and Monkman survey, 56% of respondents indicated that they would prefer double-blind review (only 25% prefer single-blind). Baggs et al. found that reviewers also preferred double-blind review because they perceived that it projected objectivity and fairness, additionally studies in fields where double-blind is the norm have shown high levels of satisfaction [10]. Despite its favorability, this methodology is not widely adopted. A survey of editors-in-chief, editors, and editorial board members of 590 chemistry journals found that 97% of the journals did not offer double-blind peer review [11].

One of the reasons for this is difficulty in truly blinding a submitted manuscript, as there is the possibility of self-references in the paper. There is the possibility of a reviewer identifying an author through unique areas of study, high volume authors, or self-references in the manuscript. The instances of this should be comparably low, as the vast majority of article submissions do not fit this group [12]. Reviewers who can reasonably deduce the scientific authors would be expected to recuse themselves. An additional variation on the theme of unbiased review is triple blind review whereby, not only are authors and reviewers blind to each other's identities, but editors are also blind to the identity of the author. This is aimed at minimizing bias among editors. This could require that editors take a backseat in journalistic oversight and increase the responsibilities of their administrators. For some journals this is an impossible task as some editorial teams do no employ administrative staff.

The implementation of more unbiased review practices may not dramatically increase the quality of the review [13], but it can be helpful towards making the process more open, increasing confidence that merit is the defining criteria, and encouraging of valuable contributions to science. It is important to consider though, that many members of the scientific community believe that making reviewer identities open would increase review quality [14].

An Elsevier survey of journal reviewers participating in their pilot study of publishing the review of the manuscript along with the manuscript found that 91% of reviewers who participated stated that this open review process had no influence on their decision. Although, 70% of the journals editors said the pilot led to reports that were “more in depth and constructive for authors to improve the quality of their manuscript.” [15] Increasing transparency for reviewers and editors with a post-review analysis also helps authors understand decisions and drives quality. The British Medical Journal's editorial process includes steps to provide a decision letter and a report to the author detailing discussions of the article and the reasons for the journal's decision [16]. The report includes the entire prepublication history of the paper and allows the author to receive valuable feedback on their manuscript. Steps such as this allow the peer review process to be more inclusive, transparent, and unbiased for the author and reviewers and create a sense of partnership in publication. In the end, allowing scientific merit to be the driving force behind publications benefits all of society.

Ethical approval

Not applicable.

Sources of funding

None.

Author contribution

Study design and conception: AE.

Data collection, interpretation and analysis: AE, DF, MM.

Manuscript preparation: AE, DF, MM.

Critical revision of manuscript: AE, DF, MM.

All authors read and approved the final manuscript.

Research registration number

Name of the registry.

Unique Identifying number or registration ID:

Hyperlink to the registration (must be publicly accessible):

Not applicable-no human subjects or research participants’ data were utilized or collected.

Guarantor

Adel Elkbuli.

Declaration of competing interest

None.