Literature DB >> 34909739

JID Innovations and Peer Review.

Russell P Hall.   

Abstract

Entities:  

Year:  2021        PMID: 34909739      PMCID: PMC8659384          DOI: 10.1016/j.xjidi.2021.100056

Source DB:  PubMed          Journal:  JID Innov        ISSN: 2667-0267


× No keyword cloud information.
JID Innovations was launched in late 2020 with the goal of providing open access publication of high-quality science focused on skin biology and disease. The journal is part of a movement to help ensure that science is performed with integrity, reported accurately and openly, and available without barriers to all readers, including scientists and the public. Although the challenges of providing barrier-free access to the scientific literature are relatively straightforward, the process of ensuring high quality is often much more elusive. Although external peer review is a long-standing and critical practice in science, it continues to be subject to criticism (Csiszar, 2016; Horrobin, 2001; Smith, 2006). It is helpful to understand the history of how the practice of peer review achieved its current status, as we think about whether and how peer review can be improved in the era of open access, open science and in JID Innovations. In 1620, Sir Francis Bacon described “True suggestions for the interpretation of nature” (Novum Organum) (Bacon, 1620). Bacon proposed a methodology for assessing science that included a full knowledge of the facts, healthy skepticism, and a recognition of the issue of bias in the evaluation of facts. This work led to the formation of the Royal Society of London and ultimately to the development of the Philosophical Transactions of the Royal Society in 1665 (Spier, 2002). This journal was dedicated to the natural sciences and was the vehicle for members of the Royal Society to report the “studies and labours of the ingenious in many considerable parts of the world.” These transactions were first edited by Henry Oldenburg (https://royalsocietypublishing.org/; accessed 23 August 2021). Although Oldenburg is often cited as the founder of peer review, his efforts were more directed at collecting comments of Royal Society members rather than at critical prepublication review (Baldwin, 2018). The Royal Society of Edinburgh is thought to have been one of the first to institute a more formal evaluation system in 1731. This was documented with the publication of an editorial policy stating, “Memoirs sent by correspondence are distributed according to the subject matter to those members who are most versed in these matters. The report of their identity is not known to the author. Nothing is printed in this review which is not stamped with the mark of utility.” (Kronick, 1990). During this period, scientific journals were predominantly the product of scientific and professional societies. These societies and, by association, their members were protective of their reputations and actively assigned committees comprised of society members to determine the suitability of a material for publication (Kronick, 1990). However, it is of interest that concerns about the effectiveness of determining the suitability for publication were already known. Disclaimers regarding their publications were made by the societies that published journals. The Literary and Philosophical Society of Manchester stated, “Responsibility concerning the truth of facts, the soundness of reasoning, in the accuracy of calculations is wholly disclaimed: and must rest alone, on the knowledge, judgement or ability of the authors….” (Kronick, 1990). Throughout the 19th and early 20th centuries, peer review continued to evolve. Journal editors used anonymous reviewers, often selecting the most elite scientist for this task. These reviewers ultimately served as gatekeepers to protect the quality of the scientific literature and the reputation of their professional society (Baldwin, 2018). These anonymous reviewers were most often also members of the society publishing the journal and/or the editorial boards and made all decisions on publications. It is important to note that even this internal type of review was not universal, nor was it always welcomed. One famous anecdote is attributed to Albert Einstein who in 1936 was incensed to find that his article had been sent to another scientist (reportedly on the Editorial Board) for evaluation without his permission. Einstein wrote that he saw “… no reason to address the – in any case-erroneous – comments of your anonymous expert.” (Baldwin, 2018). It was not until the mid-20th century that individuals outside of the journal’s editorial staff were asked to evaluate manuscripts. This change was in part attributed to the increasing workload for editorial boards and editors and the need to publish findings as quickly as possible (Baldwin, 2018). The New England Journal of Medicine starting using two external reviews for all submissions in the 1960s, and Nature only began using external referees on all papers in 1973 (Baldwin, 2018; Ingelfinger, 1977). Since that time, it has become usual practice for editors and editorial boards of scientific journals to use multiple external, anonymous reviewers to assist in selecting articles for publication. It is also of interest that it was during this transition that the term peer review was first utilized, suggesting that this review process could only be done by experts who were expected to be peers of the authors (Baldwin, 2018). Baldwin (2018) also proposed that the concept of peer review was further established in our scientific culture by its utilization by funding organizations such as the National Institutes of Health and the National Science Foundation. In the 1960s and the 1970s, the United States Congressional inquiries were held focusing on the type and quality of grants awarded by the National Institutes of Health and National Science Foundation and the management of those grants. The concept of peer review by experts outside of the funding agencies was accepted by Congress and the scientific community as “the only acceptable method of evaluating scientific quality” (Baldwin, 2018). This long history provides the background for our current system of peer review, its limitations, and how we can work to improve it. It is worthwhile to consider how we currently define peer review and what we expect it to accomplish. Peer review is defined most simply as the process whereby scientific work is subjected to critical evaluation by individuals, judged to be experts, to assist in the determination of the work’s suitability for publication (Hames, 2014; Kelly et al., 2014). What is expected from peer review is much more difficult to define and often differs between scientists and laymen. Issues that peer review has been expected to address include detecting fraud in manuscripts, identifying so-called bad science, assessing the long-term impact of observations, assessing technical details and methodology, and improving the communication of results and writing style. The problems that have been associated with peer review are likewise numerous. Specific criticisms associated with peer review include difficulty in identifying the appropriate peers to serve as qualified reviewers, stifling innovation, suffering from overwhelming bias and conflicts of interest between authors and reviewers, tolerating inconsistent results among reviewers, and being slow and expensive (Horrobin, 1990; Rothwell and Martyn, 2000; Scharschmidt et al., 1994; Smith, 2006; Tennant and Ross-Hellauer, 2020; Weissmann, 2006). It is not surprising that peer review is sometimes viewed suspiciously by both the public and the scientific community. Many of the expectations for peer review have been repeatedly shown to fall short. Peer review has been shown to be ineffective at detecting fraud and errors and blind to the significance of innovative studies (Suk, 2006; Weissmann, 2006). Consistency between peer reviewers is poor, and confirmatory bias has been shown to significantly impact peer review (Mahoney, 1977; Rothwell and Martyn, 2000). In addition, the peer-review process is by its nature not transparent to the reader. Often, the reviewers are anonymous to both the authors and the readers, and the impact of peer reviewers on published manuscripts is unknown to the readers. With all the drawbacks of the current peer-review process, there are many opportunities to improve it. Technology has recently advanced to help in some aspects of review. Computer programs are utilized by many journals to help identify plagiarism and fraudulent images and figures. Editors try to ask specific questions of reviewers and develop templated documents to focus on reviewers’ comments. Reviewer training has been proposed to improve peer review; however, there has been only modest to no positive impact seen on the quality of reviews (Schroter et al., 2008, 2004). Despite these and many other attempts to improve peer review, the process remains predominantly closed to the author and the reader. Persistent problems with bias, concerns about competitors (who are after all peers) stifling or co-opting others’ works, inaccurate reviews, and sometimes rude and inappropriate comments remain significant problems (Mahoney, 1977; Mavrogenis et al., 2020; Smith, 2006). At JID Innovations, as a new open access journal, we have a unique opportunity to consider and perhaps implement some changes in peer review, with the long-term goal of furthering the communication of new ideas and findings. High-quality peer review is and will remain a core value of JID Innovations. However, I believe that it is critical to improve the peer-review process if we are to advance science more efficiently. This includes realizing the limitations of peer review. We will continue to utilize new software to enhance our ability to detect plagiarism and fraud. Our goal is to assure our authors and reviewers that we will be diligent in using technology to its fullest extent to assure the integrity of the data presented in the journal. JID Innovations also relies on the quality of our reviewers. We believe that by focusing reviewers’ efforts, we can provide authors and readers the opportunity to publish and read the highest possible quality work. Although reviewers are often asked to identify the impact of a specific observation, determining which new ideas will be the critical finding for a breakthrough is often difficult if not impossible. However, reviewers can help identify whether studies ask important questions. At JID Innovations, we ask our reviewers to focus on the question that is being asked rather than on the specific results. We believe that negative results that address important questions are critical to the advancement of science. The self-correcting nature of science cannot function correctly if only positive studies are published. Negative studies are important in both evaluating past studies as well as in defining future work. The scientific process is often iterative, requiring multiple steps to get to the final answer, and the publication of negative studies plays an important role in this process. Reviewers can also determine whether the methods utilized in a study are appropriate and whether the results are analyzed correctly. The peer-review process is most helpful to authors when it helps them to avoid overstating results and to more clearly communicate their findings. We ask reviewers to look for ways to improve a manuscript and bring it closer to publication rather than to look only for reasons to reject it, thus promoting our ultimate goal of improving the presentation of the submitted manuscript and enhancing its value for the authors and the scientific community. Recently Boerckel et al. (2021) have proposed a change in how peer reviewers think of their task. They propose that peer reviewers think of themselves as shepherds of a manuscript rather than as gatekeepers focused on excluding ideas from publication (Boerckel et al., 2021). At JID Innovations, we embrace this philosophy, and we are focused on providing new knowledge to our scientific community to advance the study of skin and skin diseases. We want to shepherd new ideas and data to our community and activate the self-correcting aspects of science. Finally, we are committed to exploring with our community changes to make the peer-review process more transparent (Ross-Hellauer and Görögh, 2019). Open Peer Review has many levels, including open identification of both the author and reviewer and open publication of the review process when an article is published (Ross-Hellauer et al., 2017; Schmidt et al., 2018). Open science includes a commitment to barrier-free access to published articles, open data, and openness in the process whereby scientific reports are evaluated. Open peer review can provide the reader with an understanding of the thought process leading to the published work, clarify the decisions that led to the publication, and help the community improve the quality of scientific communications. There are many ways to implement a more open and transparent system for peer review. Over the next 6 months, we will be discussing different possibilities to implement a more open peer-review process for JID Innovations. I am interested in your comments and thoughts on our current peer-review process as well as the potential for adopting an open peer review. I am confident that as we work together, JID Innovations can lead skin biology and dermatology research in advancing open science, resulting in a quicker and more efficient process and improving our understanding of the skin, skin diseases, and their treatment.

Conflict of Interest

The author states no conflict of interest.
  20 in total

1.  Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone?

Authors:  P M Rothwell; C N Martyn
Journal:  Brain       Date:  2000-09       Impact factor: 13.501

2.  Effects of training on quality of peer review: randomised controlled trial.

Authors:  Sara Schroter; Nick Black; Stephen Evans; James Carpenter; Fiona Godlee; Richard Smith
Journal:  BMJ       Date:  2004-03-02

3.  Something rotten at the core of science?

Authors:  D F Horrobin
Journal:  Trends Pharmacol Sci       Date:  2001-02       Impact factor: 14.819

4.  Science fraud: from patchwork mouse to patchwork data.

Authors:  Gerald Weissmann
Journal:  FASEB J       Date:  2006-04       Impact factor: 5.191

5.  Peer review in 18th-century scientific journalism.

Authors:  D A Kronick
Journal:  JAMA       Date:  1990-03-09       Impact factor: 56.272

6.  The philosophical basis of peer review and the suppression of innovation.

Authors:  D F Horrobin
Journal:  JAMA       Date:  1990-03-09       Impact factor: 56.272

7.  The New England journal of Medicine: editor's report, 1967-1977.

Authors:  F J Ingelfinger
Journal:  N Engl J Med       Date:  1977-06-30       Impact factor: 91.245

8.  Editorial Peer Reviewers as Shepherds, Rather Than Gatekeepers.

Authors:  Joel D Boerckel; Lilian I Plotkin; Natalie A Sims
Journal:  J Bone Miner Res       Date:  2021-05-06       Impact factor: 6.390

9.  Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers.

Authors:  Tony Ross-Hellauer; Arvid Deppe; Birgit Schmidt
Journal:  PLoS One       Date:  2017-12-13       Impact factor: 3.240

10.  Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide.

Authors:  Jacalyn Kelly; Tara Sadeghieh; Khosrow Adeli
Journal:  EJIFCC       Date:  2014-10-24
View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.