How to bring peer review ghostwriters out of the dark.

Early career researchers are frequent and valuable contributors to peer review. Systemic changes that acknowledge this fact would result in ethical co-reviewing, peer reviews of greater quality, and a reduction in peer reviewer burden.

INTRODUCTION

In February 2018, academic researchers, funders, editors, and publishers met to discuss Transparency, Recognition, and Innovation in Peer Review in the Life Sciences (https://asapbio.org/peer-review/summary; Polka ). Crucially, conference organizers also invited early career researchers (ECRs, such as graduate students and postdocs) to participate, including ourselves (G.S.M., then Executive Director of the Future of Research and R.S.L., then Chair of the Harvard Medical Postdoctoral Association). Meeting attendees heard from the Early Career Advisory Group at eLife, whose survey data showed that most ECRs had carried out peer review with no assistance from their supervisor, and the most common source of “training” was reading reviews of their own papers, not mentoring from their supervisors (Inside eLIFE, 2018).

To ECRs such as ourselves, this was of no surprise. These data aligned with our personal experiences and those of the thousands of ECRs we represented in our leadership roles. One of us had written a commentary prior to the meeting on this exact issue—that ECRs commonly “ghostwrite” reviews at the behest of their supervisors by reading the manuscript, writing the peer review report, and passing that back to their supervisor, who submits it under their name with or without edits or feedback to the ECR (McDowell, 2018). But this was apparently a revelation for many senior academics in the room, who interrupted the presentation to insist that this couldn’t possibly happen.

In that moment, we were struck by the stark generational divide between who is perceived to be carrying out peer review and who is actually carrying it out. We resolved to directly enumerate researchers’ experiences with and opinions of ghostwriting in peer review (McDowell ). Our study of 498 researchers, mostly postdocs in the life sciences, found three-fourths had co-authored a peer review report when they were not the invited reviewer (co-reviewed) and half had done so without being named to the journal editor (ghostwritten). These high rates of ECR participation in peer review were corroborated by a second international survey that found three-fourths of 1600 ECRs had experience responding to peer review (Jamali ).

ECR participation in peer review is apparently the norm—so is that a problem? Current policies that attempt to keep manuscripts only in the hands of invited reviewers are clearly ineffectual. They are also out of alignment with the opinions of the research community, a majority of whom find co-review to be a beneficial training exercise (95%) and naming co-reviewers to editors to be valuable (82%) (McDowell ). Instead, we believe that supporting ethical co-review by ECRs is possible and pragmatic. It requires 1) aligning journal policy with current practice and opinion, 2) building expectations for co-review into routine conversation, and 3) standardizing training in peer review. Here we summarize specific recommendations for systemic change to reduce ghostwriting in peer review which can be driven by all stakeholders—ECRs, invited reviewers, journals, and training institutions (Figure 1). We describe how adopting these common sense measures will benefit all parties and peer review (Figure 2). We also highlight key components of journal policy required to address co-review and ghostwriting by ECRs (Table 1 and Figure 3).

FIGURE 1:

Recommendations for how all key stakeholders can ensure the inclusion, training, and recognition of ECRs’ scholarship in manuscript peer review (icons made by Pause08 from www.flaticon.com).

FIGURE 2:

Benefits of adopting recommendations for all key stakeholders (icons made by Pause08 from www.flaticon.com).

TABLE 1:

Essential questions that journal policy should overtly address to bring ECR ghostwriters out of the dark.

Topic	Question	Example text
ECR status	Are graduate/medical students and/or postdocs allowed to participate in peer review? As invited reviewers? As co-reviewers with their PI?	“We recognize that invited reviewers may wish to involve their trainees in peer review. PhD/MD students and postdocs may participate in mentored co-review with an invited reviewer. Postdocs may also serve as independent invited reviewers.”
Training in peer review	Are invited reviewers allowed to involve co-reviewers for the purpose of training? If so, how? May they contribute to the report? Should their names be disclosed to the editor and how? Are they subject to policies on manuscript confidentiality and conflict of interest?	“Manuscripts may be shared with trainees of the invited reviewer for co-reviewing and/or training purposes and these ECRs may contribute text and/or ideas to the peer review report. If this occurs, the invited reviewer must disclose co-reviewer names to the editorial staff at the time of submission. All parties, including the invited reviewer and their designated trainees, must uphold the confidentiality of the manuscript and be free of conflicts of interest.”
Delegation of labor	If an invited reviewer wants their trainee to perform the review instead of them, what should they do?	“If you would like to recommend that your qualified trainee performs the review in your place, please decline the invitation and provide their name and contact information for the editor [textbox].”

FIGURE 3:

A co-reviewing workflow to guide invited reviewers through a journal’s expectations for ethical involvement of ECRs in peer review. This workflow and an editable version are available on Zenodo for use by any interested party through an open license (https://zenodo.org/record/4441072).

Recommendations: how to name and train co-reviewers

2. Align journal policy with current practice and community opinion

Peer review policies in general are not transparent. Confusion about whether ECRs are allowed to contribute is as much a driver of ghostwriting as outright bans (McDowell ). Many journals have policies that allow only invited reviewers to have access to the manuscript and perform the review (Klebel ). ECR participation is discouraged through specific language, or more often by failing to address ECRs and/or co-reviewing in written policies: a recent analysis of policies at 171 academic journals found that three-fourths did not clearly state whether co-reviewing was allowed (Klebel ). Some journals use language closely aligned with the Peer Review Guidelines of the Committee on Publication Ethics (Committee on Publication Ethics, 2019; emphasis added below):

“Do not involve anyone else in the review of a manuscript (including ECRs you are mentoring), without first obtaining permission from the journal…The names of any individuals who have helped with the review should be included so that they are associated with the manuscript in the journal’s records and can also receive due recognition for their efforts.”

While this could be interpreted (and may be intended) to mean that co-reviews are welcome after prior approval, in reality prior approval is rarely requested. As a result, this language dissuades invited reviewers who have involved co-reviewers without prior approval from acknowledging that manuscripts have been shared. Thirty-nine percent of ghostwriters who asked for authorship were denied by the invited reviewer because “journal requires prior approval to share manuscript, which was not obtained” or “journal does not allow ECRs to review” (McDowell ).

New policies could favor transparency with explicit statements in reviewer guidelines and invitation letters that address the reality that co-reviewing occurs (even providing statements of support for the practice) over prohibitive language about sharing the manuscript (see example text in Table 1). It is not technologically trivial for journals to enforce that the manuscript is viewed only by the invited reviewer. Instead, it would be beneficial to accept, and assume, that co-reviewing will occur and create mechanisms to prevent ghostwriting, such as asking reviewers to answer “Who else contributed to this review with you?” or to confirm explicitly that the review was performed alone (Figure 3). This also ensures that the journal is not liable for any potential conflicts of interest arising from the inclusion of unknown reviewers, a concern expressed recently by journal editors (Hamilton ). Journals might even consider enacting policies that recategorize the ECR as the primary reviewer in their databases and the originally invited reviewer as the co-reviewer. This would bring the ECR to the editor’s attention and make it more likely for the ECR to be invited as an independent reviewer on future manuscripts. Some individual journals are making these changes already (e.g., F1000Research; Fatone ) and more might follow should COPE lead the way on aligning policy with current practice and opinion (Table 1).

2. Build awareness and discussion of co-review into routine conversation

Policy changes at journals must be accompanied by cultural shifts that normalize open discussion about co-reviewing. Co-reviewing by ECRs is commonplace and should be acknowledged as such. A lack of communication about this topic is still the norm and often the reason why co-reviewer names are not provided to the journal: 73% of co-reviewers reported that authorship was not discussed with the invited reviewer (McDowell ). As the mentor and expert invited by the journal, principal investigators (PIs) should initiate discussions with ECRs about peer review co-authorship and journal policies on confidentiality and conflict of interest (Figure 3). These should be routine conversations that accompany the handoff of the manuscript under review and the return of the peer review report. Editors should include discussion of co-review in their letters to invited reviewers, highlighting the journal’s policy and providing clear instructions for how to ethically involve and name co-reviewers. These instructions might also prompt invited reviewers to discuss review authorship with the ECR and share feedback with them for training purposes. Figure 3 provides an example of a co-reviewing workflow that could be posted in publicly available reviewer guidelines and shared in the invitation letter to guide invited reviewers through the journal’s expectations for ethical involvement of ECRs in peer review. Once editors and PIs incorporate a discussion of co-reviewing into common practice, ECRs may feel more free to ask their PIs to share their names with the editors and to provide constructive feedback on their work.

Summary: How to Name Co-reviewers

Journals and COPE

State clear policies that acknowledge and preferably embrace co-review by ECRs.

Remove prior approval barriers to naming co-reviewers.

Create unambiguous expectations and mechanisms (ex. text boxes in report submission system) to share co-reviewer names with the editor.

PIs

Discuss co-review ethics with ECRs and editors throughout review process.

Provide journal with co-reviewer names whether prompted or not.

ECRs

Ask your PI to provide your name to the editor when submitting peer review reports.

Editors

Discuss co-review policies routinely with invited reviewers to normalize expectations.

3. Improve and standardize training in peer review

Ghostwriting is often rationalized as acceptable because it is falsely equated with training (McDowell ). However, training is not a sufficient justification for the 39% of ECRs that report the experience “I read the manuscript, wrote a full report for my PI, and was no longer involved” (Inside eLIFE, 2018; McDowell ). When (unnamed) ECRs write peer review reports on behalf of an invited reviewer without feedback from that expert, it is exploitation, not education. Moreover, naming co-reviewers to a journal editor does not undermine training, no more than naming co-authors on an article undermines training in writing manuscripts. Therefore, the delegation of labor to ECRs without feedback or acknowledgement must be considered separately from true mentored co-review, which should be celebrated as a valuable teaching tool.

For co-review to be effective training, PIs must constructively critique their supervisees’ work. Iterative feedback from “expert” to “new” reviewers leads to positive outcomes as measured by reviewers’ beliefs and objective evaluations of review quality (Doran ; Castelló ). One effective pedagogical approach asked PIs to write a peer review report on a manuscript in parallel with their trainee, thereby creating an example that could be used for comparison to the trainees’ report (Castelló ). Quality feedback is critical: when expert reviewers are not well trained in how to provide feedback, pairing experts with novice reviewers is not sufficient to improve review quality (Houry ). In that randomized control trial, experts were not trained in how to teach peer review to minimize burden and encourage their participation in the program. This speaks to the underlying problem that PIs are already overburdened by a hypercompetitive research environment that lacks concrete incentives to engage in time-consuming, quality mentorship. Structural reforms that incentivize PIs to name co-reviewers, as described above, would benefit from also including expectations that co-review be an interactive pedagogical exercise.

Peer review is integral to research and so teaching peer review should be integral to researchers’ education. Graduate schools should ensure "Peer Review 101″ is taught as one of the essential components of training in the sciences. Redirecting this responsibility from individual PIs to training programs allows for the implementation of standardized, evidence-based pedagogy that is subject to community oversight. This type of intervention is badly needed given that the most common form of “training” is reading reviews on one’s own manuscripts (McDowell ), which is limited in scope, subject to bias, and likely to perpetuate the current trend toward overcritical reports (Schneiderhan, 2013). Many graduate programs already hold journal clubs that could be converted into peer review courses if ECRs also read peer review reports, write their own, and receive individualized, iterative, constructive criticism by experienced reviewers (some examples are detailed under “Examples of formalized training in peer review” in McDowell ). Scientific societies and journals could set discipline-specific standards, distribute exemplar review reports or templates and offer peer review training. Indeed, some society journals are already filling the void left by graduate education by creating their own reviewer training (e.g., Genetics Society of America; Johnston, 2017; American Chemical Society [acsreviewerlab.org]). This benefits ECRs and the journal by creating a sustainable method for recruitment of new, qualified reviewers and society members.

Coursework in peer review should lay the groundwork to support, not replace, individualized training from a PI through co-review. It would also make peer review training equally accessible to all ECRs, rather than a privilege available only to those with PIs providing quality mentorship. This would create an important safety net to ensure that all PhDs have been sufficiently trained to perform thorough and fair peer review.

Summary: How to Train Co-reviewers

PIs

Constructively critique your co-reviewer’s work so they can learn from it.

Provide your co-reviewer with previous or side-by-side examples of your reviews.

Keep a record of your trainees that have co-reviewed as evidence of your mentorship.

ECRs

Ask your PI for feedback on your peer review work and examples of their work.

Graduate programs

Teach courses in peer review to all students using evidenced-based pedagogy.

Journals and scientific societies

Set discipline-specific expectations for peer review content and tone.

Distribute exemplar review reports or templates.

Offer peer review training.

Outcomes: bringing ghostwriters out of the dark benefits everyone

Increased review acceptance rates and diversity in the reviewer pool. Enacting policies and adopting behaviors that acknowledge and encourage credited co-review have practical benefits to all parties (Figure 2). Journal editors take care to assign reviewers based on their expertise and to prevent conflicts of interest. This effort is wasted when ghostwriting obscures who is performing the peer review and how well. By encouraging co-reviewing, more invited reviewers may have the capacity to accept invitations (Mehmani, 2019). Including and naming ECRs in peer review will also increase the reviewer pool, ameliorating concerns about reviewer burnout (Kovanis ). It will also diversify the reviewer pool, since most of the diversity of the academic population is concentrated in early career positions (Gibbs ). These measures will address growing calls to tackle the lack of diversity (e.g., gender bias; Hagan ) at all levels of the publication process (Day ; Royal Society of Chemistry, 2020; Seery, 2020) including peer review and which has been exacerbated by the COVID-19 pandemic (Squazzoni ).

Increased peer review quality. The participation of co-reviewers across career stages will also enrich peer review and the subsequent standard of published research (McDowell and Lijek, 2020), especially if deliberate training courses are provided in graduate education. Predictors of good review quality as judged by editors include being early in one’s career (Evans ; Black ; Callaham and Tercier, 2007) and closer to active research but not years of reviewing experience itself (Stossel, 1985). In an ongoing experiment, Elsevier editors rated co-reviewed reports highly and thought favorably about inviting co-reviewers to serve as an independent reviewer on future manuscripts (Mehmani, 2019). Our survey respondents held similar opinions about how ECRs and co-review benefit peer review (McDowell ). Given the hidden nature of co-review, it is possible that studies of peer review quality already include co-reviewing activity and so underreport its utility. Peer review quality will also be improved once routine graduate school courses on peer review create new generations of uniformly trained reviewers.

Increased support for PIs and recognition for ECRs. Including ECRs in peer review also benefits PIs, who should be encouraged to include their trainees as credited co-reviewers. Invited reviewers are experts by definition and so should be trusted to involve whomever they consider to be the best person in their research group to critique the research. PIs who provide feedback to the ECR and name them to the editor are right to consider co-review an integral part of quality mentorship. These examples of good mentorship should be better incentivized and formally recognized in PIs’ career evaluation processes. Naming and tracking ECR co-reviewers would provide documentation of a PI’s mentorship for their own evaluation portfolios. It may also encourage PIs to properly acknowledge co-reviewers’ contributions in the first place. When mentored co-review is not possible for an overburdened or uninterested PI, they should instead transparently delegate peer review to a qualified member of their research group, either as a credited co-reviewer or better yet by suggesting that the editor extend the review invitation to the ECR directly (Figure 3), thus benefiting both parties.

Credited co-review is also valuable as a credential that recognizes an ECR’s expertise. Being known to academic editors carries weight for ECRs and being able to document reviewing experience, especially through third parties like Publons, matters for hypercompetitive jobs, and even U.S. immigration applications (North America Immigration Law Group; Publons Advance your career: Green Card for Outstanding Researcher). Indeed, a majority of co-reviewers find value in being identified to the journal editor (McDowell ). Contributing ideas and/or text to a peer review report is work, for which authorship is a fair and reasonable reward.

Next steps: adopt systemwide changes to bring equity to peer review

Systemic problems require systemic solutions; individual action on the part of “good apple” PIs, ECRs, or editors is not sufficient. Journals, scientific societies, and graduate programs must take the burden off of individual PIs, ECRs, and editors by creating unavoidable, incentivized, and transparent structures that name and train co-reviewers. When structures are casual and opaque, it is easy for implicit biases to take hold. For example, simply including ECRs in reviewer databases does not result in their participation in peer review (eLife, 2019). Editors are human and as such are drawn to names in reviewer databases that they recognize, who are less likely to be ECRs. Through tracking what is taking place, and then reporting it publicly, eLife is able to hold itself accountable for identifying and addressing biases in who is involved in peer review, in order to work toward a more equitable and inclusive review process. We urge other journals and COPE to take a hard look at their policies, perhaps with the help of ECR and PI focus groups to check that readers’ interpretations align with journal intentions. Do policies unambiguously define how ECRs may or may not participate in peer review (Table 1)? Are there mandatory, incentivized structures to facilitate naming co-reviewers and are barriers to transparency removed? Scientific societies, graduate programs, and institutional postdoc offices should also advocate for structural changes to peer review training to refocus the burden from individuals to the academic training system. Peer review training should not be an “extra” professional development activity that an individual ECR might choose to do in their free time; it must be integrated systemically into science education for all.

CONCLUSIONS

Peer review drives decisions about the publication and funding of research and inspires public confidence in scientific findings (Kelly ). Striving for unimpeachable peer review is of utmost importance, especially in times of science denialism. Like manuscripts, peer review reports are often collaborations between PIs and their trainees. Journals should update their written policies and unwritten practices to recognize this reality. Encouraging PIs to more freely discuss co-reviewing and share co-reviewer names with editors will make peer review more transparent and fair. Naming and training everyone who contributes to peer review will benefit all parties and the integrity of peer review.