A Novel Individual Mentored Methodology to Peer Review for Residents/Fellows.

Individualized guidance and assistance with constructive criticism as a mentored activity to peer review an article helps instill required rudiments, eliminate bad habits, and is shown to be beneficial to all participants. The Society of Laparoscopic & Robotic Surgeons initiated the R/F article mentoring review opportunity in 2014. The intimacy of actively debated discourse allows exposure to various peer review techniques and debate in tandem with education regarding the merits and faults of an article's hypothesis and conclusions, and how they are evaluated for publication and responses to authors. The benefits of coaching reassessment of ideas, critical analysis, airing of disparate viewpoints; and the need to update, reinforce, and relearn science is not static and is more robust using this method.

Peer review is done by every reader each time they read a medical article. Regardless of where a physician is in their career, they are always, to some degree, peer reviewing what they read and hear. It is the constancy of learning and relearning that is the process of science; constantly modifying and updating information. Peer review is a personal and cumulative critical assessment of what is written and heard in the transfer of medical information. Communicating new findings and replicating previous experiments and studies requires critical analysis; how studies are done, conclusions reached, literature reviewed, findings evaluated, recognition of flaws, analysis of data, and attention to detail regardless of the mode of communication is integral to the assessment. No peer review process is perfect and there are many routes to judging the validity and robustness of scientific communication. Article peer review is done to assess the quality and soundness of the material presented. Article findings and conclusions do not necessarily affect modifications to how medicine is practiced or its outcomes. The effect of one peer review mentored session on how medicine is practiced is not easily traceable or knowable, nor is it the reason peer review is done. A discussion group of three; two mentors and one resident/fellow (R/F) is how the JSLS has done theirs and is now past the pilot stage. This dialogue of ideas and discussion among colleagues reduces silo thinking and expands experience and knowledge helping practitioners update, change, reflect, reassess, relearn, and assimilate new findings and concepts.

Just as a R/F is individually trained and mentored by individual oversight a novel individual method for learning how to peer review a medical article has been adopted and put into practice by the Society of Laparoscopic & Robotic Surgeons. The Society of Laparoscopic & Robotic Surgeons initiated the R/F article mentoring review opportunity in 2014. The process is intense, time restricted, and thorough. On the JSLS website, R/Fs interested in a mentoring experience can register their contact information and training status. To date, 137 R/Fs have requested mentoring. This information is sent to the Chairman of the article mentoring program. The Chairman voice contacts the R/F about their interest and sends them a letter with materials outlining different approaches to reviewing an article along with reference articles.[1-13] The JSLS Editor-in-Chief selects an appropriate, newly submitted article for mentoring. The Chairman secures a commitment of an R/F and another mentor within a 48-hour time period to prevent any further delays to completing the article evaluation. The complete process is outlined in the Table 1.

Table 1

Sequence for R/F Mentoring Peer Review Article Review

Resident/Fellow self identifies to participate in peer review mentoring.

Seminal information articles about peer review and medical literature sent to R/F.

Editor in Chief suggests an un-reviewed article for consideration for R/F mentoring.

R/F Mentoring Program Chairman contacts a R/F for availability.

If not available, the Chairman asks another R/F.

If yes, the Chairman asks seasoned peer reviewer to assist until one accepts.

The article is made available to the Chairman, the assistant mentor and R/F through Editorial Manager.

Time commitments of the journal as sacrosanct and always met or the article is given back to the Editor in Chief for review by others

A time is agreed upon within the review parameters of the journal to have a conference call, Zoom or Skype meeting.

The R/F must write up their opinion of the article following the journal guidelines justifying their opinions and offer suggestions and criticisms to the Editor in Chief and the authors.

The discussion meeting is led by the Chairman, allowing the R/F to explain their review and substantiate their position.

The Chairman and assistant reviewer work in tandem, supporting, questioning and suggesting alternative views or supporting the R/F. This is done in an atmosphere of openness, willingness to learn about alternative thoughts, review new and old information, without hierarchy, status or ego oversite, and maintaining curiosity and constructive debate as a way to learn.

The Editor-in-Chief uses the Chairman and assistant mentor as an official qualified peer reviewer for his consideration for acceptance, revision, or rejection. The R/F follows the same guidelines as the regular designated peer reviewers and submits a written review critique of the article, with reasons for their decisions, and including detailed, specific, and corrective advice.

The next stage is a discussion session which lasts from 40 to 80 minutes done as a telephone conferencing, Skype or Zoom video conference, both nationally and internationally. The R/F is treated as a valuable and integral part of the review and discussion that is open, frank, and without any stress from their training institution or teachers. The discussion setting is casual, with appropriate discipline staying on task, but allowing tangents to be pursued. The Chairman leads the discussion having the R/F give their overall evaluation followed by the group discussing specific responses or questions generated by the article both positively and negatively. The objective is to assess the article’s methodology, correctness of data assessment, rationale and appropriateness of conclusions, ways for the author(s) to improve communication, and validation of their findings. It is also to learn from each other the intricacies of evaluating an article in ways that are not written about or easily decided. The R/F and assistant mentor write the Chairman an assessment of what went well, what did not, and what should be added or subtracted from the process. Changes and modifications are implemented when necessary. The R/F and assistant mentors rated the sessions as informative, worthwhile, and would recommend them to other R/Fs and mentors. Having mentors point out ways to do a nuanced evaluation not found in how-to review articles was rated important by the R/Fs. Having a private, open discussion was also a positive aspect of this type of mentoring review because it was interactive and personal.

The mentors share their review approaches from years of experience, pointing out deficiencies and overstatements in thinking, and aspects of article reviewing that are not published or talked about. A most important component is openness to learning and scientific debate. This requires a basic structure for evaluation of medical articles that is fair, unbiased, truthful, analyzed properly, accurate, and logical. There is a body of work regarding steps of peer review that is helpful to a point. This is like reading about swimming without the reader ever experiencing getting wet or being in a body of water to do the “swimming” they read about. Technical reading and actual execution of critical analysis of the task are two different activities.

Peer reviewing for journals and reading medical articles already published is a continuous work in progress. Reinterpreting information, rethinking, relearning, checking the robustness and rigor of content, assessing claims and updating knowledge is the obligation of all practitioners. Skim reading is not a method of critical assessment of medical articles. Slower, critical evaluation, fact checking, attention to detail, separating associations from causation requires time and commitment. Fact exposure and context discrimination takes time to assimilate and understand. A willingness to readjust, relearn and unlearn are necessary components of education. Having open ended discussion reduces and exposes article author and reviewer bias, reveals perspectives, generates new questions or solidifies accepted knowns. It reduces gaps in thinking, challenges accountability and commitment to evidence, opinions and facts. Mentors are cognizant to not unnecessarily criticize the authors or R/F. Most mistakes are honest and correctable but need to be addressed and adjusted. The individual peer reviewer only has to justify their position to themselves. In the small group environment there is agreement, disagreement, discussion, and thoughtful exposure to other perspectives. Constructive engagement without becoming disagreeable is a mainstay and ground rule for debating the article, which is part of peer review. A peer reviewer calls out falsehoods, poor methodology, incomplete data, association vs. causation, over interpretation of results, not correcting for or recognizing confounding variables, and unsupported conclusions. What is usually missing from most articles is understanding and appreciating variation and uncertainty. This happens by not acknowledging uncertainty, leaving in junk data, and resorting to creating a hypothesis that fits a statistical significance. Often overlooked is checking references and the statements or interpretations made that when checked are incorrect or not made at all.

Weighing how evidence is presented, obtained, evaluated, and put into meaning and relevance is part of mentoring. Articles can be influenced by authors’ incentives and unfounded claims that peer reviewers need to recognize. R/Fs can be naïve about the effect misinformed statements have and take articles at face-value and often do not search for the rest of the story. Experimental methods and their results do not necessarily result in certainty and since data is used as part of communication an understanding of how data is manipulated is part of reviewing an article. Reviewers must consider these aspects of statistics in their evaluation. “Statistics is the science of uncertainty and variation, but data-based claims in the scientific literature tend to be stated deterministically.” And “Is statistical communication about exploration and discovery of the unexpected, or is it about making a persuasive, data-based case to back up an argument?”[14] And “statistics is a form of modern alchemy, transforming the uncertainty and variation of the laboratory and field measurements into clean scientific conclusions that can be taken as truth”[15] transformed into certainty. The reader interprets article statements through their context that is not necessarily the same as the authors. Evidence are facts or observations used to support an assertion and provide or suggest their truth and must be represented as such in articles. A seminal question for reasoned argumentation is, what evidence would change your mind?

The mentoring procedure described here is unique, effective, and accepted by R/F and mentors. It is valuable, robust, and meaningful. Rapport and comradery are fostered by a loosely structured forum. The basics of experimental design are evaluated for appropriateness and completeness. Since a depth of knowledge and experience is not easily obtained by R/Fs, mentors provide guidance from practice and experience. The result is a mixture of collective wisdom, support, and constantly reassessing medical knowledge.