Preprint articles as a tool for teaching data analysis and scientific communication.

The skill of analyzing and interpreting research data is central to the scientific process, yet it is one of the hardest skills for students to master. While instructors can coach students through the analysis of data that they have either generated themselves or obtained from published articles, the burgeoning availability of preprint articles provides a new potential pedagogical tool. We developed a new method in which students use a cognitive apprenticeship model to uncover how experts analyzed a paper and compare the professional's cognitive approach to their own. Specifically, students first critique research data themselves and then identify changes between the preprint and final versions of the paper that were likely the results of peer review. From this activity, students reported diverse insights into the processes of data presentation, peer review, and scientific publishing. Analysis of preprint articles is therefore a valuable new tool to strengthen students' information literacy and understanding of the process of science.

Introduction

The advent of online publishing has ushered in new and diverse ways of sharing academic knowledge. New venues have emerged such as online open-access journals, academically-oriented Twitter feeds [1,2], podcasts [3,4], and the growing popularity of preprint servers. Collectively, these new methods seek to disseminate scientific information more quickly and more directly to scientific researchers, medical practitioners, the lay media, and the general public. They have also had a dramatic impact on the traditional publishing and peer review process; examples of changes within the traditional publishing system include the use of open review in which reviewers names and/or comments are included online with the published article [5,6], post-publication peer review [7,8], and a rethinking of the objective and subjective criteria that ought to form the basis for peer review [9]. As educators, we must ensure that our students are aware of the modern means by which scientific information is disseminated as well as the strengths and limitations of each mechanism.

Preprint servers are an important part of this new communication landscape. Preprints are complete manuscripts that have not gone through peer review but may be published to an online preprint server in a parallel and independent process to journal submission. The first preprint server to be founded in 1991 was arXiv (https://arxiv.org/) which serves the physics, astronomy, computer science, engineering, and mathematics communities, followed by bioRxiv (https://www.biorxiv.org/) for the life sciences in 2013 [10] and medRxiv (https://www.medrxiv.org/) in 2019 for the health sciences [11]. Although the frequency varies by discipline, the use of preprint servers to disseminate research has increased dramatically from 2014, when there were 4,012 authors of preprint articles on the bioRxiv server, to 2018, when there were 106,231 [12]. Indeed, publication on a preprint server is increasingly becoming the first step of publishing a research article, and most manuscripts that are submitted as preprints eventually go on to be peer-reviewed and published in academic journals [13]. This broad acceptance of preprint publication has been further and dramatically accelerated by the COVID-19 pandemic as fully 2/3 of all preprints posted to medRxiv in 2020 were related to SARS-CoV2 and COVID-19 [14].

This unique publishing environment offers new opportunities for science education. Strong pedagogical approaches such as the CREATE model [15], article annotation [16], and rhetorical analysis [17] have been developed to teach students the scientific research process and how research data is communicated in journal articles. While peer review is recognized for its importance in keeping low-quality research out of the public discussion and in quality assurance [18], there are fewer methods that teach students the process of scientific publication and peer review [19,20]. This is a missed opportunity to develop the students’ abilities to communicate science and to use quantitative reasoning, as stated in Vision and Change [21]. Indeed, initial forays in this area have found that having students engage in evaluation of their peers’ work increases students’ understanding of the scientific publishing process and the role of peer review within that process [22], identifies and corrects student misconceptions [23], and enhances their writing and critical thinking skills [19]. At the professional level, both making peer reviews open and accessible post-publication and mentoring trainees in the review process have been noted for their pedagogical value [24,25].

Methods that engage students in peer review as a pedagogical tool often have students act as reviewers of either published journal articles or their classmates’ work. One challenge in this approach is that students struggle with interpreting research data [26], an exceptionally difficult skill, and are known to read articles passively and superficially, focusing more on the narrative of the text than on the data presented in the figures and tables [27,28]. We therefore developed a cognitive apprenticeship approach [29-31] to develop students’ skills in critiquing research papers and experimental data. In the cognitive apprenticeship model, students develop skills by engaging in realistic, real-world tasks. Unlike a traditional apprenticeship, cognitive apprenticeship helps students to first reveal and then to master the cognitive and metacognitive skills and processes used by experts [31]. Central to this process are the principles of modeling, in which students see how experts approach problems, coaching, in which students are guided in reflection on how their novice process differs from the expert method, and scaffolding, in which students are provided with cognitive tasks of increasing complexity that can help them to achieve “successive approximation of mature practice” [30].

We applied the cognitive apprenticeship approach to the teaching of data interpretation and critiques by having students review articles themselves and then see how professionals critique the same paper, thereby revealing and making explicit the expert’s thinking and analysis of the manuscript. This professional peer review can be deduced from changes that have occurred between the preprint and final versions of an article; therefore, by uncovering this professional assessment, students can reflect on the differences between their assessment and the peer reviewer’s. Studying these professional reviews can help students to pinpoint the most important aspects of paper, to critically evaluate the quality of the data, and to identify areas where information is not effectively communicated. Although there are other tools that can help students to see how professionals critique papers (for example peer reviews that are posted online), having students compare the preprint and final articles themselves gives them an active and engaging way to discover weak areas in the original paper and to reveal this information for themselves. Importantly, this method can be employed for research papers regardless of their topical content, and we found success with this method using 10 different articles at three different institutions.

We hypothesized that the detailed comparison of the preprint and published, peer-reviewed versions of an article would help students to develop skills in the analysis of research articles as well as provide students with unique insights into the research process. Indeed, we found that comparison of preprint and final versions of a paper: (1) helped students to critique research data by uncovering weaknesses in the preprint version that they initially failed to identify, (2) helped students to appreciate the role of peer review in ensuring publication quality, (3) helped students to understand and critique the process by which scientific results are communicated.

Methods

We determined the educational benefits of analyzing preprint articles for students in three courses during the Fall 2020 semester: BL481: Biology Research I at a mid-sized private comprehensive university, BIOL 451: Biology of RNA at a mid-sized public comprehensive university, and BIOL314: Medical Genetics at a small private liberal arts college for men as well as one course during the Spring 2021 semester: BL322: Synthetic Biology with Lab at the same mid-sized private comprehensive university. BL481 is an independent 3-credit research course in which students worked closely with a faculty research mentor (3 students enrolled and participated in this study). BIOL451: Biology of RNA (7 students enrolled and participated in this study), BIOL314: Medical Genetics (13 students enrolled and participated in this study), and BL322: Synthetic Biology with Lab (24 students enrolled, of which 6 chose the option to participate in this study) are upper-level courses for students in the biological sciences Two students did not complete all three parts of the project and their work was not included; in total 27 students from the 3 institutions participated in this study. They ranged from sophomore to seniors, with 2 being sophomores, 8 being juniors, and 17 being seniors. Because of the COVID-19 pandemic, courses at the private comprehensive university were being taught entirely online in Fall 2020, while classes in the spring semester and at the other institutions were held in-person but socially distanced.

A strength of the learning process described here is that it is independent of the content of the research being discussed. At each institution, different research preprints and final articles were therefore used. At the private comprehensive university, students each analyzed a different pairing of preprint and published paper; articles focused on genome rearrangement in yeast [32,33], reconstruction of the SARS-CoV2 genome [34,35], synthesis and mutation of a bacteriophage genome [36,37], gene expression burden [38,39], design of RNA regulators [40,41], noninvasive prenatal tests [42,43], programmable gene regulation [44,45], and metabolic engineering [46,47]. At the public comprehensive university, all students analyzed a paper on the topic of long noncoding RNA [48,49], and at the liberal arts college, all students analyzed a paper on molecular analysis of neurosensory syndrome [50,51].

In designing our study methodology, we followed three steps of the cognitive apprenticeship: mimicking, modeling, and coaching, For each of these stages, students completed and submitted worksheets as separate assignments: (1) for mimicking professional practice, students analyzed the preprint version of the article (S1 File), (2) for modeling, students compared the preprint and final published version of the article (S2 File), and (3) for coaching, students reflected on the learning experience (S3 File). Each stage proceeded sequentially, and students were not given the next assignment until the previous one had been submitted. Students met with the instructor and discussed each part of the assignment close to the day on which it was submitted. Instructors prompted students to discuss their findings, what aspects of the assignment were easy or difficult for them to accomplish, and what was unexpected or surprising in their analysis of the preprint or final paper. All parts of this study and assessment methods were approved by the Institutional Review Board of Loyola University Maryland (HS-2021-013) and written consent was obtained from all participants.

To analyze whether analysis of preprints and final papers enabled students to better critique research data, we first analyzed student worksheet part 1. For each instance in which students made a critique of the preprint article, that critique was classified by a faculty grader as falling into one of the following 4 categories: (1) additional repetition, controls, or an extension of the experiment is needed, (2) statistical test(s) are needed, (3) clarification of either the text or depictions of the data is needed, (4) inappropriate conclusions have been drawn from data. These data were compiled and both the average number of critiques per student and the range were calculated. Student comments regarding formatting (font size, figure formatting, or how graphs are labeled, as examples) were not included as critiques, but comments indicating that the data was difficult to interpret or needed additional labeling were included as critiques. For student worksheet part 2, we first began by noting any changes that students identified between the preprint and final versions of the results section of the paper; these changes were sorted into the same 4 categories described above so that the student critiques of the results from part 1 could be compared to the actual changes in the final paper that were identified in part 2. For student worksheet part 2, we analyzed not only the results section but also the introduction and discussion sections. Changes in these sections were categorized as to whether they were additions, deletions, or changes to the information and whether they occurred in the background information, experiments and their results, or conclusions. To analyze the student reflections in part 3, we used coding analysis for qualitative data according to the methods described in Saldana [52]. First level coding was performed using three different methods: descriptive, in vivo, and values coding, and axial coding was used for second-level coding and inductive analysis of themes.

Results

Role of the preprint and final paper analysis in helping students to critique research data

Students in our classes had a range of academic preparation and some had little previous exposure to primary literature. We therefore wanted students to first become familiar with the research paper that they were analyzing. Students’ first worksheet therefore began with a self-guided analysis of the introduction section of the preprint paper in which they were asked to identify the main problem being addressed, the importance of answering that question, the most important background information, and the hypothesis being tested (A copy of the student worksheet is available as S1 File). In this first part of the assignment, students also analyzed the discussion section to identify the major conclusions, restate the importance of the work, and reflect on any future directions indicated by the author(s). Although we did not analyze these portions of the worksheet for this study, they provided students with the deep familiarity that they required to analyze the research results.

We then determined what critiques students made of the preprint (S1 File; an example of a completed student worksheet is included as S4 File). In this exercise, students are therefore acting in the role of peer reviewers for the preprint; because most of the preprints were posted on the preprint server within days or weeks of when they were submitted to a research journal, the preprint should be nearly equivalent to a journal article at the time of submission. On average, students provided 2.04 critiques of the results section of the preprint article that they analyzed (Table 1); four students had no critiques while all others identified between 1 and 9. Students’ most common critiques included the need for additional clarification of the data or its interpretation (average of 0.96 per student) and the need for additional repetition, controls, or extension of experiments (average of 0.88 critiques per student). Students were less likely to indicate that statistical tests were used inappropriately or insufficiently (average of 0.08) or that data was interpreted incorrectly (average 0.13). While the number and types of weaknesses present will certainly vary depending on the preprint article selected, students were most likely to identify places where they feel clarification is needed; while this may be unsurprising for novice readers, it provides a preliminary suggestion that students may not be equally adept at identifying all types of manuscript weaknesses.

Table 1

Suggested and actual changes between preprint and final versions.

	Student Critique of Preprint (Part 1)		Student Analysis of Final Paper (Part 2)
	Avg.	Range	Avg.	Range
Additional repetition, controls, or extension of experiment	0.88	0–5	1.58	0–5
Statistical test needed	0.08	0–1	0.09	0–1
Clarification needed	0.96	0–3	0.67	0–4
Inappropriate conclusions drawn from data	0.13	0–1	1.04	0–3
Total number of critiques	2.04	0–9	3.38	0–10
Changes in common between parts 1 and 2	0.08	0–1
Changes in part 2 not identified in part 1	3.25	0–10

The average and range for the number of critiques of the preprint version of the article (based on the worksheet Student Critique of Preprint Part 1; S1 File) and the average and range for the actual number of changes that students identified between the preprint and final versions of the paper (based on the worksheet Student Analysis of Final Paper Part 2; S2 File).

We next determined whether students were able to compare the preprint and final versions of the article to identify changes (S2 File; an example of a completed student worksheet is included as S5 File); these changes presumably represent weaknesses in the preprint that were identified by professional scientists (Table 1). In this analysis, we did not include any superficial changes to formatting or figure quality; we also did not include changes to factual information or data values as these might be the result of error correction by the authors that a peer reviewer would be unlikely to anticipate. Central to our analysis is the assumption that most of the remaining substantive changes between the preprint and final versions of the article are due to the peer review process; while some might be due to changes initiated by the authors either independently of the peer review process or due to community feedback following the publication of the preprint, these changes still represent ones initiated by science professionals to remediate weaknesses in the preprint, and we therefore believe that these are worth including in our tabulations. Students identified an average of 3.38 changes between the results sections of the preprint and final versions of the paper (range of 0–10 changes). The most common changes were to the conclusions drawn from the data (average of 1.04 changes), clarification of how the data is presented or described (average of 0.67 changes), and the need for additional experiments or changes to how the data is presented (average of 1.58 changes). There were therefore a greater number of changes made between the preprint and final versions (average of 3.38) compared to the number of critiques (ie. suggested changes) made by students (average of 2.04).

For some types of changes, the number of changes that occurred was close to the number suggested by students (for example, students identified an average of 0.96 instances where clarification was needed while an average of 0.67 clarification changes were actually detected in the final version). But importantly, the identity of these changes differed. Indeed, on average there were only 0.08 changes per paper that were common between the student critique and the actual changes that were made between the preprint and final versions, and only 2 students critiqued the preprint in a way consistent with changes actually made to the paper. This indicates a significant difference in the critiques made by the students (in part 1 of their assignment) and by science professionals (changes between the preprint and final versions that students identified in part 2), as could be expected [28]. In all cases except one, students were able to identify changes (an average of 3.25 per paper; range of 0 to 10) between the preprint and final versions that did not correspond to an issue that they identified in their own critiques. This exercise was therefore able to help students uncover for themselves areas of weakness in the original preprint that they had missed; it is therefore another valuable pedagogical tool to help upper-level students learn to rigorously analyze research data.

While our primary objective was to use this technique to help students learn to analyze research data and data presentation, we used the opportunity to have students also analyze changes to the introduction and discussion sections of the paper as this could be an additional tool to help them to understand the scientific publishing process. Because students are not experts in the discipline, we did not think it reasonable to ask them to perform their own critiques of the introduction and discussion sessions of the preprint, so we only asked them to identify changes between the preprint and final versions (Student worksheet part 2, S2 File). Students identified additions, deletions, and changes of information, with additions of information (perhaps unsurprisingly for the peer review process) being the most common (Fig 1). In each instance, students were able to identify changes between the two versions, with an average of 7.58 changes (range of 2–20) identified per article. This activity therefore provided students with additional insight into the types of changes that can be introduced and requested by reviewers through the peer review process.

Fig 1

Average number of changes between the preprint and final versions.

The number of changes as identified by students in the introduction (light gray bars), results (hatched bars), and discussion (dark gray bars) sections. Error bars represent the standard deviations for each value.

Role of the preprint and final paper analysis in helping students to appreciate the role of peer review

Because our method was effective in having students use an active learning approach to identify evidence of peer review in the final versions of published papers, we wanted to extend this analysis by determining whether this process also helped them to learn more about the role and importance of peer review. To determine this, the third part of the student assignment asked them to respond to several questions including: (1) What surprised you most when you compared the preprint and final versions of the paper? (2) Is it easy or hard to distinguish which additions were made because the authors had more time to acquire more data and additions that were made due to requests from the peer reviewers? How do you think you can you tell the two apart? (3) What, if anything, did you learn about presenting and interpreting research data from this activity? (4) Do you think that this paper and the publishing process that it went through achieved an appropriate balance between getting research results out quickly versus the need for high-quality, complete, and reproducible research?

When comparing the preprint and final versions of the paper, some students were surprised at the number of changes between the two versions (9 students) while others commented on how few changes there were between the two (13 students). Interestingly, sometimes these comments came from the same student! In these cases, students were surprised that there were few changes, but that the changes were as substantial as the addition of an entire experiment. For example, a student commented:

This would have taken considerably more lab work so would have involved a significant effort by the research team after putting out the preprint. I was even more surprised as this experiment was probably not absolutely vital to validate the research performed. However, I do believe it probably does improve the overall strength of the research.

-male student, senior class

Students were also surprised at the types of changes that they observed; in addition to adding new experiments, these included changes that clarified information or emphasized new topics, large-scale rearrangements of entire sections of the paper to restructure the argument, the fact that the majority of changes were additions of information, and changes to how figure were presented and formatted that led to better emphasis on specific aspects of the data and better integration of data figures with interpretation of results.

In terms of whether these changes were the result of the peer review process or whether they were due to the authors having more time, the students were evenly divided as to how easy that was to determine. Changes that they felt were indicative of reviewer comments included those that provided greater clarity, better flow to the paper, additional background information, removal of uncorroborated claims or speculation, and changes to the interpretation of the data to include more uncertainty. Additionally, students predicted that reviewer input could be detected when experiments in the preprint had noticeable defects that were fixed in the final version, when statistical tests were added, and when experiments were added that provided more consistency.

Personally, I learned that the scientific publishing process can cause highly noticeable changes to one’s paper in the results, interpretation of results, and discussion portions of the paper…. In particular, this paper had shown me the difference in quality in the explanation of the results and in discussion of the overall paper. Furthermore, I learned that the requests of peer reviewers can often work to help a paper be more clarified in its explanations. Many times throughout the paper, I saw an improvement in the wording of certain interpretations that allowed me to better understand what they’re [trying] experimenting with and the significance of the results.

-male student, sophomore class

On the other hand, they predicted that changes to data values or the addition of additional crucial data was the result of the researchers having more time. Responses to this question therefore indicate that students understand the role of peer reviewers in ensuring that data is interpreted properly, that the paper is clear to read, and that defects in experimental procedures or data presentation are rectified.

Student reflections therefore demonstrate that this exercise helped them to better understand the process of peer review, but there were other significant components of presenting and interpreting research data that they also learned (Table 2). They mentioned that the exercise helped them to better understand how to read scientific papers (including the need to pay attention to detail and the utility of jumping around between multiple sections of the paper), to understand the importance of precise wording in scientific writing, to properly use and present scientific data, to interpret data, and to understand how the process of scientific publishing and peer review occurs. Students therefore reflected on incredibly diverse aspects of the scientific process. While this exercise intended to help them to build skills in data analysis and to understand the role of peer review, they clearly gained a greater breadth of understanding about data presentation, interpretation, and publishing.

Table 2

Results of student reflections on the preprint analysis activity.

Reading papers	The changes focus the reader on important points.
	“There are many bits of information that I miss while reading.”
	The figures and text complement each other.
	Each figure aimed to prove one claim.
	It reinforced the importance of figures relative to the text interpretations.
	The need to scrutinize data
	Reading scientific literature is a skill that must be developed over time.
	Can skip around the paper when reading
	Scrutiny is required to fully grasp the meaning of figures and data.
	“I also learned that not fully understanding certain vocabulary in a paper does not mean I can’t understand the bigger picture.”
Language in scientific writing	The importance of wording
	Every detail has to be explained
	Write to the reader; you need to communicate effectively.
	It’s best to be simple and concise; leave interpretation to the discussion.
	Need to communicate findings effectively and to be as specific as possible to prevent misinterpretations
Use and presentation of data	Experiments build on previous work.
	Scientists use multiple pieces of evidence to support claims.
	“I found that there is a lot that goes into making a claim, in effort to come up with data that is consistent and considers all aspects of the claim being made.”
	“Need to….obtain not only enough, but quality experimental results”
	Scientists present both the strongest research and also weaker data that provides context.
	Scientists use multiple methods to prove one claim.
	Need to present data effectively
	The way that the data is presented is important.
	Data may not fit the scientists’ theory.
Interpretation of data	“Sometimes you don’t interpret the data the same way the scientist who did the experiment.”
	“Even if the overwhelming preponderance of evidence is in your favor, you must provide alternatives that could explain the findings of the data.”
	Initial thinking about the data may change and may become more advanced as the publication process advances.
	Judgement is required: even researchers don’t always know what is important to support their thesis.
Scientific publishing and peer review	How specialized certain scientists, and their fields of interest are
	In final paper, “the information is clearer and more focused”.
	Data is shared long before the publication date.
	Going from the preprint to the final is a “shaving down process”.
	“I learned just how important details can be when publishing scientific research. Even the smallest things were commented on by the reviewers and would prevent the piece from being published until corrected.”

Student responses regarding what they learned about the process of peer review, writing journal articles, presenting data, and interpreting research data are presented (second column) along with categorization of those responses (first column). Student responses are summarized except where direct quotes are indicated.

Role of the preprint and final paper analysis in helping students to understand scientific communication

To assess the effect of this exercise on students’ understanding of scientific communication, students were asked to respond to several questions in part 3 of the assignment: (1) Do you feel that the level of attention that this non-peer reviewed article received was appropriate? (2) Do you think that preprint servers are a good way to get results out in a hurry? Should we maintain this process, or are there changes that should be implemented? (3) What, if anything, did you learn about the scientific publishing process from this activity?

In terms of whether the level of attention that the paper received was appropriate, whether the process should be maintained, and if any changes should be implemented, students pointed out several reasons why their research articles should have received a good level of attention (Fig 2). They felt that preprint servers are a good way to get "raw, unfiltered data out in a hurry". This was especially true for research with strong public health implications:

Fig 2

Student views of preprint articles and what changes could be implemented in the preprint process.

Student responses to the worksheet questions of whether “preprint servers are a good way to get results out in a hurry”, whether “the [preprint] publishing process … achieved an appropriate balance between getting research results out quickly … versus the need for high-quality, complete, and reproducible research?” and whether “we [should] maintain this process, or are there changes that should be implemented?” Representative codes resulting from qualitative analysis are depicted.

I think preprints are a good idea to some extent. Waiting for all the data to get published would take way too long, especially for topics that are greatly affecting the world like SARS-CoV-2. Having preprints allows for the information to be shared quicker and can help other researchers expand their knowledge of the topic to propel their research forward.

-female student, junior class

Students expressed that preprints brought attention to the research results, and that this could be beneficial both for papers in highly popular and impactful fields, like SARS-coV2 and COVID-19 currently, as well as for papers in more niche fields that would otherwise receive little notice. The many benefits to getting results out quickly was noted, including the ability of scientists to “mark their territory”, show how far their research has progressed, prevent other scientists from duplicating their research and wasting time, and providing a venue for scientists to gain pre-publication feedback (a "less condemning preview for authors to receive criticisms"). This was seen to be important for propelling research forward, and students recognized that the results could be useful for other scientists, even if the data were not perfect. These imperfections were noted to be tempered by the fact that the work is mostly viewed by professionals; because the majority of papers don’t grab the attention of general public, students expressed that scientists understand the stage of development of preprint papers and can interpret the results in light of the fact that they are not yet peer-reviewed. The fact that there were few differences between the preprint and final versions of the article reassured students that the preprint versions were of high quality. Some also noted that it was the responsibility of the authors to submit high-quality work ("ability to produce good data is dependent on the researchers") and to understand the nature of preprints when using the data ("Most people could reason and understand that a paper still under review isn’t the best evidence to act upon"). Students also expressed clear, although differing, criteria for whether preprints should get a high level of coverage; these criteria included the quality of the work, if it was peer-reviewed, and if it had public health relevance.

Students were asked to reflect on whether their preprint article received an appropriate level of attention by examining the metrics on bioRxiv, which consist of the number of times the HTML and PDF versions were downloaded and how many times the preprint was tweeted or read on Mendeley. Although they differed in their degree of agreement, most students (20 of24) felt that the level of attention generated by the preprint articles was appropriate (Fig 2), with 3 of these feeling that their preprint should have received more attention. In contrast, 4 students felt either moderately or quite strongly that the preprints received too much attention. These students were all concerned about potential errors in the research, with one student saying about the article they reviewed that it was “lucky the research was well done". Five students indicated that preprints were more likely to have incorrect information compared to peer-reviewed articles. This could "bring[s] a lot the false information to the real world" and "may be a source for providing misinformation". The 4 students with the strongest objections, as well as some of the students who still felt the level of attention was appropriate, were worried about future research being planned based on research results disseminated through preprint articles. Numerous students suggested some modifications that could improve the process (Fig 2). One suggestion was that preprints should not be cited by other research articles and that this standard could curtail new research being based on potentially faulty preprint data.

Students also expressed concerns about the nature of preprint articles being properly understood. Responses indicated that while students understood the value to sharing results prior to publication, they also recognized that one cannot assume that all data is of equal quality (i.e. that all data is “correct”). Some expressed concerns that the public might not understand the review process that the paper has or has not been through and suggested safeguards such as the addition of a disclaimer that the article is still under review so that the information could be “taken with a grain of salt” or a delay before publishing the preprint version on the online server. Several made distinctions between different types of reader and expressed less concern if the preprint was read by scientists than if it was read by the general public or media. In part 3 of the student worksheets, we asked students to note the preprint’s metrics including how many times it was downloaded or tweeted; a few students referred to the gatekeeping function of media and highlighted that the tweets referencing the preprint came from media accounts. There was concern that "the media often doesn’t understand the process" and one student even worried that the media doesn’t want to "note the caveats about the data which often steals the thunder they are trying to harness". The most sophisticated students therefore noted the role of the media as an interpreter of scientific information, although one with its own priorities and biases.

Finally, we asked students to reflect on what, if anything, they learned about the scientific publishing process from this activity (Fig 3). Students learned about the mechanics of the scientific publishing process that they were previously unaware of, including how long the processes of conducting research and publishing take. One commented that they learned:

Fig 3

Student learning gains resulting from the preprint analysis activity.

Student responses about “what, if anything, [they learned] about presenting and interpreting research data” and “about the scientific publishing process” were analyzed and representative first-level (bullet points) and second-level (title) codes are shown.

how much detail goes into publishing papers. In science classes…our extent of learning goes into one lab class where we finish our experiment. Researchers spend months and years trying to gather information and create new discoveries to benefit their respective communities.

-female student, senior class

It was noted that the revision process involves both large changes, such as the addition of new experiments and ideas, as well as much smaller edits. In fact, “some of the most significant changes do not need to be huge”, and “these changes are not apparent at first glance but can have a large impact on the paper.” Most importantly, students seemed to have learned the role and importance of peer review. They noted that the process improved the quality of the finished product. “It took a whole year to perfect this paper which really shows they care about perfecting their research.” One student also commented on the high standards required for publication in a scientific journal: “I learned that the standard to publish in recognized journals is very high and this paper serves as a great example of how minor changes allow authors and researchers to meet this standard.” Students also appreciated the function of peer review in ensuring the clarity of the paper: “Publishing really makes an author focus on how the paper is received by the reader”. Impressively, one student made the incisive observation that “you will always have to interpret your data”, and others noted the role of peer review in ensuring that data is appropriately explained and interpreted.

Students’ insight into the publishing process also included their observations about the many individuals and interests that have a significant influence on the publishing process. They noted the use of multiple reviewers and the need to “cater to the requests of reviewers” through a process that was “never a straight path” and that involves “back and forth during the review process”. The competitive nature of the process was also commented upon: "What was most surprising to learn was the amount of influence that publishers and editors have over an article’s ability to be published", as well as the political nature of publishing and the need to “establish[ing] territory”. One of the articles analyzed involved human subjects; because the final version had fewer patient details than the preprint, one reader commented upon the ethics involved in publishing some types of research data.

The most interesting thing I think I learned from this was the aspect of ethics involved in releasing research papers. This study involve[d] individuals expressing alleles that resulted in a particular phenotype that resulted from a consanguineous family history. Releasing photos to illustrate these phenotypes (exposing the patient’s identity) and writing about the family history could present moral and ethical issues that I think a lot of people don’t consider when doing research, although the information conveyed in releasing the photos and family history is beneficial for the reader in understanding the study.

-male student, senior class

Collectively, the students therefore reflected on the mechanics of paper publishing and the scientific communication process, the importance of peer review for ensuring high-quality research, and the multiple competing interests involved in both publishing and interpreting research studies.

Discussion

Preprints are a relatively new entrant into the process of scientific publishing, yet their popularity has soared and has been further catalyzed by the COVID-19 pandemic [14]. Several excellent methods are widely used for teaching students the structure of scientific articles and ways to effectively read articles to maximize understanding [53]. We wanted to add the use of preprint articles into this pedagogical toolbox with the hypothesis that it would help students to be more careful readers of scientific literature and to develop a more sophisticated understanding of the scientific publishing process.

Preprint articles and the cognitive apprenticeship model of training

To help students with one of the more challenging aspects of reviewing scientific articles, critiquing research data, we used a scaffolded learning approach, employing pairs of preprint and final published articles as pedagogical tools (Fig 4). We first engaged students in a process of mock peer review in which we asked them to critique the results section of the preprint article (part 1 of the assignment). Consistent with the cognitive apprenticeship model, they then identified changes between the preprint and final versions, which were presumably the result of peer review (part 2 of the assignment). This active learning process modeled professional practice for students by revealing the hidden thinking of the peer reviewers. Even though students made a significant number of critiques (average of 2.04) and found changes incorporated during the peer review process (average of 3.38 changes), few of these suggested and actual modifications addressed the same content or weakness of the article (average of 0.08). By having students compare their critique to defects in the paper identified by professional scientists, students revealed for themselves the differences in how professionals approach the peer review task, consistent with a cognitive apprenticeship approach [29,54,55]. The final stage of cognitive apprenticeship, coaching, was then provided when students engaged in a guided reflection comparing their work to the professionals through both a reflective worksheet (part 3 of the assignment) and through class discussions, as discussed below.

Fig 4

Scaffolded learning using preprint articles.

The benefits of class discussion on this activity

This exercise involved three distinct student activities that were performed sequentially. Following submission of each part of the assignment, the class discussed their results and their impressions, and class discussion is central to this activity for several reasons. First, class discussion is central to ensuring that students properly understand the mechanisms of the peer review process so that misconceptions do not persist. For example, from one student’s assignment response, it is unclear whether they understand that there is not a universal time schedule for the peer review process:

Given that the publication of this paper was completed in under a year, I believe the process of which the paper was published should be maintained if this process can produce accuracy uniformly throughout. What I mean here is that the process for this paper worked, but other studies may not be able to confirm to this time schedule of under a year. This could be troublesome for studies that, again, cannot fill this time crunch and thereby produce results that are not accurate representations of the experiments ran.

-male student, senior class

The second reason that class discussion is so valuable is that student responses varied so dramatically during this activity. The range of opinions about whether preprints were a positive development in the publishing process was quite broad, even among students in one class who all read the same paper. Most students had a moderately positive view of preprints, but others were either very strongly in favor or opposed. Most commonly, students agreed that there was a balance between the need to disseminate research results and the importance of ensuring that the work is of the high quality, but they differed in the relative importance that they assigned to these values, and this can make for a valuable classroom debate. Finally, class discussion is valuable given the wide array of different things that students inferred about the processes of scientific research, data presentation, and the process of scientific publishing and communication (Table 2). Many students had sophisticated insights, including the valuable technique of skipping between sections of a paper when reading, the importance of writing for the reader, that scientists use multiple methods or experimental techniques to bolster a claim, the fact that the meaning of data is not self-evident but must always be interpreted, and that judgment is required in this process. Taken together, these ideas make for a very rich discussion and help students to better understand and appreciate a broad range of concepts related to data presentation and scientific communication.

The effect of paper selection on this activity

This exercise was conducted at three different universities involving a total of 10 different research articles. Several aspects of papers varied and may have influenced the students’ feelings about the use of preprint articles. These included the age of the paper, the number and magnitude of changes that occurred between the preprint and final versions of the paper, and whether the topic involved highly specialized basic research or involved applied research with broad interest, such as work on SARS-CoV2 and COVID-19. Although the total number of students was small, a trend was observed that students who read papers that were older, that had fewer changes, or that involved highly specialized research tended to have a more positive view of preprint articles while those who read papers that were newer, had larger or more numerous changes, or that had the potential for a larger impact were more likely to express concerns about the use of preprints. In some cases, they recognized the importance of preprints, but still expressed concerns about the potential dissemination of data that was “incorrect”, perhaps because the results lacked proper controls or required additional experiments. For others, there was concern that the claims made by the paper might be unfounded or might extend beyond what is fair to conclude based on the data. Selection of the paper assigned to students and the degree of change between the preprint and final versions can therefore dramatically influence their first exposure to preprint articles and the peer review process. In one implementation, three students engaging in an independent research course each used a different pair of preprint and final articles. These three articles differed in the number and magnitude of changes between the preprint and final versions; therefore, this led to a robust conversation following each portion of the assignment, particularly after part 2 when students enumerated the changes between the preprint and final versions. A student’s initial conclusions based on their assigned paper can therefore be challenged when they hear the results from a different paper which may have gone through a more or less extensive revision process. Assigning more than one paper per class or group can therefore give students a broader and more nuanced understanding of preprints, peer review, and scientific publishing.

Educating students about the process of science and public communication of science

Science must increasingly cope with the spread of misleading and biased information in what some call a “post-truth” society [56]. Misunderstanding of the process of science and the appropriate role of uncertainty in the scientific process can exacerbate this problem [57]. Key factors in Americans’ level of trust in science are their own knowledge level and the sources from which they obtain information, although partisan affiliation also has a strong effect on these results [58]. It is therefore imperative that students have a clear understanding of the scientific research process both so that they can properly interpret scientific information and so that they can act as conduits of scientific information and understanding to their friends, families, and communities.

In this activity, students learned about diverse aspects of the scientific publishing process, including the nature of peer review and its importance for strengthening the quality of the final published paper (Fig 3). Students also gained a sophisticated understanding of the role of preprint articles and recognized that they could be important drivers of the public conversation around science and medicine [59,60]. They were further able to pick up on the value of preprints in bringing attention both to the preprint itself and to the peer-reviewed article when published [61]. Students also recognized that scientists will be more sophisticated in their understanding of preprints and that these articles need to be taken in context, while media and the general public may have a harder time making this distinction [62].

Impressively, students echoed several of the concerns voiced by scientific researchers about preprint articles including the accelerated pace of peer review during the current pandemic which can imperil quality control [14,63]. Indeed, much like professional scientists, many students pushed back against the demand for faster publication, stressing the need to maintain quality and research integrity [64]. They recognized the increased likelihood that topical research will impact clinical care, and made clear distinctions as to the qualities that should promote publicity for preprint articles, singling out the quality of the research and the public health relevance. Finally, several students made sophisticated arguments about the potential of preprint articles to promote misinformation, an argument that has been echoed by researchers and that is crucial given that 17–30% of SARS-CoV2/COVID-19 papers published in 2020 were preprints [14,65-67]. This exercise therefore had the unexpected and added benefit of helping students to understand the importance of the scientific publication process in conveying accurate information to the public [68] and in preventing and challenging public misconceptions [69], both of which are crucial to the continued development of a scientifically literate society.

Student self-guided analysis of the introduction, results, and discussion sections of their preprint article.

(PDF)

Click here for additional data file.

Student worksheet for comparison of the preprint and final published version of the article.

(PDF)

Click here for additional data file.

Student worksheet for reflection on the learning experience.

(PDF)

Click here for additional data file.

Example of completed student worksheet (S1 File) for self-guided analysis of the introduction, results, and discussion sections of their preprint article.

(PDF)

Click here for additional data file.

Example of completed student worksheet (S2 File) for comparison of the preprint and final published version of the article.

(PDF)

Click here for additional data file.

23 Jul 2021

PONE-D-21-18261

PREPRINT ARTICLES AS A TOOL FOR TEACHING DATA ANALYSIS AND SCIENTIFIC COMMUNICATION

PLOS ONE

Dear Dr. Scheifele,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 06 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Or Kan Soh

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Dear Author(s)

I am pleased to inform you that the reviewers have come to the conclusions for minor corrections to your manuscript. Please refer to their comments for further action.

Thank you.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

Reviewer #3: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Preprint articles as a tool for teaching data analysis and scientific communication

In the present article, the authors developed a new method in which students use a cognitive apprenticeship model to uncover how experts analyzed a paper and compare the professional’s cognitive approach to their own. In my opinion, the study is interesting and innovative, including was well delineated.

Reviewer #2: Dear authors, thank you for your collaboration

A very interesting work on teaching strategy.

I would consider making it less extensive in order that readers can locate themselves in the strategy used and the qualitative form that you have used to evaluate the effectiveness of this strategy in Higher Education Teaching.

Reviewer #3: Congratulations on building this model to develop student’s skills in peer review by engaging in realistic, real-work tasks conducted on 28 students from three institutions during the Fall 2020 semester using the cognitive apprenticeship approach.

It is encouraging to observe the teaching of this approach from the earliest stages of university education, which I believe will go a long way to improve peer review and interpreting research data.

I propose to the authors the following questions and suggestions, which might help to clarify some of these concerns:

- Could you describe the methodological design you used?

- Did any students refuse to participate, or did they not complete the entire assessment?

- Regarding the qualitative results, precisely the verbatims, could you provide information about the participant, at least age and gender?

A minor language revision is necessary to improve the manuscript understandable. Below are some grammatical errors to be corrected:

Introduction

Line 25 science to scientific

Line 36 step to steps

Line 37 peer reviewed to peer-reviewed

Line 62 Central to this process are the 5 to Central to this process is the 5

Line 68 critique by having students review articles themselves to critiques by having students review articles themselves

Line 72 difference between their assessment and the peer reviewer’s. to differences between their assessment and the peer reviewers.

Discussion

Line 477 judgement to judgment

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Tarek Mohamed Abd El-Aziz

Reviewer #2: No

Reviewer #3: Yes: Ian Blanco-Mavillard

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

24 Aug 2021

Dear Dr. Soh:

Thank you for your assistance in seeing our manuscript through the peer review process. We appreciate the careful review by the reviewers, their overall strongly positive assessment of our work, and their view that only minor corrections to our manuscript are needed. We have addressed all the reviewers’ requests and include a point-by-point response below.

1. Reviewer #2: I would consider making it less extensive in order that readers can locate themselves in the strategy used and the qualitative form that you have used to evaluate the effectiveness of this strategy in Higher Education Teaching.

While we agree with reviewer 2’s statement that it is necessary for readers to be able to see how they could use or implement this pedagogical strategy, we believe that shortening the manuscript and making it less extensive would hinder that process rather than enabling it. Because our pedagogical technique can be implemented in different classes and contexts, we felt that it is necessary to include sufficient details about each of the different instances of the technique being used that a diverse range of faculty from different institutional types can best use this teaching strategy.

2. Reviewer #3: Could you describe the methodological design you used?

We appreciate the reviewers’ request here and have included a better explanation that we believe improves the revised manuscript. Specifically, we described how we designed our study methods correspond to three steps of the cognitive apprenticeship approach (lines 113-118 in the revised manuscript with tracked changes), the learning model on which our pedagogical approach is based.

3. Reviewer #3: Did any students refuse to participate, or did they not complete the entire assessment?

No students refused to participate, but this exercise was one of two options that students could choose from in one of the participating courses; we have clarified in the manuscript that 6 of the 24 enrolled students chose to participate in this study (lines 98-99 in the revised manuscript with tracked changes). Two students did not complete the entire assignment, and we have now clarified that student work was not included unless all three parts were completed (lines 101-102 in the revised manuscript with tracked changes).

4. Reviewer #3: Regarding the qualitative results, precisely the verbatims, could you provide information about the participant, at least age and gender?

We appreciate the reviewer’s interest in characteristics of our study participants. We describe the gender and class year of all our participants in the methods section. To respond more precisely to this request while still protecting our students’ privacy, we have included the students’ gender and class year for each of the significant direct quotes, and this information can now be found on lines 257, 281, 323, 391, 429, 480 of the revised manuscript with tracked changes.

5. Reviewer 3: A minor language revision is necessary to improve the manuscript understandable. Below are some grammatical errors to be corrected:

Introduction

Line 25 science to scientific

This correction has been made.

Line 36 step to steps

We believe that this is grammatically correct as written: “publication on a preprint server is

increasingly becoming the first step of publishing a research article”.

Line 37 peer reviewed to peer-reviewed

This correction has been made.

Line 62 “Central to this process are the” to “Central to this process is the”

We believe that this is correct as written; the subject of the sentence is “the principles” and therefore the verb should be plural “are”.

Line 68 critique by having students review articles themselves to critiques by having students review articles themselves

This correction has been made.

Line 72 difference between their assessment and the peer reviewer’s. to differences between their assessment and the peer reviewers.

This correction has been made.

Discussion

Line 477 judgement to judgment

This correction has been made.

Thank you again for your help shepherding our manuscript. We hope that these changes have fully addressed the reviewers’ concerns, but if not, we would be happy to make additional revisions as necessary.

Sincerely,

Lisa Scheifele

Associate Professor and Chair

Submitted filename: Response to Reviewers.pdf

Click here for additional data file.

7 Dec 2021

PREPRINT ARTICLES AS A TOOL FOR TEACHING DATA ANALYSIS AND SCIENTIFIC COMMUNICATION

PONE-D-21-18261R1

Dear Dr. Scheifele,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Muhammad Shahzad Aslam, Ph.D.,M.Phil., Pharm-D

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have been addressed all the comments. I recommend to accept the manuscript in the present version.

Reviewer #2: Dear authors, thank you for your contribution. A very good teaching-learning exercise in sup education

Reviewer #3: The authors correctly respond to the questions of both reviewers. For my part, there are no further methodological concerns or limitations. Congratulations to the authors!

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Ian Blanco-Mavillard

13 Dec 2021

PONE-D-21-18261R1

Preprint articles as a tool for teaching data analysis and scientific communication

Dear Dr. Scheifele:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Muhammad Shahzad Aslam

Academic Editor

PLOS ONE